Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.archive.crawler.frontier
Class BdbWorkQueue

    java.lang.Object
      org.archive.crawler.frontier.WorkQueue
          org.archive.crawler.frontier.BdbWorkQueue

All Implemented Interfaces:
  java.io.Serializable, java.lang.Comparable, CrawlSubstats.HasCrawlSubstats, Frontier.FrontierGroup, Reporter

    public class BdbWorkQueue

extends WorkQueue
implements java.lang.Comparable, java.io.Serializable

One independent queue of items with the same 'classKey' (eg host).

Author:
  gojomo
See Also:
  Serialized Form

Field Summary

Fields inherited from class org.archive.crawler.frontier.WorkQueue
`classKey, substats`

Constructor Summary
`BdbWorkQueue(java.lang.String classKey, BdbFrontier frontier)` Create a virtual queue inside the given BdbMultipleWorkQueues

Method Summary
`protected void` | `deleteItem(WorkQueueFrontier frontier, CrawlURI peekItem)` Removes the given item from the queue.
`protected long` | `deleteMatchingFromQueue(WorkQueueFrontier frontier, java.lang.String match)` Delete URIs matching the given pattern from this queue.
`protected static java.lang.String` | `getPrefixClassKey(byte[] byteArray)`
`protected void` | `insertItem(WorkQueueFrontier frontier, CrawlURI curi, boolean overwriteIfPresent)` Insert the given curi, whether it is already present or not.
`protected CrawlURI` | `peekItem(WorkQueueFrontier frontier)` Returns first item from queue (does not delete)

Methods inherited from class org.archive.crawler.frontier.WorkQueue
`clearHeld, compareTo, deleteMatching, dequeue, enqueue, expend, getClassKey, getContextUURI, getCount, getReports, getSessionBalance, getSubstats, getTotalExpenditure, getWakeTime, incrementSessionBalance, isHeld, isOverBudget, isRetired, noteError, peek, refund, reportTo, reportTo, resume, setActive, setHeld, setRetired, setSessionBalance, setTotalBudget, setWakeTime, singleLineLegend, singleLineReport, singleLineReportTo, suspend, unpeek, update`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Methods inherited from interface java.lang.Comparable
`compareTo`

Constructor Detail

### BdbWorkQueue

    public BdbWorkQueue(java.lang.String classKey,
                        BdbFrontier frontier)

  Create a virtual queue inside the given BdbMultipleWorkQueues

  Parameters:
    `classKey` -

Method Detail

### deleteMatchingFromQueue

    protected long deleteMatchingFromQueue(WorkQueueFrontier frontier,
                                           java.lang.String match)
                                    throws java.io.IOException

  Description copied from class: `WorkQueue`
  Delete URIs matching the given pattern from this queue.

  Specified by:
    `deleteMatchingFromQueue` in class `WorkQueue`
  Parameters:
    `frontier` - WorkQueues manager.
    `match` - the pattern to match
  Returns:
    count of deleted URIs
  Throws:
    `java.io.IOException` - if there was a problem while deleting

### deleteItem

    protected void deleteItem(WorkQueueFrontier frontier,
                              CrawlURI peekItem)
                       throws java.io.IOException

  Description copied from class: `WorkQueue`
  Removes the given item from the queue. This is only used to remove the first item in the queue, so it is not necessary to implement a random-access queue.

  Specified by:
    `deleteItem` in class `WorkQueue`
  Parameters:
    `frontier` - Work queues manager.
  Throws:
    `java.io.IOException` - if there was a problem while deleting the item

### peekItem

    protected CrawlURI peekItem(WorkQueueFrontier frontier)
                         throws java.io.IOException

  Description copied from class: `WorkQueue`
  Returns first item from queue (does not delete)

  Specified by:
    `peekItem` in class `WorkQueue`
  Returns:
    The peeked item, or null
  Throws:
    `java.io.IOException` - if there was a problem while peeking

### insertItem

    protected void insertItem(WorkQueueFrontier frontier,
                              CrawlURI curi,
                              boolean overwriteIfPresent)
                       throws java.io.IOException

  Description copied from class: `WorkQueue`
  Insert the given curi, whether it is already present or not. Hook for subclasses.

  Specified by:
    `insertItem` in class `WorkQueue`
  Parameters:
    `frontier` - WorkQueueFrontier.
    `curi` - CrawlURI to insert.
  Throws:
    `java.io.IOException` - if there was a problem while inserting the item

### getPrefixClassKey

    protected static java.lang.String getPrefixClassKey(byte[] byteArray)

  Parameters:
    `byteArray` - Byte array to get hex string of.
  Returns:
    Hex string of passed in byte array (Used logging key-prefixes).

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2003-2008 Internet Archive. All Rights Reserved.