Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.archive.crawler.postprocessor
Class WaitEvaluator

    java.lang.Object
      javax.management.Attribute
          org.archive.crawler.settings.Type
              org.archive.crawler.settings.ComplexType
                  org.archive.crawler.settings.ModuleType
                      org.archive.crawler.framework.Processor
                          org.archive.crawler.postprocessor.WaitEvaluator

All Implemented Interfaces:
  java.io.Serializable, javax.management.DynamicMBean, CoreAttributeConstants, AdaptiveRevisitAttributeConstants

Direct Known Subclasses:
  ContentBasedWaitEvaluator

    public class WaitEvaluator

extends Processor
implements AdaptiveRevisitAttributeConstants

A processor that determines when a URI should be revisited next. Does not account for DNS and robots.txt expiration. That should be handled seperately by the Frontiers.

Author:
  Kristinn Sigurdsson
See Also:
  Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.archive.crawler.settings.ComplexType
`ComplexType.MBeanAttributeInfoIterator`

Field Summary
`static java.lang.String` | `ATTR_CHANGED_FACTOR` Factor decrease on wait when changed
`static java.lang.String` | `ATTR_DEFAULT_WAIT_INTERVAL` Fixed wait time for 'unknown' change status.
`static java.lang.String` | `ATTR_INITIAL_WAIT_INTERVAL` Default wait time after initial visit.
`static java.lang.String` | `ATTR_MAX_WAIT_INTERVAL` Maximum wait between visits
`static java.lang.String` | `ATTR_MIN_WAIT_INTERVAL` Minimum wait between visits
`static java.lang.String` | `ATTR_UNCHANGED_FACTOR` Factor increase on wait when unchanged
`static java.lang.String` | `ATTR_USE_OVERDUE_TIME` Indicates if the amount of time the URI was overdue should be added to the wait time before the new wait time is calculated.
`protected static java.lang.Double` | `DEFAULT_CHANGED_FACTOR`
`protected static java.lang.Long` | `DEFAULT_DEFAULT_WAIT_INTERVAL`
`protected static java.lang.Long` | `DEFAULT_INITIAL_WAIT_INTERVAL`
`protected static java.lang.Long` | `DEFAULT_MAX_WAIT_INTERVAL`
`protected static java.lang.Long` | `DEFAULT_MIN_WAIT_INTERVAL`
`protected static java.lang.Double` | `DEFAULT_UNCHANGED_FACTOR`
`protected static java.lang.Boolean` | `DEFAULT_USE_OVERDUE_TIME`
`(package private) java.util.logging.Logger` | `logger`

Fields inherited from class org.archive.crawler.framework.Processor
`ATTR_DECIDE_RULES, ATTR_ENABLED, attrDecideRules`

Fields inherited from class org.archive.crawler.settings.ComplexType
`definition, definitionMap`

Fields inherited from interface org.archive.crawler.frontier.AdaptiveRevisitAttributeConstants
`A_CONTENT_STATE_KEY, A_DISCARD_REVISIT, A_FETCH_OVERDUE, A_LAST_CONTENT_DIGEST, A_LAST_DATESTAMP, A_LAST_ETAG, A_NUMBER_OF_VERSIONS, A_NUMBER_OF_VISITS, A_TIME_OF_NEXT_PROCESSING, A_WAIT_INTERVAL, A_WAIT_REEVALUATED, CONTENT_CHANGED, CONTENT_UNCHANGED, CONTENT_UNKNOWN`

Fields inherited from interface org.archive.crawler.datamodel.CoreAttributeConstants
`A_ANNOTATIONS, A_CONTENT_DIGEST, A_CONTENT_TYPE, A_CREDENTIAL_AVATARS_KEY, A_DELAY_FACTOR, A_DISTANCE_FROM_SEED, A_DNS_FETCH_TIME, A_DNS_SERVER_IP_LABEL, A_ETAG_HEADER, A_FETCH_BEGAN_TIME, A_FETCH_COMPLETED_TIME, A_FETCH_HISTORY, A_FORCE_RETIRE, A_HERITABLE_KEYS, A_HTML_BASE, A_HTTP_BIND_ADDRESS, A_HTTP_PROXY_HOST, A_HTTP_PROXY_PORT, A_HTTP_TRANSACTION, A_LAST_MODIFIED_HEADER, A_LOCALIZED_ERRORS, A_META_ROBOTS, A_MINIMUM_DELAY, A_MIRROR_PATH, A_PREREQUISITE_URI, A_REFERENCE_LENGTH, A_RETRY_DELAY, A_RRECORD_SET_LABEL, A_RUNTIME_EXCEPTION, A_SOURCE_TAG, A_STATUS, HEADER_TRUNC, LENGTH_TRUNC, TIMER_TRUNC, TRUNC_SUFFIX`

Constructor Summary
`WaitEvaluator(java.lang.String name)` Constructor
`WaitEvaluator(java.lang.String name, java.lang.String description, java.lang.Long default_inital_wait_interval, java.lang.Long default_max_wait_interval, java.lang.Long default_min_wait_interval, java.lang.Double default_unchanged_factor, java.lang.Double default_changed_factor)` Constructor

Method Summary
`protected void` | `innerProcess(CrawlURI curi)` Classes subclassing this one should override this method to perform their custom actions on the CrawlURI.

Methods inherited from class org.archive.crawler.framework.Processor
`checkForInterrupt, finalTasks, getController, getDecideRule, getDefaultNextProcessor, initialTasks, innerRejectProcess, isContentToProcess, isExpectedMimeType, isHttpTransactionContentToProcess, kickUpdate, process, report, rulesAccept, rulesAccept, setDefaultNextProcessor, spawn`

Methods inherited from class org.archive.crawler.settings.ModuleType
`addElement, listUsedFiles`

Methods inherited from class org.archive.crawler.settings.ComplexType
`addElementToDefinition, checkValue, earlyInitialize, getAbsoluteName, getAttribute, getAttribute, getAttribute, getAttributeInfo, getAttributeInfo, getAttributeInfoIterator, getAttributes, getDataContainerRecursive, getDataContainerRecursive, getDefaultValue, getDescription, getElementFromDefinition, getLegalValues, getLocalAttribute, getMBeanInfo, getMBeanInfo, getParent, getPreservedFields, getSettingsHandler, getUncheckedAttribute, getValue, globalSettings, invoke, isInitialized, isOverridden, iterator, removeElementFromDefinition, setAsOrder, setAttribute, setAttribute, setAttributes, setDescription, setPreservedFields, toString, unsetAttribute`

Methods inherited from class org.archive.crawler.settings.Type
`addConstraint, equals, getConstraints, getLegalValueType, isExpertSetting, isOverrideable, isTransient, setExpertSetting, setLegalValueType, setOverrideable, setTransient`

Methods inherited from class javax.management.Attribute
`getName`

Methods inherited from class java.lang.Object
`clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

### logger

    java.util.logging.Logger logger

### ATTR_INITIAL_WAIT_INTERVAL

    public static final java.lang.String ATTR_INITIAL_WAIT_INTERVAL

  Default wait time after initial visit.

  See Also:
    Constant Field Values

### DEFAULT_INITIAL_WAIT_INTERVAL

    protected static final java.lang.Long DEFAULT_INITIAL_WAIT_INTERVAL

### ATTR_MAX_WAIT_INTERVAL

    public static final java.lang.String ATTR_MAX_WAIT_INTERVAL

  Maximum wait between visits

  See Also:
    Constant Field Values

### DEFAULT_MAX_WAIT_INTERVAL

    protected static final java.lang.Long DEFAULT_MAX_WAIT_INTERVAL

### ATTR_MIN_WAIT_INTERVAL

    public static final java.lang.String ATTR_MIN_WAIT_INTERVAL

  Minimum wait between visits

  See Also:
    Constant Field Values

### DEFAULT_MIN_WAIT_INTERVAL

    protected static final java.lang.Long DEFAULT_MIN_WAIT_INTERVAL

### ATTR_UNCHANGED_FACTOR

    public static final java.lang.String ATTR_UNCHANGED_FACTOR

  Factor increase on wait when unchanged

  See Also:
    Constant Field Values

### DEFAULT_UNCHANGED_FACTOR

    protected static final java.lang.Double DEFAULT_UNCHANGED_FACTOR

### ATTR_CHANGED_FACTOR

    public static final java.lang.String ATTR_CHANGED_FACTOR

  Factor decrease on wait when changed

  See Also:
    Constant Field Values

### DEFAULT_CHANGED_FACTOR

    protected static final java.lang.Double DEFAULT_CHANGED_FACTOR

### ATTR_DEFAULT_WAIT_INTERVAL

    public static final java.lang.String ATTR_DEFAULT_WAIT_INTERVAL

  Fixed wait time for 'unknown' change status. I.e. wait time for URIs whose content change detection is not available.

  See Also:
    Constant Field Values

### DEFAULT_DEFAULT_WAIT_INTERVAL

    protected static final java.lang.Long DEFAULT_DEFAULT_WAIT_INTERVAL

### ATTR_USE_OVERDUE_TIME

    public static final java.lang.String ATTR_USE_OVERDUE_TIME

  Indicates if the amount of time the URI was overdue should be added to the wait time before the new wait time is calculated.

  See Also:
    Constant Field Values

### DEFAULT_USE_OVERDUE_TIME

    protected static final java.lang.Boolean DEFAULT_USE_OVERDUE_TIME

Constructor Detail

### WaitEvaluator

    public WaitEvaluator(java.lang.String name)

  Constructor

  Parameters:
    `name` - The name of the module

### WaitEvaluator

    public WaitEvaluator(java.lang.String name,
                         java.lang.String description,
                         java.lang.Long default_inital_wait_interval,
                         java.lang.Long default_max_wait_interval,
                         java.lang.Long default_min_wait_interval,
                         java.lang.Double default_unchanged_factor,
                         java.lang.Double default_changed_factor)

  Constructor

  Parameters:
    `name` - The name of the module
    `description` - Description of the module
    `default_inital_wait_interval` - The default value for initial wait time
    `default_max_wait_interval` - The maximum value for wait time
    `default_min_wait_interval` - The minimum value for wait time
    `default_unchanged_factor` - The factor for changing wait times of unchanged documents (will be multiplied by this value)
    `default_changed_factor` - The factor for changing wait times of changed documents (will be divided by this value)

Method Detail

### innerProcess

    protected void innerProcess(CrawlURI curi)
                         throws java.lang.InterruptedException

  Description copied from class: `Processor`
  Classes subclassing this one should override this method to perform their custom actions on the CrawlURI.

  Overrides:
    `innerProcess` in class `Processor`
  Parameters:
    `curi` - The CrawlURI being processed.
  Throws:
    `java.lang.InterruptedException`

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2003-2008 Internet Archive. All Rights Reserved.