Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.archive.crawler.prefetch
Class Preselector

    java.lang.Object
      javax.management.Attribute
          org.archive.crawler.settings.Type
              org.archive.crawler.settings.ComplexType
                  org.archive.crawler.settings.ModuleType
                      org.archive.crawler.framework.Processor
                          org.archive.crawler.framework.Scoper
                              org.archive.crawler.prefetch.Preselector

All Implemented Interfaces:
  java.io.Serializable, javax.management.DynamicMBean, FetchStatusCodes

    public class Preselector

extends Scoper
implements FetchStatusCodes

If set to recheck the crawl's scope, gives a yes/no on whether a CrawlURI should be processed at all. If not, its status will be marked OUT_OF_SCOPE and the URI will skip directly to the first "postprocessor".

Author:
  gojomo
See Also:
  Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.archive.crawler.settings.ComplexType
`ComplexType.MBeanAttributeInfoIterator`

Field Summary
`static java.lang.String` | `ATTR_ALLOW_BY_REGEXP` indicator allowing all matching URIs
`static java.lang.String` | `ATTR_BLOCK_ALL` indicator allowing all URIs (of a given host, typically) to be blocked at this step
`static java.lang.String` | `ATTR_BLOCK_BY_REGEXP` indicator allowing all matching URIs to be blocked at this step
`static java.lang.String` | `ATTR_RECHECK_SCOPE` whether to reapply crawl scope at this step

Fields inherited from class org.archive.crawler.framework.Scoper
`ATTR_OVERRIDE_LOGGER_ENABLED`

Fields inherited from class org.archive.crawler.framework.Processor
`ATTR_DECIDE_RULES, ATTR_ENABLED, attrDecideRules`

Fields inherited from class org.archive.crawler.settings.ComplexType
`definition, definitionMap`

Fields inherited from interface org.archive.crawler.datamodel.FetchStatusCodes
`S_BLOCKED_BY_CUSTOM_PROCESSOR, S_BLOCKED_BY_QUOTA, S_BLOCKED_BY_RUNTIME_LIMIT, S_BLOCKED_BY_USER, S_CONNECT_FAILED, S_CONNECT_LOST, S_DEEMED_CHAFF, S_DEFERRED, S_DELETED_BY_USER, S_DNS_SUCCESS, S_DOMAIN_PREREQUISITE_FAILURE, S_DOMAIN_UNRESOLVABLE, S_GETBYNAME_SUCCESS, S_OTHER_PREREQUISITE_FAILURE, S_OUT_OF_SCOPE, S_PREREQUISITE_UNSCHEDULABLE_FAILURE, S_PROCESSING_THREAD_KILLED, S_ROBOTS_PRECLUDED, S_ROBOTS_PREREQUISITE_FAILURE, S_RUNTIME_EXCEPTION, S_SERIOUS_ERROR, S_TIMEOUT, S_TOO_MANY_EMBED_HOPS, S_TOO_MANY_LINK_HOPS, S_TOO_MANY_RETRIES, S_UNATTEMPTED, S_UNFETCHABLE_URI, S_UNQUEUEABLE`

Constructor Summary
`Preselector(java.lang.String name)` Constructor.

Method Summary
`protected void` | `innerProcess(CrawlURI curi)` Classes subclassing this one should override this method to perform their custom actions on the CrawlURI.

Methods inherited from class org.archive.crawler.framework.Scoper
`initialTasks, isInScope, isOverrideLogger, outOfScope`

Methods inherited from class org.archive.crawler.framework.Processor
`checkForInterrupt, finalTasks, getController, getDecideRule, getDefaultNextProcessor, innerRejectProcess, isContentToProcess, isExpectedMimeType, isHttpTransactionContentToProcess, kickUpdate, process, report, rulesAccept, rulesAccept, setDefaultNextProcessor, spawn`

Methods inherited from class org.archive.crawler.settings.ModuleType
`addElement, listUsedFiles`

Methods inherited from class org.archive.crawler.settings.ComplexType
`addElementToDefinition, checkValue, earlyInitialize, getAbsoluteName, getAttribute, getAttribute, getAttribute, getAttributeInfo, getAttributeInfo, getAttributeInfoIterator, getAttributes, getDataContainerRecursive, getDataContainerRecursive, getDefaultValue, getDescription, getElementFromDefinition, getLegalValues, getLocalAttribute, getMBeanInfo, getMBeanInfo, getParent, getPreservedFields, getSettingsHandler, getUncheckedAttribute, getValue, globalSettings, invoke, isInitialized, isOverridden, iterator, removeElementFromDefinition, setAsOrder, setAttribute, setAttribute, setAttributes, setDescription, setPreservedFields, toString, unsetAttribute`

Methods inherited from class org.archive.crawler.settings.Type
`addConstraint, equals, getConstraints, getLegalValueType, isExpertSetting, isOverrideable, isTransient, setExpertSetting, setLegalValueType, setOverrideable, setTransient`

Methods inherited from class javax.management.Attribute
`getName`

Methods inherited from class java.lang.Object
`clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

### ATTR_RECHECK_SCOPE

    public static final java.lang.String ATTR_RECHECK_SCOPE

  whether to reapply crawl scope at this step

  See Also:
    Constant Field Values

### ATTR_BLOCK_ALL

    public static final java.lang.String ATTR_BLOCK_ALL

  indicator allowing all URIs (of a given host, typically) to be blocked at this step

  See Also:
    Constant Field Values

### ATTR_BLOCK_BY_REGEXP

    public static final java.lang.String ATTR_BLOCK_BY_REGEXP

  indicator allowing all matching URIs to be blocked at this step

  See Also:
    Constant Field Values

### ATTR_ALLOW_BY_REGEXP

    public static final java.lang.String ATTR_ALLOW_BY_REGEXP

  indicator allowing all matching URIs

  See Also:
    Constant Field Values

Constructor Detail

### Preselector

    public Preselector(java.lang.String name)

  Constructor.

  Parameters:
    `name` - Name of this processor.

Method Detail

### innerProcess

    protected void innerProcess(CrawlURI curi)

  Description copied from class: `Processor`
  Classes subclassing this one should override this method to perform their custom actions on the CrawlURI.

  Overrides:
    `innerProcess` in class `Processor`
  Parameters:
    `curi` - The CrawlURI being processed.

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2003-2008 Internet Archive. All Rights Reserved.