Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.archive.crawler.deciderules
Class SurtPrefixedDecideRule

    java.lang.Object
      javax.management.Attribute
          org.archive.crawler.settings.Type
              org.archive.crawler.settings.ComplexType
                  org.archive.crawler.settings.ModuleType
                      org.archive.crawler.deciderules.DecideRule
                          org.archive.crawler.deciderules.ConfiguredDecideRule
                              org.archive.crawler.deciderules.PredicatedDecideRule
                                  org.archive.crawler.deciderules.SurtPrefixedDecideRule

All Implemented Interfaces:
  java.io.Serializable, javax.management.DynamicMBean, SeedListener

Direct Known Subclasses:
  NotSurtPrefixedDecideRule, OnDomainsDecideRule, OnHostsDecideRule, ScopePlusOneDecideRule

    public class SurtPrefixedDecideRule

extends PredicatedDecideRule
implements SeedListener

Rule applies configured decision to any URIs that, when expressed in SURT form, begin with one of the prefixes in the configured set. The set can be filled with SURT prefixes implied or listed in the seeds file, or another external file. The "also-check-via" option to implement "one hop off" scoping derives from a contribution by Shifra Raffel of the California Digital Library.

Author:
  gojomo
See Also:
  Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.archive.crawler.settings.ComplexType
`ComplexType.MBeanAttributeInfoIterator`

Field Summary
`static java.lang.String` | `ATTR_ALSO_CHECK_VIA` Whether the 'via' of CrawlURIs should also be checked to see if it is prefixed by the set of SURT prefixes
`static java.lang.String` | `ATTR_REBUILD_ON_RECONFIG` Whether every config change should trigger a rebuilding of the prefix set.
`static java.lang.String` | `ATTR_SEEDS_AS_SURT_PREFIXES`
`static java.lang.String` | `ATTR_SURTS_DUMP_FILE`
`static java.lang.String` | `ATTR_SURTS_SOURCE_FILE`
`static java.lang.Boolean` | `DEFAULT_ALSO_CHECK_VIA`
`static java.lang.Boolean` | `DEFAULT_REBUILD_ON_RECONFIG`
`protected SurtPrefixSet` | `surtPrefixes`

Fields inherited from class org.archive.crawler.deciderules.ConfiguredDecideRule
`ALLOWED_TYPES, ATTR_DECISION`

Fields inherited from class org.archive.crawler.deciderules.DecideRule
`ACCEPT, PASS, REJECT`

Fields inherited from class org.archive.crawler.settings.ComplexType
`definition, definitionMap`

Constructor Summary
`SurtPrefixedDecideRule(java.lang.String name)` Usual constructor.

Method Summary
`void` | `addedSeed(CandidateURI curi)`
`protected void` | `buildSurtPrefixSet()` Construct the set of prefixes to use, from the seed list ( which may include both URIs and '+'-prefixed directives).
`protected void` | `dumpSurtPrefixSet()` Dump the current prefixes in use to configured dump file (if any)
`protected boolean` | `evaluate(java.lang.Object object)` Evaluate whether given object's URI is covered by the SURT prefix set
`protected java.io.File` | `getSeedfile()` Dig through everything to get the crawl-global seeds file.
`void` | `kickUpdate()` Re-read prefixes after an update.
`protected java.lang.String` | `prefixFrom(java.lang.String uri)`
`protected void` | `readPrefixes()`

Methods inherited from class org.archive.crawler.deciderules.PredicatedDecideRule
`decisionFor`

Methods inherited from class org.archive.crawler.deciderules.ConfiguredDecideRule
`singlePossibleNonPassDecision`

Methods inherited from class org.archive.crawler.deciderules.DecideRule
`getController`

Methods inherited from class org.archive.crawler.settings.ModuleType
`addElement, listUsedFiles`

Methods inherited from class org.archive.crawler.settings.ComplexType
`addElementToDefinition, checkValue, earlyInitialize, getAbsoluteName, getAttribute, getAttribute, getAttribute, getAttributeInfo, getAttributeInfo, getAttributeInfoIterator, getAttributes, getDataContainerRecursive, getDataContainerRecursive, getDefaultValue, getDescription, getElementFromDefinition, getLegalValues, getLocalAttribute, getMBeanInfo, getMBeanInfo, getParent, getPreservedFields, getSettingsHandler, getUncheckedAttribute, getValue, globalSettings, invoke, isInitialized, isOverridden, iterator, removeElementFromDefinition, setAsOrder, setAttribute, setAttribute, setAttributes, setDescription, setPreservedFields, toString, unsetAttribute`

Methods inherited from class org.archive.crawler.settings.Type
`addConstraint, equals, getConstraints, getLegalValueType, isExpertSetting, isOverrideable, isTransient, setExpertSetting, setLegalValueType, setOverrideable, setTransient`

Methods inherited from class javax.management.Attribute
`getName`

Methods inherited from class java.lang.Object
`clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

### ATTR_SURTS_SOURCE_FILE

    public static final java.lang.String ATTR_SURTS_SOURCE_FILE

See Also:
  Constant Field Values

### ATTR_SEEDS_AS_SURT_PREFIXES

    public static final java.lang.String ATTR_SEEDS_AS_SURT_PREFIXES

See Also:
  Constant Field Values

### ATTR_SURTS_DUMP_FILE

    public static final java.lang.String ATTR_SURTS_DUMP_FILE

See Also:
  Constant Field Values

### ATTR_REBUILD_ON_RECONFIG

    public static final java.lang.String ATTR_REBUILD_ON_RECONFIG

  Whether every config change should trigger a rebuilding of the prefix set.

  See Also:
    Constant Field Values

### DEFAULT_REBUILD_ON_RECONFIG

    public static final java.lang.Boolean DEFAULT_REBUILD_ON_RECONFIG

### ATTR_ALSO_CHECK_VIA

    public static final java.lang.String ATTR_ALSO_CHECK_VIA

  Whether the 'via' of CrawlURIs should also be checked to see if it is prefixed by the set of SURT prefixes

  See Also:
    Constant Field Values

### DEFAULT_ALSO_CHECK_VIA

    public static final java.lang.Boolean DEFAULT_ALSO_CHECK_VIA

### surtPrefixes

    protected SurtPrefixSet surtPrefixes

Constructor Detail

### SurtPrefixedDecideRule

    public SurtPrefixedDecideRule(java.lang.String name)

  Usual constructor.

  Parameters:
    `name` -

Method Detail

### evaluate

    protected boolean evaluate(java.lang.Object object)

  Evaluate whether given object's URI is covered by the SURT prefix set

  Specified by:
    `evaluate` in class `PredicatedDecideRule`
  Parameters:
    `object` - Item to evaluate.
  Returns:
    true if item, as SURT form URI, is prefixed by an item in the set

### readPrefixes

    protected void readPrefixes()

### dumpSurtPrefixSet

    protected void dumpSurtPrefixSet()

  Dump the current prefixes in use to configured dump file (if any)

### buildSurtPrefixSet

    protected void buildSurtPrefixSet()

  Construct the set of prefixes to use, from the seed list ( which may include both URIs and '+'-prefixed directives).

### kickUpdate

    public void kickUpdate()

  Re-read prefixes after an update.

  Overrides:
    `kickUpdate` in class `DecideRule`
  See Also:
    `CrawlScope.kickUpdate()`

### getSeedfile

    protected java.io.File getSeedfile()

  Dig through everything to get the crawl-global seeds file. Add self as listener while at it.

  Returns:
    Seed list file

### addedSeed

    public void addedSeed(CandidateURI curi)

  Specified by:
    `addedSeed` in interface `SeedListener`

### prefixFrom

    protected java.lang.String prefixFrom(java.lang.String uri)

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2003-2008 Internet Archive. All Rights Reserved.