Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.archive.crawler.postprocessor
Class LowDiskPauseProcessor

    java.lang.Object
      javax.management.Attribute
          org.archive.crawler.settings.Type
              org.archive.crawler.settings.ComplexType
                  org.archive.crawler.settings.ModuleType
                      org.archive.crawler.framework.Processor
                          org.archive.crawler.postprocessor.LowDiskPauseProcessor

All Implemented Interfaces:
  java.io.Serializable, javax.management.DynamicMBean

    public class LowDiskPauseProcessor

extends Processor

Processor module which uses 'df -k', where available and with the expected output format (on Linux), to monitor available disk space and pause the crawl if free space on monitored filesystems falls below certain thresholds.

See Also:
  Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.archive.crawler.settings.ComplexType
`ComplexType.MBeanAttributeInfoIterator`

Field Summary
`static java.lang.String` | `ATTR_MONITOR_MOUNTS` List of mounts to monitor; should match "Mounted on" column of 'df' output
`static java.lang.String` | `ATTR_PAUSE_THRESHOLD` Space available level below which a crawl-pause should be triggered.
`static java.lang.String` | `ATTR_RECHECK_THRESHOLD` Amount of content received between each recheck of free space
`static java.util.regex.Pattern` | `AVAILABLE_EXTRACTOR`
`protected int` | `contentSinceCheck`
`static java.lang.String` | `DEFAULT_MONITOR_MOUNTS`
`static int` | `DEFAULT_PAUSE_THRESHOLD`
`static int` | `DEFAULT_RECHECK_THRESHOLD`
`static java.util.regex.Pattern` | `VALID_DF_OUTPUT`

Fields inherited from class org.archive.crawler.framework.Processor
`ATTR_DECIDE_RULES, ATTR_ENABLED, attrDecideRules`

Fields inherited from class org.archive.crawler.settings.ComplexType
`definition, definitionMap`

Constructor Summary
`LowDiskPauseProcessor(java.lang.String name)`

Method Summary
`protected void` | `innerProcess(CrawlURI curi)` Notes a CrawlURI's content size in its running tally.

Methods inherited from class org.archive.crawler.framework.Processor
`checkForInterrupt, finalTasks, getController, getDecideRule, getDefaultNextProcessor, initialTasks, innerRejectProcess, isContentToProcess, isExpectedMimeType, isHttpTransactionContentToProcess, kickUpdate, process, report, rulesAccept, rulesAccept, setDefaultNextProcessor, spawn`

Methods inherited from class org.archive.crawler.settings.ModuleType
`addElement, listUsedFiles`

Methods inherited from class org.archive.crawler.settings.ComplexType
`addElementToDefinition, checkValue, earlyInitialize, getAbsoluteName, getAttribute, getAttribute, getAttribute, getAttributeInfo, getAttributeInfo, getAttributeInfoIterator, getAttributes, getDataContainerRecursive, getDataContainerRecursive, getDefaultValue, getDescription, getElementFromDefinition, getLegalValues, getLocalAttribute, getMBeanInfo, getMBeanInfo, getParent, getPreservedFields, getSettingsHandler, getUncheckedAttribute, getValue, globalSettings, invoke, isInitialized, isOverridden, iterator, removeElementFromDefinition, setAsOrder, setAttribute, setAttribute, setAttributes, setDescription, setPreservedFields, toString, unsetAttribute`

Methods inherited from class org.archive.crawler.settings.Type
`addConstraint, equals, getConstraints, getLegalValueType, isExpertSetting, isOverrideable, isTransient, setExpertSetting, setLegalValueType, setOverrideable, setTransient`

Methods inherited from class javax.management.Attribute
`getName`

Methods inherited from class java.lang.Object
`clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

### ATTR_MONITOR_MOUNTS

    public static final java.lang.String ATTR_MONITOR_MOUNTS

  List of mounts to monitor; should match "Mounted on" column of 'df' output

  See Also:
    Constant Field Values

### DEFAULT_MONITOR_MOUNTS

    public static final java.lang.String DEFAULT_MONITOR_MOUNTS

See Also:
  Constant Field Values

### ATTR_PAUSE_THRESHOLD

    public static final java.lang.String ATTR_PAUSE_THRESHOLD

  Space available level below which a crawl-pause should be triggered.

  See Also:
    Constant Field Values

### DEFAULT_PAUSE_THRESHOLD

    public static final int DEFAULT_PAUSE_THRESHOLD

See Also:
  Constant Field Values

### ATTR_RECHECK_THRESHOLD

    public static final java.lang.String ATTR_RECHECK_THRESHOLD

  Amount of content received between each recheck of free space

  See Also:
    Constant Field Values

### DEFAULT_RECHECK_THRESHOLD

    public static final int DEFAULT_RECHECK_THRESHOLD

See Also:
  Constant Field Values

### contentSinceCheck

    protected int contentSinceCheck

### VALID_DF_OUTPUT

    public static final java.util.regex.Pattern VALID_DF_OUTPUT

### AVAILABLE_EXTRACTOR

    public static final java.util.regex.Pattern AVAILABLE_EXTRACTOR

Constructor Detail

### LowDiskPauseProcessor

    public LowDiskPauseProcessor(java.lang.String name)

Parameters:
  `name` - Name of this writer.

Method Detail

### innerProcess

    protected void innerProcess(CrawlURI curi)

  Notes a CrawlURI's content size in its running tally. If the recheck increment of content has passed through since the last available-space check, checks available space and pauses the crawl if any monitored mounts are below the configured threshold.

  Overrides:
    `innerProcess` in class `Processor`
  Parameters:
    `curi` - CrawlURI to process.

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2003-2008 Internet Archive. All Rights Reserved.