Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.archive.crawler.framework
Class Checkpointer

    java.lang.Object
      org.archive.crawler.framework.Checkpointer

All Implemented Interfaces:
  java.io.Serializable

    public class Checkpointer

extends java.lang.Object
implements java.io.Serializable

Runs checkpointing. Also keeps history of crawl checkpoints Generally used by CrawlController only but also has static utility methods classes that need to participate in a checkpoint can use.

Author:
  gojomo, stack
See Also:
  Serialized Form

Nested Class Summary
`class` | `Checkpointer.CheckpointingThread` Thread to run the checkpointing.

Field Summary
`static java.text.DecimalFormat` | `INDEX_FORMAT`

Constructor Summary
`Checkpointer(CrawlController cc, java.io.File checkpointDir)` Create a new CheckpointContext with the given store directory
`Checkpointer(CrawlController cc, java.lang.String prefix)` Create a new CheckpointContext with the given store directory

Method Summary
`void` | `checkpoint()` Run a checkpoint of the crawler.
`protected void` | `checkpointFailed()`
`protected void` | `checkpointFailed(java.lang.Exception e)` Note that a checkpoint failed
`protected void` | `checkpointFailed(java.lang.String message)`
`(package private) void` | `cleanup()`
`protected void` | `clearCheckpointInProgressDirectory()`
`protected java.io.File` | `createCheckpointInProgressDirectory()`
`static java.lang.String` | `formatCheckpointName(java.lang.String prefix, int index)`
`java.io.File` | `getCheckpointInProgressDirectory()`
`protected CrawlController` | `getController()`
`int` | `getNextCheckpoint()`
`java.lang.String` | `getNextCheckpointName()`
`java.util.List` | `getPredecessorCheckpoints()`
`protected void` | `initialize(CrawlController cc, java.lang.String prefix)`
`boolean` | `isAtBeginning()`
`protected boolean` | `isCheckpointErrors()`
`boolean` | `isCheckpointFailed()`
`boolean` | `isCheckpointing()`
`void` | `recover(CrawlController cc)` Call when recovering from a checkpoint.
`protected void` | `setCheckpointErrors(boolean checkpointErrors)`
`protected void` | `writeValidity()`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### INDEX_FORMAT

    public static final java.text.DecimalFormat INDEX_FORMAT

Constructor Detail

### Checkpointer

    public Checkpointer(CrawlController cc,
                        java.io.File checkpointDir)

  Create a new CheckpointContext with the given store directory

  Parameters:
    `cc` - CrawlController instance thats hosting this Checkpointer.
    `checkpointDir` - Where to store checkpoint.

### Checkpointer

    public Checkpointer(CrawlController cc,
                        java.lang.String prefix)

  Create a new CheckpointContext with the given store directory

  Parameters:
    `cc` - CrawlController instance thats hosting this Checkpointer.
    `prefix` - Prefix for checkpoint label.

Method Detail

### initialize

    protected void initialize(CrawlController cc,
                              java.lang.String prefix)

### cleanup

    void cleanup()

### getNextCheckpoint

    public int getNextCheckpoint()

  Returns:
    Returns the nextCheckpoint index.

### checkpoint

    public void checkpoint()

  Run a checkpoint of the crawler.

### createCheckpointInProgressDirectory

    protected java.io.File createCheckpointInProgressDirectory()

### clearCheckpointInProgressDirectory

    protected void clearCheckpointInProgressDirectory()

### getController

    protected CrawlController getController()

### getNextCheckpointName

    public java.lang.String getNextCheckpointName()

  Returns:
    next checkpoint name (zero-padding string).

### formatCheckpointName

    public static java.lang.String formatCheckpointName(java.lang.String prefix,
                                                        int index)

### writeValidity

    protected void writeValidity()

### getCheckpointInProgressDirectory

    public java.io.File getCheckpointInProgressDirectory()

  Returns:
    Checkpoint directory. Name of the directory is the name of this current checkpoint. Null if no checkpoint in progress.

### isCheckpointing

    public boolean isCheckpointing()

  Returns:
    True if a checkpoint is in progress.

### checkpointFailed

    protected void checkpointFailed(java.lang.Exception e)

  Note that a checkpoint failed

  Parameters:
    `e` - Exception checkpoint failed on.

### checkpointFailed

    protected void checkpointFailed(java.lang.String message)

### checkpointFailed

    protected void checkpointFailed()

### isCheckpointFailed

    public boolean isCheckpointFailed()

  Returns:
    True if current/last checkpoint failed.

### isAtBeginning

    public boolean isAtBeginning()

  Returns:
    Return whether this context is at a new crawl, never- checkpointed state.

### recover

    public void recover(CrawlController cc)

  Call when recovering from a checkpoint. Call this after instance has been revivifyied post-serialization to amend counters and directories that effect where checkpoints get stored from here on out.

  Parameters:
    `cc` - CrawlController instance.

### getPredecessorCheckpoints

    public java.util.List getPredecessorCheckpoints()

  Returns:
    Returns the predecessorCheckpoints.

### isCheckpointErrors

    protected boolean isCheckpointErrors()

### setCheckpointErrors

    protected void setCheckpointErrors(boolean checkpointErrors)

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2003-2008 Internet Archive. All Rights Reserved.