Fault Detection and Recovery
Prev | Chapter 4. Architecture Fundamentals | Next

## Fault Detection and Recovery

Key to keeping the a distributed application operating is to ensure services are still running. Utilizing Fault Detection Policies, Rio provides the essential capability to detect existence of running and available services. If services become unavailable, or fail to meet processing requirements, the recovery of those services to available compute resources is enabled by provisioning mechanisms.

A Fault Detection Handler is loaded by client entities when a service is provisioned, has the service's proxy set, and periodically tests to make sure the service is reachable. The Fault Detection Handler can implement custom fault detection algorithms and protocols to determine service reachability, and different heuristics for failure detection and recovery for different failure types.

### Fault Detection Handlers

Rio provides the following fault detection handlers. Two of them use a basic polling strategy (AdminFaultDetectionHandler and StandardFaultDetectionHandler). The HeartbeatFaultDetectionHandler has

It should be noted that the amount of time it takes for the AdminFaultDetectionHandler and the StandardFaultDetectionHandler to determine service failure is calculated as follows:

    ((num_retries + 1) * (connectivity_timeout)) + (retry_delay * num_retries)

Additionally, Fault Detection Handler also register with Lookup Services for ServiceRegistrar.TRANSITION_MATCH_NOMATCH transitions for the service being monitored. If the service is adminstratively removed from the network, or the service monitoring lease is between lease renewal points and the service has actually been removed from the network, the transition will be noted and FaultDetectionListener instances will be notified of the failure.

- AdminFaultDetectionHandler

  The AdminFaultDetectionHandler can be used to monitor a service whose proxy provides support for the net.jini.admin.Administrable interface. If the service proxy does implement the net.jini.admin.Administrable, the AdminFaultDetectionHandler invokes the Administrable.getAdmin() method periodically. If this method invocation returns successfully, the service is assumed to be available. If this method invocation results in a failure, and all retry attempts have failed, the AdminFaultDetectionHandler will notify FaultDetectionListener instances of the failure.

  If the service proxy does not implement the net.jini.admin.Administrable interface, the HeartbeatFaultDetectionHandler will not perform heartbeat monitoring, but will only create the event consumer for Lookup Service TRANSITION_MATCH_NOMATCH transitions.

- HeartbeatFaultDetectionHandler

  The HeartbeatFaultDetectionHandler can be used to monitor a service whose proxy provides support for the org.jini.rio.core.MonitorableService interface. The HeartbeatFaultDetectionHandler creates a server socket and listens for notifications from the service of the service's reachability. If the service fails to provide a heartbeat notification within the timeframe specified the HeartbeatFaultDetectionHandler will notify FaultDetectionListener instances of the failure. If the service proxy does not implement the org.jini.rio.core.MonitorableService interface, the HeartbeatFaultDetectionHandler will not perform heartbeat monitoring, but will only create the event consumer for Lookup Service TRANSITION_MATCH_NOMATCH transitions.

- StandardFaultDetectionHandler

  The StandardFaultDetectionHandler can be used to monitor a service whose proxy provides support for the org.jini.rio.core.MonitorableService interface, as well as for services that do not implement this interface. If the service does implement the org.jini.rio.core.MonitorableService, the StandardFaultDetectionHandler creates a client-side lease with the service, and attempts to renew the lease based on a configurable lease duration time. If the lease cannot be renewed and all retry attempts have failed, the StandardFaultDetectionHandler will notify FaultDetectionListener instances of the failure. If the service proxy does not implement the org.jini.rio.core.MonitorableService interface, the StandardFaultDetectionHandler will not manage a Lease, but will only create the event consumer for Lookup Service TRANSITION_MATCH_NOMATCH transitions.

Prev | Up | Next
Service Level Agreements | Home | Part III. Administration