com.bigdata.service.jini.master
Class MappedTaskMaster<S extends MappedTaskMaster.JobState,T extends AbstractAsynchronousClientTask<U,V,L>,L extends ClientLocator,U,V extends Serializable>

java.lang.Object
  com.bigdata.service.jini.master.TaskMaster<S,T,U>
      com.bigdata.service.jini.master.MappedTaskMaster<S,T,L,U,V>

Type Parameters:

S - The generic type of the MappedTaskMaster.JobState.

T - The generic type of the client task.

U - The generic type of the value returned by the client task.

V - The generic type of the resources to be tasked.

All Implemented Interfaces:

Callable<Void>

Direct Known Subclasses:

MappedRDFDataLoadMaster

public abstract class MappedTaskMaster<S extends MappedTaskMaster.JobState,T extends AbstractAsynchronousClientTask<U,V,L>,L extends ClientLocator,U,V extends Serializable>
extends TaskMaster<S,T,U>

Extends the TaskMaster to assign chunks of resources for processing to the client tasks.

The master scans a source such as the file system, a distributed file system, or a scale-out index to identify the resources to be processed. The resources are placed onto a hash-partitioned buffer, which is drained by the various clients. If a client succeeds, then the resources have been handled successfully and a new set of resources is assigned to that client. If a client fails, the resources are reassigned to a different client. If all clients fail, the job is aborted.

Version:

$Id$

Author:

Bryan Thompson

Nested Class Summary

static interface	MappedTaskMaster.ConfigurationOptions Configuration options for the RDFDataLoadMaster.
static class	MappedTaskMaster.JobState The job description for an RDFDataLoadMaster.

Nested classes/interfaces inherited from class com.bigdata.service.jini.master.TaskMaster

TaskMaster.DiscoveredServices

Field Summary

Fields inherited from class com.bigdata.service.jini.master.TaskMaster

fed, log

Constructor Summary

MappedTaskMaster(JiniFederation<?> fed)

Method Summary

protected abstract T	newClientTask(INotifyOutcome<V,L> notifyProxy, L locator) Factory for new client tasks.
protected T	newClientTask(int clientNum) Unsupported -- see #newClientTask(INotifyOutcome, int).
BlockingBuffer<V[]>	newResourceBuffer() Allocate and start processing for the buffer used to hand off resources for processing to the clients.
protected void	runJob() Runs the scanner, handing off resources to clients for processing.

Methods inherited from class com.bigdata.service.jini.master.TaskMaster

allDone, attachPerformanceCounters, awaitAll, beginJob, call, cancelAll, detachPerformanceCounters, error, execute, forceOverflow, getFederation, getJobState, innerMain, newJobState, notifyOutcome, setupJob, startClients, success, tearDownJob

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MappedTaskMaster

public MappedTaskMaster(JiniFederation<?> fed)
                 throws net.jini.config.ConfigurationException

Throws:

net.jini.config.ConfigurationException

Method Detail

runJob

protected void runJob()
               throws Exception

Runs the scanner, handing off resources to clients for processing. The clients should run until they are interrupted by the master. When the scanner is done, the resource buffer is closed. The master will continue to run until the pendingSet is empty. Once the master buffer is exhausted (closed and drained), the sinks will flush their last chunks and then IAsynchronousClientTask.close() the clientTask. This prevents workload starvation during the shutdown protocol.

Overrides:

runJob in class TaskMaster<S extends MappedTaskMaster.JobState,T extends AbstractAsynchronousClientTask<U,V,L>,U>

Throws:

Exception - Client execution problem.

InterruptedException - Master interrupted awaiting clients.

newResourceBuffer

public BlockingBuffer<V[]> newResourceBuffer()

Allocate and start processing for the buffer used to hand off resources for processing to the clients.

TODO:

Specializing the behavior of the MappedTaskMaster requires overriding the behavior of the ResourceBufferTask. It should be parameterized if possible with interfaces for interesting things, however its termination logic is tricky and overrides should not mess with that.

newClientTask

protected T newClientTask(int clientNum)

Unsupported -- see #newClientTask(INotifyOutcome, int).

Specified by:

newClientTask in class TaskMaster<S extends MappedTaskMaster.JobState,T extends AbstractAsynchronousClientTask<U,V,L>,U>

Parameters:

clientNum - The client number.

Returns:

The client task.

Throws:

UnsupportedOperationException - always.

See Also:

AbstractClientTask

newClientTask

protected abstract T newClientTask(INotifyOutcome<V,L> notifyProxy,
                                   L locator)

Factory for new client tasks.

Parameters:

notifyProxy - The proxy for the object to which the client must deliver notice of success or failure for each processed resource.

locator - The locator for the client on which the task will be executed.

Returns:

The client task.