com.bigdata.resources
Class ScatterSplitTask

java.lang.Object
  com.bigdata.journal.AbstractTask<T>
      com.bigdata.resources.AbstractResourceManagerTask<T>
          com.bigdata.resources.AbstractPrepareTask<AbstractResult>
              com.bigdata.resources.ScatterSplitTask

All Implemented Interfaces:

ITask<AbstractResult>, Callable<AbstractResult>

public class ScatterSplitTask
extends AbstractPrepareTask<AbstractResult>

Task splits an index partition into N equal sized index partitions and scatters those index partitions across data services in the federation. Unlike a normal split, this MAY result in index partitions which are under the nominal minimum size requirements. The purpose of a scatter split is to rapidly redistribute an index partition across the federation in order to increase both the potential concurrency of operations on that index partition and to permit more resources to be brought to bear on the index partition. The "equal" splits are achieved by an "adjustment" to the split handler.

The task reads from the lastCommitTime of the old journal after an overflow. It uses a key range scan to sample the index partition, building an ordered set of {key,offset} tuples. Based on the actual #of index entries and the target #of index entries per index partition, it chooses the #of output index partitions, N, and selects N-1 {key,offset} tuples to split the index partition. If the index defines a constraint on the split rule, then that constraint will be applied to refine the actual split points, e.g., so as to avoid splitting a logical row of a SparseRowStore.

Once the N-1 split points have been selected, N index segments are built - one from each of the N key ranges which those N-1 split points define. Once the index segment for each split has been built, an SplitIndexPartitionTask.AtomicUpdateSplitIndexPartitionTask will atomically re-define the source index partition as N new index partition. During the atomic update the original index partition becomes un-defined and new index partitions are defined in its place which span the same total key range and have the same data.

Version:

$Id: ScatterSplitTask.java 3034 2010-06-17 14:32:09Z thompsonbry $

Author:

Bryan Thompson

See Also:

AtomicUpdateSplitIndexPartitionTask, which MUST be invoked in order to update the index partition definitions on the live journal and the {@link MetadataIndex} as an atomic operation.

Nested Class Summary

Nested classes/interfaces inherited from class com.bigdata.journal.AbstractTask

AbstractTask.DelegateTask<T>, AbstractTask.InnerReadWriteTxServiceCallable, AbstractTask.InnerWriteServiceCallable<T>, AbstractTask.ResubmitException

Field Summary

protected long	adjustedNominalShardSize The target size of a shard for the scatter split.
protected UUID[]	moveTargets An array of move targets for the new index partitions.
protected int	nsplits The #of index partitions that will be generated when we split the source index partition.
protected com.bigdata.resources.ViewMetadata	vmd

Fields inherited from class com.bigdata.resources.AbstractResourceManagerTask

DEBUG, INFO, log, resourceManager

Fields inherited from class com.bigdata.journal.AbstractTask

checkpointNanoTime, concurrencyManager, isReadWriteTx, nanoTime_assignedWorker, nanoTime_beginWork, nanoTime_finishedWork, nanoTime_submitTask, readOnly, taskCounters, timestamp, transactionManager, tx

Constructor Summary

protected

ScatterSplitTask(com.bigdata.resources.ViewMetadata vmd, int nsplits, UUID[] moveTargets)

Method Summary

protected void	clearRefs() Method is responsible for clearing the SoftReferences held by ViewMetadata for the source view(s) on the old journal.
protected AbstractResult	doTask() Breaks the index partition into N splits, where N was specified to the ctor, and redistributes those splits onto the move targets using a round robin.

Methods inherited from class com.bigdata.resources.AbstractResourceManagerTask

toString

Methods inherited from class com.bigdata.journal.AbstractTask

assertResource, assertRunning, assertUnisolated, call, clearLoggingContext, dropIndex, getCommitTime, getIndex, getJournal, getOnlyResource, getResource, getResourceManager, getTaskCounters, getTaskName, getTimestamp, isResource, registerIndex, setupLoggingContext, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

vmd

protected final com.bigdata.resources.ViewMetadata vmd

nsplits

protected final int nsplits

The #of index partitions that will be generated when we split the source index partition.

moveTargets

protected final UUID[] moveTargets

An array of move targets for the new index partitions. The index partitions will be assigned to the move targets using a round robin process. If one of the move targets is this data service, then the corresponding index partition will not be moved.

adjustedNominalShardSize

protected final long adjustedNominalShardSize

The target size of a shard for the scatter split. This is computed by dividing the size of the compact segment on the disk by the #of desired splits.

Constructor Detail

ScatterSplitTask

protected ScatterSplitTask(com.bigdata.resources.ViewMetadata vmd,
                           int nsplits,
                           UUID[] moveTargets)

Parameters:

vmd - The metadata for the index partition to be split.

nsplits - The index will be split into this many index partitions without regard to the #of tuples in each split.

moveTargets - An array of move targets for the new index partitions. The index partitions will be assigned to the move targets using a round robin process. If one of the move targets is this data service, then the corresponding index partition will not be moved.

Method Detail

clearRefs

protected void clearRefs()

Description copied from class: AbstractPrepareTask

Method is responsible for clearing the SoftReferences held by ViewMetadata for the source view(s) on the old journal.

Note: This method MUST be invoked in order to permit those references to be cleared more eagerly than the end of the entire asynchronous overflow operation (which is when the task references would themselves go out of scope and become available for GC).

Specified by:

clearRefs in class AbstractPrepareTask<AbstractResult>

doTask

protected AbstractResult doTask()
                         throws Exception

Breaks the index partition into N splits, where N was specified to the ctor, and redistributes those splits onto the move targets using a round robin.

Specified by:

doTask in class AbstractTask<AbstractResult>

Returns:

A SplitResult if the index partition was split into 2 or more index partitions -or- a BuildResult iff the index partition was not split.

Throws:

Exception - The exception that will be thrown by AbstractTask.call() iff the operation fails.

InterruptedException - This exception SHOULD be thrown if Thread.interrupted() becomes true during execution.