AbstractMasterStats (bigdata® v1.2.1)

Overview

Package

Class

Use

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

com.bigdata.service.ndx.pipeline
Class AbstractMasterStats<L,HS extends AbstractSubtaskStats>

java.lang.Object
  com.bigdata.service.ndx.pipeline.AbstractMasterStats<L,HS>


Type Parameters:: L - The generic type of the key used to lookup subtasks in the internal map.; HS - The generic type of the subtask statistics.

Direct Known Subclasses:: AbstractRunnableMasterStats

public abstract class AbstractMasterStats<L,HS extends AbstractSubtaskStats>
extends Object
extends Object

Abstract base class providing statistics for the AbstractMasterTask and a factory for the statistics for the subtasks.

Note: Since there are concurrent threads which need to write on the counters on this class the practice is to be synchronized on this reference before you update those counters. Without this, the counters might not update correctly as thing like a += 2 may not produce the correct result.

Note: The use of AtomicLongs provides atomic visibility changes relied on by some unit tests.

Version:: $Id: AbstractMasterStats.java 2265 2009-10-26 12:51:06Z thompsonbry $
Author:: Bryan Thompson

Field Summary
`AtomicLong`	`chunksIn` The #of chunks drained from the `BlockingBuffer` by the `AbstractMasterTask`.
`AtomicLong`	`chunksOut` The #of chunks written onto index partitions using RMI.
`AtomicLong`	`chunksTransferred` The #of chunks transferred from the master to the sinks.
`long`	`elapsedRedirectNanos` Elapsed nanoseconds handling a redirect.
`long`	`elapsedSinkChunkWaitingNanos` Elapsed time across sinks waiting for another chunk to be ready so that it can be written onto the index partition.
`long`	`elapsedSinkChunkWritingNanos` Elapsed nanoseconds across sinks writing chunks on an index partition (RMI requests).
`long`	`elapsedSinkOfferNanos` Elapsed nanoseconds the master spends offering a chunk for transfer to a sink.
`AtomicLong`	`elementsIn` The #of elements drained from the `BlockingBuffer` by the `AbstractMasterTask`.
`AtomicLong`	`elementsOnSinkQueues` The #of elements on the output sink queues.
`AtomicLong`	`elementsOut` The #of elements in the output chunks written onto the index partitions (not including any eliminated duplicates).
`AtomicLong`	`elementsTransferred` The #of elements transferred from the master to the sinks.
`AtomicInteger`	`masterCreateCount` The #of master tasks which have been created for the index whose asynchronous write statistics are reported on by this object.
`protected ConcurrentWeakValueCache<Integer,AbstractMasterTask>`	`masters` Weak value hash map of the active masters.
`AtomicLong`	`redirectCount` The #of redirects (`StaleLocatorException`s) that were handled.
`AtomicLong`	`subtaskEndCount` The #of subtasks which have finished (either the buffer has been closed and all buffered data has been flushed -or- the task was interrupted or otherwise threw an exception).
`AtomicLong`	`subtaskIdleTimeoutCount` The #of subtasks which were closed due to an idle timeout.
`AtomicLong`	`subtaskStartCount` The #of subtasks which have started.

Constructor Summary
`AbstractMasterStats()`

Method Summary
`int`	`getActiveSinkCount()` The #of active sinks.
`CounterSet`	`getCounterSet()` Return a `CounterSet` which may be used to report the statistics on the index write operation.
`int`	`getMasterActiveCount()` The approximate #of active master tasks.
`int`	`getMaximumPartitionCount()` The maximum #of distinct partitions for which the master has caused subtasks to be created at any given time.
`Map<L,HS>`	`getSubtaskStats()` Return a snapshot of the statistics for each index partition.
`HS`	`getSubtaskStats(L locator)` Return the statistics object for the specified index partition and never `null` (a new instance is created if none exists).
`protected abstract HS`	`newSubtaskStats(L locator)` Factory for the subtask statistics.
`String`	`toString()`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

subtaskStartCount

public final AtomicLong subtaskStartCount

The #of subtasks which have started.

subtaskEndCount

public final AtomicLong subtaskEndCount

The #of subtasks which have finished (either the buffer has been closed and all buffered data has been flushed -or- the task was interrupted or otherwise threw an exception).

subtaskIdleTimeoutCount

public final AtomicLong subtaskIdleTimeoutCount

The #of subtasks which were closed due to an idle timeout.

redirectCount

public final AtomicLong redirectCount

The #of redirects (StaleLocatorExceptions) that were handled.

elapsedRedirectNanos

public long elapsedRedirectNanos

Elapsed nanoseconds handling a redirect.

chunksIn

public final AtomicLong chunksIn

The #of chunks drained from the BlockingBuffer by the AbstractMasterTask.

elementsIn

public final AtomicLong elementsIn

The #of elements drained from the BlockingBuffer by the AbstractMasterTask.

elementsOut

public final AtomicLong elementsOut

The #of elements in the output chunks written onto the index partitions (not including any eliminated duplicates).

elementsOnSinkQueues

public final AtomicLong elementsOnSinkQueues

The #of elements on the output sink queues. This is incremented when a chunk of elements is transferred onto an output sink queue and decremented when a chunk of elements is drained from an output sink queue. It does not reflect the #of elements on the master queue, which can be approximated as elementsTransferred/ chunksTransferred X the averageMasterQueueSize). Neither does it include the #of elements in a prepared or outstanding write on an index partition.

chunksTransferred

public final AtomicLong chunksTransferred

The #of chunks transferred from the master to the sinks. Where there is more than one index partition, there will be more than one sink and each chunk written on the master will be divided among the sinks based on the key-ranges of the tuples in the chunks. Therefore each chunk drained from the master will be registered as some larger #of chunks transferred to the sink(s) for that master.

elementsTransferred

public final AtomicLong elementsTransferred

The #of elements transferred from the master to the sinks. Where there is more than one index partition, there will be more than one sink and each chunk written on the master will be divided among the sinks based on the key-ranges of the tuples in the chunks. This reduces the average chunk size entering the sink accordingly.

chunksOut

public final AtomicLong chunksOut

The #of chunks written onto index partitions using RMI.

elapsedSinkOfferNanos

public long elapsedSinkOfferNanos

Elapsed nanoseconds the master spends offering a chunk for transfer to a sink.

elapsedSinkChunkWaitingNanos

public long elapsedSinkChunkWaitingNanos

Elapsed time across sinks waiting for another chunk to be ready so that it can be written onto the index partition.

elapsedSinkChunkWritingNanos

public long elapsedSinkChunkWritingNanos

Elapsed nanoseconds across sinks writing chunks on an index partition (RMI requests).

masters

protected final ConcurrentWeakValueCache<Integer,AbstractMasterTask> masters

Weak value hash map of the active masters.

masterCreateCount

public final AtomicInteger masterCreateCount

The #of master tasks which have been created for the index whose asynchronous write statistics are reported on by this object.

Constructor Detail