com.bigdata.resources
Class ResourceManager

java.lang.Object
  com.bigdata.resources.ResourceEvents
      com.bigdata.resources.StoreManager
          com.bigdata.resources.IndexManager
              com.bigdata.resources.OverflowManager
                  com.bigdata.resources.ResourceManager

All Implemented Interfaces:

IResourceManager, IPartitionIdFactory, IServiceShutdown

public abstract class ResourceManager
extends OverflowManager
implements IPartitionIdFactory

The ResourceManager has broad responsibility for journal files, index segment files, maintaining index views during overflow processing, and managing the transparent decomposition of scale-out indices and the distribution of the key-range index partitions for those scale-out indices.

This class is implemented in several layers:

ResourceManager

Concrete implementation.

OverflowManager

Overflow processing.

IndexManager

Manages indices

StoreManager

Manages the journal and index segment files, including the release of old resources.

ResourceEvents

Event reporting API

Version:

$Id: ResourceManager.java 2265 2009-10-26 12:51:06Z thompsonbry $

Author:

Bryan Thompson

TODO:

Transparent promotion of unpartitioned indices to indicate that support delete markers and can therefore undergo OverflowManager.overflow()? This is done by defining one partition that encompases the entire legal key range and setting the resource metadata for the view. However, I am not sure that the situation is likely to arise except if trying to import data from a Journal into an IBigdataFederation.

Transparent promotion of indices to support delete markers on OverflowManager.overflow()? We don't need to maintain delete markers until the first overflow event....

Do NOT break the ability to use concurrency control on unpartitioned indices -- note that overflow handling will only work on that support deletion markers., Document backup procedures for the journal (the journal is a log-structured store; it can be deployed on RAID for media robustness; can be safely copied using normal file copy mechanisms and restored; can be compacted offline; a compact snapshot of a commit point (such as the last commit point) can be generated online and used for recovery; and can be exported onto index segments which can later be restored to any journal, but those index segments need additional metadata to recreate the appropriate relations which is found in the sparse row store) and the federation (relies on service failover (primary and secondaries) and a history retention policy; can be recovered from existing index partitions in a bottom up matter, but that recovery code has not been written).

Nested Class Summary

static interface	ResourceManager.IResourceManagerCounters Interface defines and documents the counters and counter namespaces for the ResourceManager.
static interface	ResourceManager.Options ResourceManager options.

Nested classes/interfaces inherited from class com.bigdata.resources.OverflowManager

OverflowManager.IIndexPartitionTaskCounters, OverflowManager.IOverflowManagerCounters

Nested classes/interfaces inherited from class com.bigdata.resources.IndexManager

IndexManager.IIndexManagerCounters, IndexManager.IndexCache, IndexManager.IndexSegmentStats

Nested classes/interfaces inherited from class com.bigdata.resources.StoreManager

StoreManager.IStoreManagerCounters, StoreManager.ManagedJournal

Field Summary

protected static org.apache.log4j.Logger

log Logger.

Fields inherited from class com.bigdata.resources.OverflowManager

accelerateSplitThreshold, asynchronousOverflowCounter, asynchronousOverflowMillis, asynchronousOverflowStartMillis, asyncOverflowEnabled, asyncOverflowFailedCounter, asyncOverflowTaskCancelledCounter, asyncOverflowTaskFailedCounter, compactingMerge, copyIndexThreshold, hotSplitThreshold, indexPartitionBuildCounter, indexPartitionJoinCounter, indexPartitionMergeCounter, indexPartitionMoveCounter, indexPartitionReceiveCounter, indexPartitionSplitCounter, indexPartitionTailSplitCounter, joinsEnabled, maximumBuildSegmentBytes, maximumJournalsPerView, maximumMoves, maximumMovesPerTarget, maximumOptionalMergesPerOverflow, maximumSegmentsPerView, minimumActiveIndexPartitions, movePercentCpuTimeThreshold, overflowAllowed, overflowCancelledWhenJournalFull, overflowTasksConcurrent, overflowThreshold, overflowTimeout, percentOfSplitThreshold, scatterSplitEnabled, synchronousOverflowCounter, synchronousOverflowMillis, tailSplitThreshold

Fields inherited from class com.bigdata.resources.IndexManager

buildTasks, staleLocatorCache

Fields inherited from class com.bigdata.resources.StoreManager

accelerateOverflowThreshold, bytesDeleted, bytesUnderManagement, dataDir, indexCacheLock, journalBytesUnderManagement, journalDeleteCount, journalReopenCount, journalsDir, lastCommitTimePreserved, lastOverflowTime, liveJournalRef, maximumJournalSizeAtOverflow, purgeResourcesMillis, resourceService, segmentBytesUnderManagement, segmentsDir, segmentStoreDeleteCount, segmentStoreReopenCount, storeCache, tmpDir

Constructor Summary

ResourceManager(Properties properties)
(Re-)open the ResourceManager.

Method Summary

IConcurrencyManager	getConcurrencyManager() The object used to control access to the index resources.
CounterSet	getCounters() WARNING: The DataService transfers all of the children from this object into the hierarchy reported by AbstractFederation.getServiceCounterSet() and this object will be empty thereafter.
CounterSet	getIndexManagerCounters() The counter set that corresponds to the IndexManager.
int	nextPartitionId(String scaleOutIndexName) Requests a new index partition identifier from the MetadataService for the specified scale-out index (RMI).
void	setConcurrencyManager(IConcurrencyManager concurrencyManager)

Methods inherited from class com.bigdata.resources.OverflowManager

doSynchronousOverflow, getAsynchronousOverflowCount, getSynchronousOverflowCount, isOverflowAllowed, isOverflowEnabled, overflow, shouldOverflow, shutdown, shutdownNow

Methods inherited from class com.bigdata.resources.IndexManager

buildIndexSegment, getIndex, getIndexCacheCapacity, getIndexCacheSize, getIndexCounters, getIndexCounters, getIndexOnStore, getIndexPartitionGone, getIndexRetentionTime, getIndexSegmentCacheCapacity, getIndexSegmentCacheSize, getIndexSources, getIndexSources, getStaleLocatorCount, listIndexPartitions, markAndGetDelta, setIndexPartitionGone

Methods inherited from class com.bigdata.resources.StoreManager

addResource, assertNotOpen, assertOpen, assertRunning, awaitRunning, deleteResource, deleteResources, getBytesUnderManagement, getCommitTimeStrictlyGreaterThan, getDataDir, getDataDirFreeSpace, getIndexSegmentFile, getIndexSegmentFile, getJournal, getJournalBytesUnderManagement, getLiveJournal, getManagedJournalCount, getManagedSegmentCount, getProperties, getReleaseTime, getResourceServicePort, getResourcesForTimestamp, getSegmentBytesUnderManagement, getStoreCacheSize, getStoreCounters, getTempDirFreeSpace, getTmpDir, isOpen, isRunning, isStarting, isTransient, munge, newFileFilter, nextTimestamp, openStore, overrideJournalExtent, purgeOldResources, purgeOldResources, retentionSetAdd, retentionSetRemove, setReleaseTime

Methods inherited from class com.bigdata.resources.ResourceEvents

closeJournal, closeTx, closeUnisolatedBTree, deleteJournal, dropUnisolatedBTree, extendJournal, isolateIndex, openJournal, openTx, openUnisolatedBTree

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.bigdata.journal.IResourceManager

getDataService, getDataServiceUUID, getFederation

Field Detail

log

protected static final org.apache.log4j.Logger log

Logger.

Constructor Detail

ResourceManager

public ResourceManager(Properties properties)

(Re-)open the ResourceManager.

Note: You MUST use setConcurrencyManager(IConcurrencyManager) after calling this constructor (the parameter can not be passed in since there is a circular dependency between the IConcurrencyManager and ManagedJournal#getLocalTransactionManager().

Parameters:

properties - See ResourceManager.Options.

See Also:

DataService.start()

Method Detail

getCounters

public CounterSet getCounters()

WARNING: The DataService transfers all of the children from this object into the hierarchy reported by AbstractFederation.getServiceCounterSet() and this object will be empty thereafter.

Specified by:

getCounters in interface IResourceManager

getIndexManagerCounters

public CounterSet getIndexManagerCounters()

The counter set that corresponds to the IndexManager.

getConcurrencyManager

public IConcurrencyManager getConcurrencyManager()

Description copied from class: StoreManager

The object used to control access to the index resources.

Specified by:

getConcurrencyManager in class StoreManager

setConcurrencyManager

public void setConcurrencyManager(IConcurrencyManager concurrencyManager)

Specified by:

setConcurrencyManager in class StoreManager

nextPartitionId

public int nextPartitionId(String scaleOutIndexName)

Requests a new index partition identifier from the MetadataService for the specified scale-out index (RMI).

Specified by:

nextPartitionId in interface IPartitionIdFactory

Parameters:

scaleOutIndexName - The name of the scale-out index.

Returns:

The new index partition identifier.

Throws:

RuntimeException - if something goes wrong.