com.bigdata.journal
Class Journal

java.lang.Object
  com.bigdata.journal.AbstractJournal
      com.bigdata.journal.Journal

All Implemented Interfaces:

IAtomicStore, IBTreeManager, IConcurrencyManager, IIndexManager, IIndexStore, IJournal, IResourceManager, IAddressManager, IMRMW, IMROW, IRawStore, IStoreSerializer, IServiceShutdown

public class Journal
extends AbstractJournal
implements IConcurrencyManager, IResourceManager

Concrete implementation suitable for a local and unpartitioned database.

Note: This implementation does NOT not support partitioned indices. Because all data must reside on a single journal resource there is no point to a view. Views are designed to have data on a mixture of the live journal, one or more historical journals, and one or more IndexSegments.

See Also:

ResourceManager, which supports views.

Nested Class Summary

static interface

Journal.Options Options understood by the Journal.

Field Summary

Fields inherited from class com.bigdata.journal.AbstractJournal

deleteOnClose, doubleSync, forceOnCommit, log, properties, ROOT_NAME2ADDR, tmpDir

Fields inherited from interface com.bigdata.rawstore.IAddressManager

NULL

Constructor Summary

Journal(Properties properties)
Create or re-open a journal.

Method Summary

void	abort(long tx) Abort a transaction.
long	commit(long tx) Commit a transaction.
Future<Journal>	compact(File outFile) Compacts the named indices found on this journal as of the most recent commit point, writing their view onto a new Journal.
void	deleteResources() Deletes the backing file(s) (if any).
protected void	discardCommitters() This method is invoked by AbstractJournal.abort() when the store must discard any hard references that it may be holding to objects registered as ICommitters.
ConcurrencyManager	getConcurrencyManager()
CounterSet	getCounters() Return counters reporting on various aspects of the journal.
File	getDataDir() The directory in which the journal's file is located -or- null if the journal is not backed by a file.
DataService	getDataService() The local DataService whose resources are being managed.
UUID	getDataServiceUUID() Return the UUID of the IDataService whose resources are being managed.
ExecutorService	getExecutorService() Service for running arbitrary tasks in support of IResourceLocator.
IBigdataFederation	getFederation() The federation whose resources are being managed.
BigdataFileSystem	getGlobalFileSystem() Return the global file system used to store block-structured files and their metadata and as a source and sink for map/reduce processing.
SparseRowStore	getGlobalRowStore() Return the global SparseRowStore used to store named property sets.
ILocalBTreeView	getIndex(String name, long timestamp) Note: ITx.READ_COMMITTED views are given read-committed semantics using a ReadCommittedView.
StaleLocatorReason	getIndexPartitionGone(String name) Always returns null since index partition moves are not supported.
File	getIndexSegmentFile(IndexMetadata indexMetadata) Return the file on which a new IndexSegment should be written.
AbstractBTree[]	getIndexSources(String name, long timestamp) Always returns an array containing a single BTree which is the BTree loaded from the commit record whose commit timestamp is less than or equal to timestamp -or- null if there are no ICommitRecords that satisify the probe or if the named index was not registered as of that timestamp.
AbstractBTree[]	getIndexSources(String name, long timestamp, BTree btree) Always returns the BTree as the sole element of the array since partitioned indices are not supported.
AbstractJournal	getJournal(long timestamp) Always returns this.
AbstractJournal	getLiveJournal() Always returns this.
AbstractLocalTransactionManager	getLocalTransactionManager() Return the object providing the AbstractLocalTransactionManager for this journal.
DefaultResourceLocator	getResourceLocator() Return the default locator for resources that are logical index containers (relations and relation containers).
IResourceLockService	getResourceLockService() The service that may be used to acquire synchronous distributed locks without deadlock detection.
IResourceManager	getResourceManager() The object used to manage local resources.
TemporaryStore	getTempStore() A factory for TemporaryStores.
File	getTmpDir() The directory for temporary files.
ILocalTransactionManager	getTransactionManager() The client side of the transaction manager.
WriteExecutorService	getWriteService() The service on which read-write tasks are executed.
<T> List<Future<T>>	invokeAll(Collection<? extends AbstractTask<T>> tasks) Executes the given tasks, returning a list of Futures holding their status and results when all complete.
List<Future>	invokeAll(Collection<? extends AbstractTask> tasks, long timeout, TimeUnit unit) Executes the given tasks, returning a list of Futures holding their status and results when all complete or the timeout expires, whichever happens first.
boolean	isOverflowEnabled() Note: This implementation always returns false.
long	newTx(long timestamp) Create a new transaction on the Journal.
long	nextTimestamp() Deprecated. This is here for historical reasons and is only used by the test suite. Use getLocalTransactionManager() and ITimestampService.nextTimestamp().
IRawStore	openStore(UUID uuid) Note: This will only succeed if the uuid identifies this journal.
Future<Object>	overflow() Overflow processing creates a new journal, migrates the named indices on the current journal the new journal, and continues operations on the new journal.
boolean	shouldOverflow() Note: This implementation always returns false.
void	shutdown() Note: The transaction service si shutdown first, then the executorService, then the IConcurrencyManager, the ITransactionService and finally the IResourceLockService.
void	shutdownNow() Note: The IConcurrencyManager is shutdown first, then the ITransactionService and finally the IResourceManager.
<T> Future<T>	submit(AbstractTask<T> task) Submit a task (asynchronous).

Methods inherited from class com.bigdata.journal.AbstractJournal

_close, _getName2Addr, abort, assertOpen, close, closeForWrites, commit, commitNow, deserialize, deserialize, deserialize, destroy, dropIndex, ensureMinFree, finalize, force, getBufferStrategy, getByteCount, getCommitRecord, getCommitRecord, getCommitRecordIndex, getCommitRecordIndex, getCommitRecordStrictlyGreaterThan, getFile, getIndex, getIndex, getIndex, getLastCommitTime, getMaximumExtent, getMaxRecordSize, getName2Addr, getName2Addr, getOffset, getOffsetBits, getProperties, getProperty, getProperty, getProperty, getResourceMetadata, getRootAddr, getRootBlockView, getUUID, getWriteCache, isFullyBuffered, isOpen, isReadOnly, isStable, packAddr, read, registerIndex, registerIndex, registerIndex, registerIndex, rollback, serialize, setCommitter, setupCommitters, size, toAddr, toString, truncate, unpackAddr, validateIndexMetadata, write

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.bigdata.service.IServiceShutdown

isOpen

Constructor Detail

Journal

public Journal(Properties properties)

Create or re-open a journal.

Parameters:

properties - See Options.

Method Detail

getLocalTransactionManager

public AbstractLocalTransactionManager getLocalTransactionManager()

Description copied from class: AbstractJournal

Return the object providing the AbstractLocalTransactionManager for this journal.

Specified by:

getLocalTransactionManager in class AbstractJournal

getCounters

public CounterSet getCounters()

Description copied from class: AbstractJournal

Return counters reporting on various aspects of the journal.

Specified by:

getCounters in interface IConcurrencyManager

Specified by:

getCounters in interface IResourceManager

Specified by:

getCounters in interface IRawStore

Overrides:

getCounters in class AbstractJournal

getTmpDir

public File getTmpDir()

Description copied from interface: IResourceManager

The directory for temporary files.

Specified by:

getTmpDir in interface IResourceManager

getDataDir

public File getDataDir()

The directory in which the journal's file is located -or- null if the journal is not backed by a file.

Specified by:

getDataDir in interface IResourceManager

openStore

public IRawStore openStore(UUID uuid)

Note: This will only succeed if the uuid identifies this journal.

Specified by:

openStore in interface IResourceManager

Parameters:

uuid - The UUID identifying that store file.

Returns:

The open IRawStore.

getIndexSources

public AbstractBTree[] getIndexSources(String name,
                                       long timestamp)

Always returns an array containing a single BTree which is the BTree loaded from the commit record whose commit timestamp is less than or equal to timestamp -or- null if there are no ICommitRecords that satisify the probe or if the named index was not registered as of that timestamp.

Specified by:

getIndexSources in interface IResourceManager

Parameters:

name -

timestamp -

Returns:

The sources for the index view or null if the index was not defined as of the timestamp.

Throws:

UnsupportedOperationException - If the timestamp is ITx.READ_COMMITTED. You MUST use getIndex(String, long) in order to obtain a view that has ITx.READ_COMMITTED semantics.

See Also:

FusedView

getLiveJournal

public final AbstractJournal getLiveJournal()

Always returns this.

Specified by:

getLiveJournal in interface IResourceManager

getJournal

public final AbstractJournal getJournal(long timestamp)

Always returns this.

Specified by:

getJournal in interface IResourceManager

Parameters:

timestamp - A transaction identifier, ITx.UNISOLATED for the unisolated index view, ITx.READ_COMMITTED, or timestamp for a historical view no later than the specified timestamp.

Returns:

The corresponding journal for that timestamp -or- null if no journal has data for that timestamp, including when a historical journal with data for that timestamp has been deleted.

compact

public Future<Journal> compact(File outFile)

Compacts the named indices found on this journal as of the most recent commit point, writing their view onto a new Journal. This method MAY be used concurrently with the Journal but writes after the selected commit point WILL NOT be reflected in the output file. Typical uses are to reduce the space required by the backing store, to improve locality in the backing store, and to make a backup of the most recent commit point.

Parameters:

outFile - The file on which the new journal will be created.

Returns:

The Future on which you must wait for the CompactTask to complete. The already open journal is accessible using Future.get(). If you are backing up data, then be sure to shutdown the returned Journal so that it can release its resources.

getIndex

public ILocalBTreeView getIndex(String name,
                                long timestamp)

Note: ITx.READ_COMMITTED views are given read-committed semantics using a ReadCommittedView. This means that they can be cached since the view will update automatically as commits are made against the Journal.

Specified by:

getIndex in interface IIndexStore

Specified by:

getIndex in interface IResourceManager

Overrides:

getIndex in class AbstractJournal

Parameters:

name - The index name.

timestamp - A positive timestamp (a possible commit time on the store).

Returns:

The index or null iff there is no index registered with that name for that timestamp.

See Also:

IndexManager.getIndex(String, long)

getIndexSources

public AbstractBTree[] getIndexSources(String name,
                                       long timestamp,
                                       BTree btree)

Always returns the BTree as the sole element of the array since partitioned indices are not supported.

Specified by:

getIndexSources in interface IResourceManager

btree - A BTree.

Returns:

The source(s) for the view associated with that BTree.

newTx

public long newTx(long timestamp)

Create a new transaction on the Journal.

Parameters:

timestamp - A positive timestamp for a historical read-only transaction as of the first commit point LTE the given timestamp, ITx.READ_COMMITTED for a historical read-only transaction as of the most current commit point on the Journal as of the moment that the transaction is created, or ITx.UNISOLATED for a read-write transaction.

Returns:

The transaction identifier.

See Also:

ITransactionService.newTx(long)

abort

public void abort(long tx)

Abort a transaction.

Parameters:

tx - The transaction identifier.

See Also:

ITransactionService.abort(long)

commit

public long commit(long tx)
            throws ValidationError

Commit a transaction.

Parameters:

tx - The transaction identifier.

Returns:

The commit time assigned to that transaction.

Throws:

ValidationError

See Also:

ITransactionService.commit(long)

nextTimestamp

public long nextTimestamp()

Deprecated. This is here for historical reasons and is only used by the test suite. Use getLocalTransactionManager() and ITimestampService.nextTimestamp().

Returns the next timestamp from the ILocalTransactionManager.

getConcurrencyManager

public ConcurrencyManager getConcurrencyManager()

shutdown

public void shutdown()

Note: The transaction service si shutdown first, then the executorService, then the IConcurrencyManager, the ITransactionService and finally the IResourceLockService.

Specified by:

shutdown in interface IConcurrencyManager

Specified by:

shutdown in interface IJournal

Specified by:

shutdown in interface IServiceShutdown

Overrides:

shutdown in class AbstractJournal

See Also:

AbstractJournal.shutdownNow()

shutdownNow

public void shutdownNow()

Note: The IConcurrencyManager is shutdown first, then the ITransactionService and finally the IResourceManager.

Specified by:

shutdownNow in interface IConcurrencyManager

Specified by:

shutdownNow in interface IJournal

Specified by:

shutdownNow in interface IServiceShutdown

Overrides:

shutdownNow in class AbstractJournal

See Also:

AbstractJournal.shutdown()

deleteResources

public void deleteResources()

Description copied from class: AbstractJournal

Deletes the backing file(s) (if any).

Note: This is the core implementation of delete and handles event reporting.

Specified by:

deleteResources in interface IResourceManager

Specified by:

deleteResources in interface IRawStore

Overrides:

deleteResources in class AbstractJournal

submit

public <T> Future<T> submit(AbstractTask<T> task)

Description copied from interface: IConcurrencyManager

Submit a task (asynchronous). Tasks will execute asynchronously in the appropriate thread pool with as much concurrency as possible.

Note: Unisolated write tasks will NOT return before the next group commit (exceptions may be thrown if the task fails or the commit fails). The purpose of group commits is to provide higher throughput for writes on the store by only syncing the data to disk periodically rather than after every write. Group commits are scheduled by the #commitService. The trigger conditions for group commits may be configured using ConcurrencyManager.Options. If you are using the store in a single threaded context then you may set Options#WRITE_SERVICE_CORE_POOL_SIZE to ONE (1) which has the effect of triggering commit immediately after each unisolated write. However, note that you can not sync a disk more than ~ 30-40 times per second so your throughput in write operations per second will never exceed that for a single-threaded application writing on a hard disk. (Your mileage can vary if you are writing on a transient store or using a durable medium other than disk).

Note: The isolated indices used by a read-write transaction are NOT thread-safe. Therefore a partial order is imposed over concurrent tasks for the same transaction that seek to read or write on the same index(s). Full concurrency is allowed when different transactions access the same index(s), but write-write conflicts MAY be detected during commit processing.

Note: The following exceptions MAY be wrapped by Future.get() for tasks submitted via this method:

ValidationError

An unisolated write task was attempting to commit the write set for a transaction but validation failed. You may retry the entire transaction.

InterruptedException

A task was interrupted during execution and before the task had completed normally. You MAY retry the task, but note that this exception is also generated when tasks are cancelled when the journal is being IConcurrencyManager.shutdown() after the timeout has expired or IConcurrencyManager.shutdownNow(). In either of these cases the task will not be accepted by the journal.

Specified by:

submit in interface IConcurrencyManager

Parameters:

task - The task.

Returns:

The Future that may be used to resolve the outcome of the task.

invokeAll

public List<Future> invokeAll(Collection<? extends AbstractTask> tasks,
                              long timeout,
                              TimeUnit unit)
                       throws InterruptedException

Description copied from interface: IConcurrencyManager

Executes the given tasks, returning a list of Futures holding their status and results when all complete or the timeout expires, whichever happens first. Note that a completed task could have terminated either normally or by throwing an exception. The results of this method are undefined if the given collection is modified while this operation is in progress.

Note: Contract is based on ExecutorService.invokeAll(Collection, long, TimeUnit) but only the Futures of the submitted tasks are returned.

Specified by:

invokeAll in interface IConcurrencyManager

Parameters:

tasks - The tasks.

Returns:

The Futures of all tasks that were submitted prior to the expiration of the timeout.

Throws:

InterruptedException - if interrupted while waiting, in which case unfinished tasks are cancelled.

invokeAll

public <T> List<Future<T>> invokeAll(Collection<? extends AbstractTask<T>> tasks)
                          throws InterruptedException

Description copied from interface: IConcurrencyManager

Executes the given tasks, returning a list of Futures holding their status and results when all complete. Note that a completed task could have terminated either normally or by throwing an exception. The results of this method are undefined if the given collection is modified while this operation is in progress.

Note: Contract is per ExecutorService.invokeAll(Collection)

Specified by:

invokeAll in interface IConcurrencyManager

Parameters:

tasks - The tasks.

Returns:

Their Futures.

Throws:

InterruptedException - if interrupted while waiting, in which case unfinished tasks are cancelled.

getResourceManager

public IResourceManager getResourceManager()

Description copied from interface: IConcurrencyManager

The object used to manage local resources.

Specified by:

getResourceManager in interface IConcurrencyManager

getTransactionManager

public ILocalTransactionManager getTransactionManager()

Description copied from interface: IConcurrencyManager

The client side of the transaction manager.

Specified by:

getTransactionManager in interface IConcurrencyManager

getWriteService

public WriteExecutorService getWriteService()

Description copied from interface: IConcurrencyManager

The service on which read-write tasks are executed.

Specified by:

getWriteService in interface IConcurrencyManager

shouldOverflow

public boolean shouldOverflow()

Note: This implementation always returns false. As a consequence the journal capacity will simply be extended by AbstractJournal.write(ByteBuffer) until the available disk space is exhausted.

Specified by:

shouldOverflow in interface IResourceManager

Returns:

This implementation returns false since overflow is NOT supported.

isOverflowEnabled

public boolean isOverflowEnabled()

Note: This implementation always returns false.

Specified by:

isOverflowEnabled in interface IResourceManager

overflow

public Future<Object> overflow()

Description copied from interface: IResourceManager

Overflow processing creates a new journal, migrates the named indices on the current journal the new journal, and continues operations on the new journal. Typically this involves updating the view of the named indices such that they read from a fused view of an empty index on the new journal and the last committed state of the index on the old journal.

Note: When this method returns true journal references MUST NOT be presumed to survive this method. In particular, the old journal MAY be closed out by this method and marked as read-only henceforth.

Note: The caller MUST ensure that they have an exclusive lock on the WriteExecutorService such that no task is running with write access to the live journal.

Note: The implementation MUST NOT write on the old journal - those writes will not be made restart safe by the WriteExecutorService - but it MAY write on the new journal.

Specified by:

overflow in interface IResourceManager

Returns:

The Future for the task handling post-processing of the old journal.

getIndexSegmentFile

public File getIndexSegmentFile(IndexMetadata indexMetadata)

Description copied from interface: IResourceManager

Return the file on which a new IndexSegment should be written. The file will exist but will have zero length.

Specified by:

getIndexSegmentFile in interface IResourceManager

Parameters:

indexMetadata - The index metadata.

Returns:

The file.

Throws:

UnsupportedOperationException - since overflow() is not supported.

getFederation

public IBigdataFederation getFederation()

Description copied from interface: IResourceManager

The federation whose resources are being managed.

Specified by:

getFederation in interface IResourceManager

Throws:

UnsupportedOperationException - always.

getDataService

public DataService getDataService()

Description copied from interface: IResourceManager

The local DataService whose resources are being managed.

Specified by:

getDataService in interface IResourceManager

Throws:

UnsupportedOperationException - always.

getDataServiceUUID

public UUID getDataServiceUUID()

Description copied from interface: IResourceManager

Return the UUID of the IDataService whose resources are being managed.

Specified by:

getDataServiceUUID in interface IResourceManager

Throws:

UnsupportedOperationException - always.

getIndexPartitionGone

public StaleLocatorReason getIndexPartitionGone(String name)

Always returns null since index partition moves are not supported.

Specified by:

getIndexPartitionGone in interface IResourceManager

Parameters:

name - The name of an index partition.

Returns:

The reason (split, join, or move) -or- null iff the index partition is not known to be gone.

getGlobalRowStore

public SparseRowStore getGlobalRowStore()

Description copied from interface: IIndexStore

Return the global SparseRowStore used to store named property sets.

Specified by:

getGlobalRowStore in interface IIndexStore

See Also:

GlobalRowStoreSchema

getGlobalFileSystem

public BigdataFileSystem getGlobalFileSystem()

Description copied from interface: IIndexStore

Return the global file system used to store block-structured files and their metadata and as a source and sink for map/reduce processing.

Specified by:

getGlobalFileSystem in interface IIndexStore

See Also:

BigdataFileSystem

discardCommitters

protected void discardCommitters()

Description copied from class: AbstractJournal

This method is invoked by AbstractJournal.abort() when the store must discard any hard references that it may be holding to objects registered as ICommitters.

The default implementation discards the btree mapping names to named btrees.

Subclasses MAY extend this method to discard their own committers but MUST NOT override it completely.

Overrides:

discardCommitters in class AbstractJournal

getTempStore

public TemporaryStore getTempStore()

Description copied from interface: IIndexStore

A factory for TemporaryStores. TemporaryStores are thread-safe and may be used by multiple processes at once. Old TemporaryStores are eventually retired by the factory and their storage is reclaimed once they are finalized (after they are no longer in use by any process). The decision to retire a TemporaryStore is either made implicitly, when it is no longer weakly reachable, or explicitly, when it has grown large enough that no new processes should begin using that TemporaryStore. In the latter case, the TemporaryStore will remain available to the process(es) using it and a new TemporaryStore will be allocated and made available to the caller.

It is important that processes do not hold a hard reference to a TemporaryStore beyond the end of the process as that will prevent the TemporaryStore from being finalized. Holding reference to an AbstractBTree created on a TemporaryStore is equivalent to holding a hard reference to the TemporaryStore itself since the AbstractBTree holds onto the backing IRawStore using a hard reference.

Specified by:

getTempStore in interface IIndexStore

Returns:

A TemporaryStore.

getResourceLocator

public DefaultResourceLocator getResourceLocator()

Description copied from interface: IIndexStore

Return the default locator for resources that are logical index containers (relations and relation containers).

Specified by:

getResourceLocator in interface IIndexStore

getResourceLockService

public IResourceLockService getResourceLockService()

Description copied from interface: IIndexStore

The service that may be used to acquire synchronous distributed locks without deadlock detection.

Specified by:

getResourceLockService in interface IIndexStore

getExecutorService

public ExecutorService getExecutorService()

Description copied from class: AbstractJournal

Service for running arbitrary tasks in support of IResourceLocator. There is no concurrency control associated with this service, but tasks run here may submit tasks to the ConcurrencyManager.

Specified by:

getExecutorService in interface IIndexStore

Specified by:

getExecutorService in class AbstractJournal