com.bigdata.journal
Class AbstractJournal

java.lang.Object
  com.bigdata.journal.AbstractJournal

All Implemented Interfaces:

IAtomicStore, IBTreeManager, IIndexManager, IIndexStore, IJournal, IAddressManager, IMRMW, IMROW, IRawStore, IStoreSerializer

Direct Known Subclasses:

Journal, StoreManager.ManagedJournal

public abstract class AbstractJournal
extends Object
implements IJournal

The journal is an append-only persistence capable data structure supporting atomic commit, named indices, and transactions. Writes are logically appended to the journal to minimize disk head movement.

This class is an abstract implementation of the IJournal interface that does not implement the IConcurrencyManager, IResourceManager, or ITransactionService interfaces. There are several classes which DO support all of these features, relying on the AbstractJournal for their underlying persistence store. These include:

Journal

A concrete implementation that may be used for a standalone immortal database complete with concurrency control and transaction management.

DataService

A class supporting remote clients, key-range partitioned indices, concurrency, and scale-out.

IBigdataClient

Clients connect to an IBigdataFederation, which is the basis for the scale-out architecture. There are several variants of a federation available, including:

LocalDataServiceClient

Purely local operations against a DataService with full concurrency controls and transaction management

EmbeddedClient

Operations against a collection of services running in the same JVM with full concurrency controls, transaction management, and key-range partitioned indices.

JiniClient

Operations against a collection of services running on a distributed services framework such as Jini with full concurrency controls, transaction management, and key-range partitioned indices. This is the scale-out solution.

Limitations

The IIndexStore implementation on this class is NOT thread-safe. The basic limitation is that the mutable BTree is NOT thread-safe. The getIndex(String) method exposes this mutable BTree. If you use this method to access the mutable BTree then YOU are responsible for avoiding concurrent writes on the returned object.

See IConcurrencyManager.submit(AbstractTask) for a thread-safe API that provides suitable concurrency control for both isolated and unisolated operations on named indices. Note that the use of the thread-safe API does NOT protect against applications that directly access the mutable BTree using getIndex(String).

The IRawStore interface on this class is thread-safe. However, this is a low-level API that is not used by directly by most applications. The BTree class uses this low-level API to read and write its nodes and leaves on the store. Applications generally use named indices rather than the IRawStore interface.

Note: transaction processing MAY occur be concurrent since the write set of a each transaction is written on a distinct TemporaryStore. However, without additional concurrency controls, each transaction is NOT thread-safe and MUST NOT be executed by more than one concurrent thread. Again, see IConcurrencyManager.submit(AbstractTask) for a high-concurrency API for both isolated operations (transactions) and unisolated operations. Note that the TemporaryStore backing a transaction will spill automatically from memory onto disk if the write set of the transaction grows too large.

Commit processing

The journal maintains two root blocks. Commit updates the root blocks using the Challis algorithm. (The root blocks are updated using an alternating pattern and "timestamps" are recorded at the head and tail of each root block to detect partial writes. See IRootBlockView and RootBlockView.) When the journal is backed by a disk file, the data are optionally flushed to disk on commit. If desired, the writes may be flushed before the root blocks are updated to ensure that the writes are not reordered - see Options.DOUBLE_SYNC.

Version:

$Id: AbstractJournal.java 2265 2009-10-26 12:51:06Z thompsonbry $

Author:

Bryan Thompson

TODO:

Checksums and/or record compression are currently handled on a per-BTree or other persistence capable data structure basis. It is nice to be able to choose for which indices and when ( Journal vs IndexSegment) to apply these algorithms. However, it might be nice to factor their application out a bit into a layered api - as long as the right layering is correctly re-established on load of the persistence data structure. In that view the IRawStore either computes checksums or it does not and the checksums is stored in the record, perhaps in the last 4 bytes. The checksum itself would not be visible at the IRawStore API layer., There are lots of annoying ways in which asynchronously closing the journal, e.g., using close() or shutdown() can cause exceptions to be thrown out of concurrent threads. It would be nice if we could throw a single exception that indicated that the journal had been asynchronously closed.

Field Summary

protected boolean	deleteOnClose Option set by the test suites causes the file backing the journal to be deleted when the journal is closed.
protected boolean	doubleSync Option controls whether the journal forces application data to disk before updating the root blocks.
protected ForceEnum	forceOnCommit Option controls how the journal behaves during a commit.
protected static org.apache.log4j.Logger	log Logger.
protected Properties	properties A clone of the properties used to initialize the Journal.
static int	ROOT_NAME2ADDR The index of the root address containing the address of the persistent Name2Addr mapping names to BTrees registered for the store.
File	tmpDir The directory that should be used for temporary files.

Fields inherited from interface com.bigdata.rawstore.IAddressManager

NULL

Constructor Summary

protected	AbstractJournal(Properties properties) Create or re-open a journal.
protected	AbstractJournal(Properties properties, ByteBuffer writeCache) Create or re-open a journal.

Method Summary

protected void	_close() Core implementation of immediate shutdown handles event reporting.
protected Name2Addr	_getName2Addr() Return the "live" BTree mapping index names to the last metadata record committed for the named index.
void	abort() Discards any unisolated writes since the last commitNow(long)() and also discards the unisolated (aka live) btree objects, reloading them from the current ICommitRecord on demand.
protected void	assertOpen() Assert that the store is open.
void	close() Invokes shutdownNow().
void	closeForWrites(long closeTime) Restart safe conversion of the store into a read-only store with the specified closeTime.
long	commit() Atomic commit (synchronous).
protected long	commitNow(long commitTime) An atomic commit is performed by directing each registered ICommitter to flush its state onto the store using ICommitter.handleCommit(long).
void	deleteResources() Deletes the backing file(s) (if any).
Object	deserialize(byte[] b) De-serialize an object using the Java serialization mechanisms.
Object	deserialize(byte[] b, int off, int len) De-serialize an object using the Java serialization mechanisms.
Object	deserialize(ByteBuffer buf) De-serialize an object using the Java serialization mechanisms.
void	destroy() Closes the store immediately, deletes its persistent resources, and clears any records for the store from the IGlobalLRU.
protected void	discardCommitters() This method is invoked by abort() when the store must discard any hard references that it may be holding to objects registered as ICommitters.
void	dropIndex(String name) Drops the named index.
long	ensureMinFree(long minFree) Make sure that the journal has at least the specified number of bytes of unused capacity remaining in the user extent.
protected void	finalize() Closes out the journal iff it is still open.
void	force(boolean metadata) Force the data to stable storage.
IBufferStrategy	getBufferStrategy() The delegate that implements the BufferMode.
int	getByteCount(long addr) The length of the datum in bytes.
ICommitRecord	getCommitRecord() Returns a read-only view of the most recently committed ICommitRecord containing the root addresses.
ICommitRecord	getCommitRecord(long commitTime) Return the ICommitRecord for the most recent committed state whose commit timestamp is less than or equal to timestamp.
protected CommitRecordIndex	getCommitRecordIndex() Return the current state of the index that resolves timestamps to ICommitRecords.
protected CommitRecordIndex	getCommitRecordIndex(long addr) Create or re-load the index that resolves timestamps to ICommitRecords.
ICommitRecord	getCommitRecordStrictlyGreaterThan(long commitTime) Return the first commit record whose timestamp is strictly greater than the given commitTime.
CounterSet	getCounters() Return counters reporting on various aspects of the journal.
abstract ExecutorService	getExecutorService() Service for running arbitrary tasks in support of IResourceLocator.
File	getFile() The backing file -or- null if there is no backing file for the store.
BTree	getIndex(long checkpointAddr) A canonicalizing mapping for historical BTrees.
BTree	getIndex(String name) Return the mutable view of the named index (aka the "live" or ITx.UNISOLATED index).
BTree	getIndex(String name, ICommitRecord commitRecord) Returns a read-only named index loaded from a ICommitRecord.
IIndex	getIndex(String name, long commitTime) Return a view of the named index as of the specified timestamp.
long	getLastCommitTime() The database wide timestamp of the most recent commit on the store or 0L iff there have been no commits.
abstract AbstractLocalTransactionManager	getLocalTransactionManager() Return the object providing the AbstractLocalTransactionManager for this journal.
long	getMaximumExtent() The maximum extent before a commit() will #overflow().
int	getMaxRecordSize() The maximum length of a record that may be written on the store.
IIndex	getName2Addr() A read-only view of the Name2Addr object mapping index names to the most recent committed Name2Addr.Entry for the named index.
IIndex	getName2Addr(long commitTime) Return a read-only view of the Name2Addr object as of the specified commit time.
long	getOffset(long addr) The offset on the store at which the datum is stored.
int	getOffsetBits()
Properties	getProperties() A copy of the properties used to initialize this journal.
protected static String	getProperty(Properties properties, String name, String defaultValue) Resolves the property value (static variant for ctor initialization).
protected String	getProperty(String name, String defaultValue) Resolves the property value.
protected <E> E	getProperty(String name, String defaultValue, IValidator<E> validator) Resolves, parses, and validates the property value.
IResourceMetadata	getResourceMetadata() A description of this store in support of the scale-out architecture.
long	getRootAddr(int index) The last address stored in the specified root slot as of the last committed state of the store.
IRootBlockView	getRootBlockView() Return a read-only view of the current root block.
UUID	getUUID() Return the UUID which identifies this IRawStore.
static ByteBuffer	getWriteCache(Properties properties) Used to allocate the ByteBuffer for the BufferMode.Disk write cache.
boolean	isFullyBuffered() True iff the store is fully buffered (all reads are against memory).
boolean	isOpen() true iff the store is open.
boolean	isReadOnly() Return true if the journal was opened in a read-only mode
boolean	isStable() True iff backed by stable storage.
void	packAddr(DataOutput out, long addr) Pack the address onto the output stream.
ByteBuffer	read(long addr) Read the data (unisolated).
void	registerIndex(IndexMetadata metadata) Registers a named index.
BTree	registerIndex(String name) Deprecated. This is only used by the test suites.
BTree	registerIndex(String name, BTree ndx) Registers a named index (core impl).
BTree	registerIndex(String name, IndexMetadata metadata) Registers a named index.
void	rollback() Rollback a journal to its previous commit point.
byte[]	serialize(Object obj) Serialize an object using the Java serialization mechanisms.
void	setCommitter(int rootSlot, ICommitter committer) Set a persistence capable data structure for callback during the commit protocol.
protected void	setupCommitters() Invoked when a journal is first created, re-opened, or when the committers have been discarded.
void	shutdown() Shutdown the journal (running tasks will run to completion, but no new tasks will start).
void	shutdownNow() Immediate shutdown (running tasks are canceled rather than being permitted to complete).
long	size() The #of application data bytes written on the store (does not count any headers or root blocks that may exist for the store).
long	toAddr(int nbytes, long offset) Converts a byte count and offset into a long integer.
String	toString(long addr) A human readable representation of the address.
void	truncate() Truncate the backing buffer such that there is no remaining free space in the journal.
long	unpackAddr(DataInput in) Unpack the address from the input stream.
protected void	validateIndexMetadata(String name, IndexMetadata metadata) Provides an opportunity to validate some aspects of the IndexMetadata for an index partition.
long	write(ByteBuffer data) Write the data (unisolated).

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.bigdata.journal.IIndexStore

getGlobalFileSystem, getGlobalRowStore, getResourceLocator, getResourceLockService, getTempStore

Field Detail

log

protected static final org.apache.log4j.Logger log

Logger.

ROOT_NAME2ADDR

public static final transient int ROOT_NAME2ADDR

The index of the root address containing the address of the persistent Name2Addr mapping names to BTrees registered for the store.

See Also:

Constant Field Values

properties

protected final Properties properties

A clone of the properties used to initialize the Journal.

tmpDir

public final File tmpDir

The directory that should be used for temporary files.

doubleSync

protected final boolean doubleSync

Option controls whether the journal forces application data to disk before updating the root blocks.

forceOnCommit

protected final ForceEnum forceOnCommit

Option controls how the journal behaves during a commit.

deleteOnClose

protected final boolean deleteOnClose

Option set by the test suites causes the file backing the journal to be deleted when the journal is closed.

Constructor Detail

AbstractJournal

protected AbstractJournal(Properties properties)

Create or re-open a journal.

Note: Creating a new journal registers some internal indices but does NOT perform a commit. Those indices will become restart safe with the first commit.

Parameters:

properties - The properties as defined by Options.

Throws:

RuntimeException - If there is a problem when creating, opening, or reading from the journal file.

See Also:

Options

AbstractJournal

protected AbstractJournal(Properties properties,
                          ByteBuffer writeCache)

Create or re-open a journal.

Note: Creating a new journal registers some internal indices but does NOT perform a commit. Those indices will become restart safe with the first commit.

Parameters:

properties - The properties as defined by Options.

writeCache - When non-null and when BufferMode.Disk is selected, this ByteBuffer will be used as the write cache. This allows the same write cache to be used by the next journal when the current journal overflows. This is necessitated by JVM bug 6210541 which describes a failure by releaseTemporaryDirectBuffer() to release temporary direct ByteBuffers that are allocated for channel IO.

Throws:

RuntimeException - If there is a problem when creating, opening, or reading from the journal file.

See Also:

Options

Method Detail

_getName2Addr

protected Name2Addr _getName2Addr()

Return the "live" BTree mapping index names to the last metadata record committed for the named index. The keys are index names (unicode strings). The values are the names and the last known address of the named btree.

The "live" name2addr index is required for unisolated writers regardless whether they are adding an index, dropping an index, or just recovering the "live" version of an existing named index.

Operations that read on historical Name2Addr objects can of course be concurrent. Those objects are loaded from an ICommitRecord. See getIndex(String, ICommitRecord).

Note: the "live" Name2Addr index is a mutable BTree. All access to the object MUST be synchronized on that object.

getName2Addr

public IIndex getName2Addr()

A read-only view of the Name2Addr object mapping index names to the most recent committed Name2Addr.Entry for the named index. The keys are index names (unicode strings). The values are Name2Addr.Entrys containing the names, commitTime, and last known Checkpoint address of the named BTree on the Journal.

getName2Addr

public IIndex getName2Addr(long commitTime)

Return a read-only view of the Name2Addr object as of the specified commit time.

Parameters:

commitTime - A commit time.

Returns:

The read-only view -or- null if there is no commit record for that commitTime.

See Also:

getName2Addr()

getMaximumExtent

public final long getMaximumExtent()

The maximum extent before a commit() will #overflow(). In practice, overflow tries to trigger before this point in order to avoid extending the journal.

See Also:

Options.MAXIMUM_EXTENT

getWriteCache

public static ByteBuffer getWriteCache(Properties properties)

Used to allocate the ByteBuffer for the BufferMode.Disk write cache.

Parameters:

properties - The properties that will be used to configure the journal.

Returns:

The write cache buffer -or- null iff no buffer is required (either because the selected BufferMode does not support a write cache, because the journal would be opened in a read-only mode, or because the configured write cache capacity was ZERO(0)).

See Also:

Options.BUFFER_MODE, Options.READ_ONLY, Options.WRITE_CACHE_CAPACITY

getProperty

protected static String getProperty(Properties properties,
                                    String name,
                                    String defaultValue)

Resolves the property value (static variant for ctor initialization).

See Also:

Configuration.getProperty(IIndexManager, Properties, String, String, String)

getProperty

protected String getProperty(String name,
                             String defaultValue)

Resolves the property value.

See Also:

Configuration.getProperty(IIndexManager, Properties, String, String, String)

getProperty

protected <E> E getProperty(String name,
                            String defaultValue,
                            IValidator<E> validator)

Resolves, parses, and validates the property value.

Parameters:

name - The property name.

defaultValue - The default value.

Returns:

getProperties

public final Properties getProperties()

Description copied from interface: IJournal

A copy of the properties used to initialize this journal.

Specified by:

getProperties in interface IJournal

TODO:

consider making the properties restart safe so that they can be read from the journal. This will let some properties be specified on initialization while letting others default or be overridden on restart. This is trivially accomplished by dedicating a root slot to a Properties object, or a flattened Properties object serialized as key-value pairs, in which case the data could just be loaded into a btree and the btree api could be used to change the persistent properties as necessary.

getBufferStrategy

public final IBufferStrategy getBufferStrategy()

The delegate that implements the BufferMode.

Note: this method MUST NOT check to see whether the journal is open since we need to use it if we want to invoke IRawStore.deleteResources() and we can only invoke that method once the journal is closed.

getExecutorService

public abstract ExecutorService getExecutorService()

Service for running arbitrary tasks in support of IResourceLocator. There is no concurrency control associated with this service, but tasks run here may submit tasks to the ConcurrencyManager.

Specified by:

getExecutorService in interface IIndexStore

shutdown

public void shutdown()

Shutdown the journal (running tasks will run to completion, but no new tasks will start).

Note: You SHOULD use this method rather than close() for normal shutdown of the journal.

Specified by:

shutdown in interface IJournal

See Also:

shutdownNow()

shutdownNow

public void shutdownNow()

Immediate shutdown (running tasks are canceled rather than being permitted to complete).

Specified by:

shutdownNow in interface IJournal

See Also:

shutdown()

finalize

protected void finalize()
                 throws Exception

Closes out the journal iff it is still open.

Overrides:

finalize in class Object

Throws:

Exception

getCounters

public CounterSet getCounters()

Return counters reporting on various aspects of the journal.

Specified by:

getCounters in interface IRawStore

getFile

public final File getFile()

Description copied from interface: IRawStore

The backing file -or- null if there is no backing file for the store.

Specified by:

getFile in interface IRawStore

_close

protected void _close()

Core implementation of immediate shutdown handles event reporting.

deleteResources

public void deleteResources()

Deletes the backing file(s) (if any).

Note: This is the core implementation of delete and handles event reporting.

Specified by:

deleteResources in interface IRawStore

Throws:

IllegalStateException - if the journal is open.

truncate

public void truncate()

Truncate the backing buffer such that there is no remaining free space in the journal.

Note: The caller MUST have exclusive write access to the journal. When the ConcurrencyManager is used, that means that the caller MUST have an exclusive lock on the WriteExecutorService.

ensureMinFree

public long ensureMinFree(long minFree)

Make sure that the journal has at least the specified number of bytes of unused capacity remaining in the user extent.

Note: You need an exclusive write lock on the journal to extend it.

Parameters:

minFree - The minimum #of bytes free for the user extent.

Returns:

The #of bytes of free space remaining in the user extent.

closeForWrites

public void closeForWrites(long closeTime)

Restart safe conversion of the store into a read-only store with the specified closeTime.

This implementation sets the "closeTime" on the root block such that the journal will no longer accept writes, flushes all buffered writes, and releases any write cache buffers since they will no longer be used. This method is normally used when one journal is being closed out for writes during synchronous overflow processing and new writes will be buffered on a new journal. This has advantages over closing the journal directly including that it does not disturb concurrent readers.

Note: The caller MUST have exclusive write access to the journal. When the ConcurrencyManager is used, that means that the caller MUST have an exclusive lock on the WriteExecutorService.

Note: This does NOT perform a commit - any uncommitted writes will be discarded.

Throws:

IllegalStateException - If there are no commits on the journal.

TODO:

There should also be an option to convert a journal from BufferMode.Direct to BufferMode.Disk. We would want to do that not when the journal is sealed but as soon as asynchronous overflow processing is done. Ideally this will not require us to close and reopen the journal since that will disturb concurrent readers.

close

public void close()

Invokes shutdownNow().

Specified by:

close in interface IRawStore

destroy

public void destroy()

Description copied from interface: IRawStore

Closes the store immediately, deletes its persistent resources, and clears any records for the store from the IGlobalLRU.

Specified by:

destroy in interface IIndexStore

Specified by:

destroy in interface IRawStore

See Also:

IRawStore.deleteResources()

assertOpen

protected void assertOpen()

Assert that the store is open.

Note: You can see an IllegalStateException thrown out of here if there are tasks running during shutdown() and one of the various task services times out while awaiting termination. Such exceptions are normal since the store was closed asynchronously while task(s) were still running.

Throws:

IllegalStateException - if the store is closed.

getUUID

public final UUID getUUID()

Description copied from interface: IRawStore

Return the UUID which identifies this IRawStore. This supports both IRawStore.getResourceMetadata() and the LRUNexus.

Specified by:

getUUID in interface IRawStore

getResourceMetadata

public final IResourceMetadata getResourceMetadata()

Description copied from interface: IRawStore

A description of this store in support of the scale-out architecture.

Specified by:

getResourceMetadata in interface IRawStore

isOpen

public boolean isOpen()

Description copied from interface: IRawStore

true iff the store is open.

Specified by:

isOpen in interface IRawStore

Returns:

true iff the store is open.

isReadOnly

public boolean isReadOnly()

Return true if the journal was opened in a read-only mode

Specified by:

isReadOnly in interface IRawStore

isStable

public boolean isStable()

Description copied from interface: IRawStore

True iff backed by stable storage.

Specified by:

isStable in interface IRawStore

isFullyBuffered

public boolean isFullyBuffered()

Description copied from interface: IRawStore

True iff the store is fully buffered (all reads are against memory). Implementations MAY change the value returned by this method over the life cycle of the store, e.g., to conserve memory a store may drop or decrease its buffer if it is backed by disk.

Note: This does not guarantee that the OS will not swap the buffer onto disk.

Specified by:

isFullyBuffered in interface IRawStore

getRootBlockView

public final IRootBlockView getRootBlockView()

Description copied from interface: IAtomicStore

Return a read-only view of the current root block.

Specified by:

getRootBlockView in interface IAtomicStore

Returns:

The current root block.

getLastCommitTime

public final long getLastCommitTime()

Description copied from interface: IIndexStore

The database wide timestamp of the most recent commit on the store or 0L iff there have been no commits. In a local database, this timestamp is generated by a local timestamp service. In a distributed database, this timestamp is generated by a shared timestamp service. The timestamps returned by this method are strictly increasing for a given store and for a given database.

This method is useful if you plan to issue a series of read-consistent requests against the most current commit point of the database.

Specified by:

getLastCommitTime in interface IIndexStore

Returns:

The timestamp of the most recent commit on the store or 0L iff there have been no commits.

See Also:

IRootBlockView.getLastCommitTime()

setCommitter

public final void setCommitter(int rootSlot,
                               ICommitter committer)

Set a persistence capable data structure for callback during the commit protocol.

Note: the committers must be reset after restart or whenever the committers are discarded (the committers are themselves transient objects).

Specified by:

setCommitter in interface IAtomicStore

Parameters:

rootSlot - The slot in the root block where the address of the ICommitter will be recorded.

committer - The commiter.

abort

public void abort()

Discards any unisolated writes since the last commitNow(long)() and also discards the unisolated (aka live) btree objects, reloading them from the current ICommitRecord on demand.

Note: The WriteExecutorService handles commit group and uses an index Checkpoint strategy so that it is able to abort individual tasks simply by discarding their changes and without interrupting concurrent writers. An abort() is therefore an action of last resort and is generally triggered by things such as running out of disk space or memory within the JVM.

If a Threads is interrupted in the midst of an IO operation on a Channel then the channel will be asynchronously closed by the JDK. Since some IBufferStrategys use a FileChannel to access the backing store, this means that we need to re-open the backing store transparently so that we can continue operations after the commit group was aborted. This is done automatically when we re-load the current ICommitRecord from the root blocks of the store.

Specified by:

abort in interface IAtomicStore

rollback

public void rollback()

Rollback a journal to its previous commit point.

Note: You MUST have an exclusive write lock on the journal.

Note: To restore the last root block we copy the alternative root block over the current root block. That gives us two identical root blocks and restores us to the root block that was in effect before the last commit.

getLocalTransactionManager

public abstract AbstractLocalTransactionManager getLocalTransactionManager()

Return the object providing the AbstractLocalTransactionManager for this journal.

commit

public long commit()

Description copied from interface: IAtomicStore

Atomic commit (synchronous).

Note: if the commit fails (by throwing any kind of exception) then you MUST invoke IAtomicStore.abort() to abandon the current write set.

Specified by:

commit in interface IAtomicStore

Returns:

The timestamp assigned to the ICommitRecord -or- 0L if there were no data to commit.

commitNow

protected long commitNow(long commitTime)

An atomic commit is performed by directing each registered ICommitter to flush its state onto the store using ICommitter.handleCommit(long). The address returned by that method is the address from which the ICommitter may be reloaded (and its previous address if its state has not changed). That address is saved in the ICommitRecord under the index for which that committer was registered. We then force the data to stable store, update the root block, and force the root block and the file metadata to stable store.

Parameters:

commitTime - The commit time either of a transaction or of an unisolated commit. Note that when mixing isolated and unisolated commits you MUST use the same ITimestampService for both purposes.

Returns:

The timestamp assigned to the commit record -or- 0L if there were no data to commit.

force

public void force(boolean metadata)

Description copied from interface: IRawStore

Force the data to stable storage. While this is NOT sufficient to guarantee an atomic commit, the data must be forced to disk as part of an atomic commit protocol.

Specified by:

force in interface IRawStore

Parameters:

metadata - If true, then force both the file contents and the file metadata to disk.

size

public long size()

Description copied from interface: IRawStore

The #of application data bytes written on the store (does not count any headers or root blocks that may exist for the store).

Specified by:

size in interface IRawStore

write

public long write(ByteBuffer data)

Description copied from interface: IRawStore

Write the data (unisolated).

Specified by:

write in interface IRawStore

Parameters:

data - The data. The bytes from the current Buffer.position() to the Buffer.limit() will be written and the Buffer.position() will be advanced to the Buffer.limit() . The caller may subsequently modify the contents of the buffer without changing the state of the store (i.e., the data are copied into the store).

Returns:

A long integer formed that encodes both the offset from which the data may be read and the #of bytes to be read. See IAddressManager.

read

public ByteBuffer read(long addr)

Description copied from interface: IRawStore

Read the data (unisolated).

Specified by:

read in interface IRawStore

Parameters:

addr - A long integer that encodes both the offset from which the data will be read and the #of bytes to be read. See IAddressManager.toAddr(int, long).

Returns:

The data read. The buffer will be flipped to prepare for reading (the position will be zero and the limit will be the #of bytes read).

getRootAddr

public final long getRootAddr(int index)

Description copied from interface: IAtomicStore

The last address stored in the specified root slot as of the last committed state of the store.

Specified by:

getRootAddr in interface IAtomicStore

Parameters:

index - The index of the root address to be retrieved.

Returns:

The address stored at that index.

getCommitRecord

public ICommitRecord getCommitRecord()

Returns a read-only view of the most recently committed ICommitRecord containing the root addresses.

Returns:

The current ICommitRecord and never null.

discardCommitters

protected void discardCommitters()

This method is invoked by abort() when the store must discard any hard references that it may be holding to objects registered as ICommitters.

The default implementation discards the btree mapping names to named btrees.

Subclasses MAY extend this method to discard their own committers but MUST NOT override it completely.

setupCommitters

protected void setupCommitters()

Invoked when a journal is first created, re-opened, or when the committers have been discarded.

The basic implementation sets up the btree that is responsible for resolving named btrees.

Subclasses may extend this method to setup their own committers.

getCommitRecordIndex

protected CommitRecordIndex getCommitRecordIndex()

Return the current state of the index that resolves timestamps to ICommitRecords.

Note: The returned object is NOT safe for concurrent operations and is NOT systematically protected by the use of synchronized blocks within this class.

Returns:

The CommitRecordIndex.

TODO:

If you need access to this object in an outside class consider using getRootBlockView(), IRootBlockView.getCommitRecordIndexAddr(), and getCommitRecord(long) to obtain a distinct instance suitable for read-only access.

getCommitRecordIndex

protected CommitRecordIndex getCommitRecordIndex(long addr)

Create or re-load the index that resolves timestamps to ICommitRecords.

Note: The returned object is NOT cached. When addr is non-IAddressManager.NULL, each invocation will return a distinct CommitRecordIndex object. This behavior is partly for historical reasons but it does serve to protect the live CommitRecordIndex from outside access. This is important since access to the live CommitRecordIndex is NOT systematically protected by synchronized within this class.

Parameters:

addr - The root address of the index -or- 0L if the index has not been created yet.

Returns:

The CommitRecordIndex for that address or a new index if 0L was specified as the address.

See Also:

_commitRecordIndex

getCommitRecord

public ICommitRecord getCommitRecord(long commitTime)

Description copied from interface: IAtomicStore

Return the ICommitRecord for the most recent committed state whose commit timestamp is less than or equal to timestamp. This is used by a transaction to locate the committed state that is the basis for its operations.

Specified by:

getCommitRecord in interface IAtomicStore

Parameters:

commitTime - The timestamp of interest.

Returns:

The ICommitRecord for the most recent committed state whose commit timestamp is less than or equal to timestamp -or- null iff there are no ICommitRecords that satisfy the probe.

TODO:

the CommitRecordIndex is a possible source of thread contention since transactions need to use this code path in order to locate named indices but the WriteExecutorService can also write on this index. I have tried some different approaches to handling this.

getCommitRecordStrictlyGreaterThan

public ICommitRecord getCommitRecordStrictlyGreaterThan(long commitTime)

Return the first commit record whose timestamp is strictly greater than the given commitTime.

Parameters:

commitTime - The commit time.

Returns:

The commit record -or- null if there is no commit record whose timestamp is strictly greater than commitTime.

getIndex

public IIndex getIndex(String name,
                       long commitTime)

Description copied from interface: IIndexStore

Return a view of the named index as of the specified timestamp.

Specified by:

getIndex in interface IIndexStore

Parameters:

commitTime - A positive timestamp (a possible commit time on the store).

name - The index name.

Returns:

The index or null iff there is no index registered with that name for that timestamp.

Throws:

UnsupportedOperationException - If you pass in ITx.UNISOLATED, ITx.READ_COMMITTED, or a timestamp that corresponds to a read-write transaction since those are not "commit times".

getIndex

public BTree getIndex(String name,
                      ICommitRecord commitRecord)

Returns a read-only named index loaded from a ICommitRecord. The BTree will be marked as read-only, it will NOT permit writes, and BTree#getLastCommitTime(long) will report the value associated with Name2Addr.Entry.commitTime for the historical Name2Addr instance for that ICommitRecord.

Returns:

The named index -or- null iff the named index did not exist as of that commit record.

getIndex

public final BTree getIndex(long checkpointAddr)

A canonicalizing mapping for historical BTrees.

Note: This method imposes a canonicalizing mapping and ensures that there will be at most one instance of the historical index at a time. This guarentee is used to facilitate buffer management. Writes on indices returned by this method are NOT allowed.

Note: This method marks the BTree as read-only but does not set BTree.setLastCommitTime(long) since it does not have access to the Name2Addr.Entry.commitTime, only the BTrees checkpointAddr and Checkpoint record. See getIndex(String, ICommitRecord) which does set BTree.setLastCommitTime(long).

Note: The canonicalizing mapping for unisolated BTrees is maintained by the ITx.UNISOLATED Name2Addr instance.

Parameters:

checkpointAddr - The address of the Checkpoint record for the BTree.

Returns:

The BTree loaded from that Checkpoint.

See Also:

Options.HISTORICAL_INDEX_CACHE_CAPACITY

registerIndex

public BTree registerIndex(String name)

Deprecated. This is only used by the test suites.

Registers a named index. Once registered the index will participate in atomic commits.

Note: A named index must be registered outside of any transaction before it may be used inside of a transaction.

Note: You MUST commit() before the registered index will be either restart-safe or visible to new transactions.

Parameters:

name - The name that can be used to recover the index.

Returns:

The object that would be returned by getIndex(String).

Throws:

IllegalStateException - if there is an index already registered under that name.

See Also:

Options#BTREE_BRANCHING_FACTOR

registerIndex

public void registerIndex(IndexMetadata metadata)

Registers a named index. Once registered the index will participate in atomic commits.

Note: A named index must be registered outside of any transaction before it may be used inside of a transaction.

Note: You MUST commit() before the registered index will be either restart-safe or visible to new transactions.

Specified by:

registerIndex in interface IIndexManager

Parameters:

metadata - The metadata describing the index.

registerIndex

public BTree registerIndex(String name,
                           IndexMetadata metadata)

Registers a named index. Once registered the index will participate in atomic commits.

Note: A named index must be registered outside of any transaction before it may be used inside of a transaction.

Note: You MUST commit() before the registered index will be either restart-safe or visible to new transactions.

Specified by:

registerIndex in interface IBTreeManager

Parameters:

name - The name that can be used to recover the index.

metadata - The metadata describing the index.

Returns:

The object that would be returned by IBTreeManager.getIndex(String).

validateIndexMetadata

protected void validateIndexMetadata(String name,
                                     IndexMetadata metadata)

Provides an opportunity to validate some aspects of the IndexMetadata for an index partition.

registerIndex

public BTree registerIndex(String name,
                           BTree ndx)

Registers a named index (core impl). Once registered the index will participate in atomic commits.

Note: A named index must be registered outside of any transaction before it may be used inside of a transaction.

Note: You MUST commit() before the registered index will be either restart-safe or visible to new transactions.

Specified by:

registerIndex in interface IBTreeManager

Parameters:

name - The name that can be used to recover the index.

ndx - The btree.

Returns:

The object that would be returned by IBTreeManager.getIndex(String).

dropIndex

public void dropIndex(String name)

Drops the named index. The index will no longer participate in atomic commits and will not be visible to new transactions. Resources are NOT reclaimed on the AbstractJournal (it is an immortal store) and historical states of the index will continue to be accessible.

Specified by:

dropIndex in interface IIndexManager

Parameters:

name - The name of the index to be dropped.

getIndex

public BTree getIndex(String name)

Return the mutable view of the named index (aka the "live" or ITx.UNISOLATED index). This object is NOT thread-safe. You MUST NOT write on this index unless you KNOW that you are the only writer. See ConcurrencyManager, which handles exclusive locks for ITx.UNISOLATED indices.

Specified by:

getIndex in interface IBTreeManager

Parameters:

name - The name of the index.

Returns:

The mutable view of the index.

See Also:

getIndex(String, long)

deserialize

public final Object deserialize(byte[] b,
                                int off,
                                int len)

Description copied from interface: IStoreSerializer

De-serialize an object using the Java serialization mechanisms.

Specified by:

deserialize in interface IStoreSerializer

Parameters:

b - A byte[] containing a serialized object.

off - The offset of the first byte to de-serialize.

len - The #of bytes in the object record.

Returns:

The de-serialized object.

deserialize

public final Object deserialize(byte[] b)

Description copied from interface: IStoreSerializer

De-serialize an object using the Java serialization mechanisms.

Specified by:

deserialize in interface IStoreSerializer

Parameters:

b - A byte[] containing a serialized object.

Returns:

The de-serialized object.

deserialize

public final Object deserialize(ByteBuffer buf)

Description copied from interface: IStoreSerializer

De-serialize an object using the Java serialization mechanisms.

Specified by:

deserialize in interface IStoreSerializer

Parameters:

buf - A buffer containing a serialized object. The bytes from Buffer.position() to Buffer.limit() will be de-serialized and the position will be advanced to the limit.

Returns:

The de-serialized object.

serialize

public final byte[] serialize(Object obj)

Description copied from interface: IStoreSerializer

Serialize an object using the Java serialization mechanisms.

Specified by:

serialize in interface IStoreSerializer

Parameters:

obj - A Serializable object.

Returns:

The serialized object state.

getOffset

public final long getOffset(long addr)

Description copied from interface: IAddressManager

The offset on the store at which the datum is stored. While this is often the offset of a byte into a file on disk, implementations MAY assign offsets using other strategies.

Specified by:

getOffset in interface IAddressManager

Parameters:

addr - The opaque identifier that is the within store locator for some datum.

Returns:

The offset of that datum.

getByteCount

public final int getByteCount(long addr)

Description copied from interface: IAddressManager

The length of the datum in bytes.

Specified by:

getByteCount in interface IAddressManager

Parameters:

addr - The opaque identifier that is the within store locator for some datum.

Returns:

The offset of that datum.

packAddr

public final void packAddr(DataOutput out,
                           long addr)
                    throws IOException

Description copied from interface: IAddressManager

Pack the address onto the output stream.

Specified by:

packAddr in interface IAddressManager

Parameters:

out - The output stream.

addr - The opaque identifier that is the within store locator for some datum.

Throws:

IOException

toAddr

public final long toAddr(int nbytes,
                         long offset)

Description copied from interface: IAddressManager

Converts a byte count and offset into a long integer.

Specified by:

toAddr in interface IAddressManager

Parameters:

nbytes - The byte count.

offset - The byte offset.

Returns:

The long integer.

toString

public final String toString(long addr)

Description copied from interface: IAddressManager

A human readable representation of the address.

Specified by:

toString in interface IAddressManager

Parameters:

addr - The opaque identifier that is the within store locator for some datum.

Returns:

A human readable representation.

unpackAddr

public final long unpackAddr(DataInput in)
                      throws IOException

Description copied from interface: IAddressManager

Unpack the address from the input stream.

Specified by:

unpackAddr in interface IAddressManager

Parameters:

in - The input stream.

Returns:

The opaque identifier that is the within store locator for some datum.

Throws:

IOException

getOffsetBits

public final int getOffsetBits()

getMaxRecordSize

public final int getMaxRecordSize()

The maximum length of a record that may be written on the store.