com.bigdata.btree
Class IndexMetadata

java.lang.Object
  com.bigdata.btree.IndexMetadata

All Implemented Interfaces:

IKeyBuilderFactory, Externalizable, Serializable, Cloneable

Direct Known Subclasses:

HTreeIndexMetadata, MetadataIndex.MetadataIndexMetadata, Stream.StreamIndexMetadata

public class IndexMetadata
extends Object
implements Serializable, Externalizable, Cloneable, IKeyBuilderFactory

The persistent and mostly immutable metadata for a AbstractBTree. This class allows you to configured several very important aspects of the B+Tree (and other persistence capable data structures) behavior. Read on.

An instance of this class is required in order to create a BTree or IndexSegment. Further, when registering a scale-out index you will first create an instance of this class that will serve as the metadata template for all index resources which are part of that scale-out index.

Delete markers, version timestamps, and isolatable indices

By default a BTree does not maintain delete markers and a request to delete an entry under a key will cause that entry to be removed from the live version of the index. However, such indices do not support "overflow" (they can not be evicted onto read-only IndexSegments) and as such they do not support scale-out).

The SparseRowStore handles a "delete" of a property value by writing a null value under a key and does NOT require the use of index entry delete markers, even in scale-out deployments. A compacting merge of a SparseRowStore applies a history policy based on a consideration of the timestamped property values, including values bound to a null.

Delete markers combined with an ordered set of index resources is sufficient to support all features of range-partitioned indices, including compacting merge. Given three index resources {A,B,C} for a single index partition, the order over the resources gives us the guarantee that any index entry in A will be more recent than any index enty in B or C. So when reading a fused view we always stop once we have an index entry for a key, even if that entry has the deleted flag set.

Delete markers occupy very little space in the leaf data structure (one bit each), however when they are used a deleted index entry is NOT removed from the index. Instead, the key remains in the leaf paired to a delete bit and a null value (or simply elided). These "deleted" entries can only be removed from the index by a compacting merge. When transactional isolation is used, the criteria for removing deleted entries are stronger - they must no longer be visible to any active or allowable transaction as determined by the transaction manager, see below for more on this.

Transaction isolation requires delete markers plus version timestamps. The version timestamps in the unisolated index (the live index) give the timestamp of the commit during which the index entry was last updated. The timestamp in the write set of transaction is copy from the index view corresponding to the ground state of the transaction the first time that index entry is overwritten within that transaction (there is a special case when the index entry was not pre-existing - we assign the start time of the transaction in that case so when we validate we are only concerned that the entry is either not found (never written) or that the entry exists with the same timestamp - other conditions are write-write conflicts). On commit we write the commit time on the updated index entries in the unisolated index.

History policies and timestamps

There are in fact two kinds of timestamps in use - isolatable indices place a timestamp on the index entry itself while the SparseRowStore places a timestamp in the key. One consequence of this is that it is very efficient to read historical data from the SparseRowStore since the data are right there in key order. On the other hand, reading historical data from an isolatable index requires reading from each historical commit state of that index which is not interest (this is NOT efficient). This is why the SparseRowStore design places timestamps in the key - so that the application can efficiently read both the current and historical property values within a logical row.

Regardless of whether the timestamp is on the index entry (as it always is for isolatable indices) or in the key (SparseRowStore), the existence of timestamps makes it possible for an application to specify a history policy governing when property values will be deleted.

When an index participates in transactions the transaction manager manages the life cycle of overwritten and deleted index entries (and the resources on which the indices exist). This is done by preserving such data until no transaction exists that can read from those resources. Unless an immortal store is desired, the "purge time" is set at a time no more recent than the earliest fully isolated transaction (either a read-only tx as of the start time of the tx or a read-write tx as of its start time). The role of a "history policy" with transactions is therefore how much history to buffer between the earliest running tx and the chosen "purge time". When the transaction manager updates the "purge time" it notifies the journal/data services. Resources having no data later than the purge time may be deleted and SHOULD NOT be carried forward when building new index segments.

History policies for non-transactional indices are somewhat different. An scale-out index without timestamps will buffer historical data only until the next compacting merge of a given index partition. The compacting merge uses the fused view of the resources comprising the index partition and only writes out the undeleted index entries.

If an application instead chooses to use timestamps in a non-transactional index then (a) timestamps must be assigned by either the client or the data service; and (b) applications can specify a history policy where data older than a threshold time (but not #of versions) will be eradicated. This approach is possible, but not well-supported in the higher level APIs.

The SparseRowStore design is more flexible since it allows (a) fast access to historical property values for the same "row"; and (b) history policies that may be specified in terms of the #of versions, the age of a datum, and that can keep at least N versions of a datum. This last point is quite important as it allows you to retain the entirety of the most current revision of the logical row, even when some datums are much older than others.

Serialization

Note: Derived classes SHOULD extend the Externalizable interface and explicitly manage serialization versions so that their metadata may evolve in a backward compatible manner.

Version:

$Id: IndexMetadata.java 6372 2012-06-29 20:39:41Z thompsonbry $

Author:

Bryan Thompson

See Also:

Serialized Form

TODO:

Make sure that metadata for index partition "zones" propagates with the partition metadata so that appropriate policies are enforcable locally (access control, latency requirements, replication, purging of historical deleted versions, etc)., add optional property containing IndexMetadata to be used as of the next overflow so that people can swap out key and value serializers and the like during overflow operations. the conops is that you map the new metadata over the index partitions and either lazily or eagerly overflow is triggered and the resulting BTree and IndexSegment objects will begin to use the new key / value serializers, etc. while the existing objects will still have their old key/val serializers and therefore can still be read.

Nested Class Summary

static interface

IndexMetadata.Options Options and their defaults for the com.bigdata.btree package and the BTree and IndexSegment classes.

Constructor Summary

IndexMetadata()
De-serialization constructor only - DO NOT use this ctor for creating a new instance! It will result in a thrown exception, typically from firstCheckpoint().

IndexMetadata(IIndexManager indexManager, Properties properties, String namespace, UUID indexUUID, IndexTypeEnum indexType)
Constructor used to configure a new named B+Tree.

IndexMetadata(String name, UUID indexUUID)
Constructor used to configure a new named BTree.

IndexMetadata(UUID indexUUID)
Constructor used to configure a new unnamed B+Tree.

Method Summary

IndexMetadata	clone() Makes a copy of the persistent data, clearing the address of the IndexMetadata record on the cloned copy.
Checkpoint	firstCheckpoint() Create an initial Checkpoint for a new persistence capable data structure described by this metadata record.
AsynchronousIndexWriteConfiguration	getAsynchronousIndexWriteConfiguration() The asynchronous index write API configuration for this index.
void	getAsynchronousIndexWriteConfiguration(AsynchronousIndexWriteConfiguration newVal) Set the asynchronous index write API configuration for this index.
BloomFilterFactory	getBloomFilterFactory() Return the bloom filter factory.
int	getBranchingFactor() The branching factor for a mutable BTree.
String	getBTreeClassName() The name of a class derived from BTree that will be used to re-load the index.
IRecordCompressorFactory	getBtreeRecordCompressorFactory() Return the record-level compression provider for a BTree (may be null, which implies no compression).
String	getCheckpointClassName() The name of the Checkpoint class used by the index.
IConflictResolver	getConflictResolver() The optional object for handling write-write conflicts.
boolean	getDeleteMarkers() When true the index will write a delete marker when an attempt is made to delete the entry under a key.
int	getIndexSegmentBranchingFactor() The branching factor used when building an IndexSegment (default is 4096).
boolean	getIndexSegmentBufferNodes() Return true iff the nodes region for the IndexSegment should be fully buffered by the IndexSegmentStore.
IRecordCompressorFactory	getIndexSegmentRecordCompressorFactory() Return the record-level compression provider for an IndexSegment (may be null, which implies no compression).
IndexTypeEnum	getIndexType() The type of the associated persistence capable data structure.
UUID	getIndexUUID() The unique identifier for the (scale-out) index whose data is stored in this B+Tree data structure.
UUID	getInitialDataServiceUUID() The UUID of the DataService on which the first partition of the scale-out index should be created.
IKeyBuilder	getKeyBuilder() Factory for thread-safe IKeyBuilder objects for use by ITupleSerializer.serializeKey(Object) and possibly others.
int	getMaxRecLen() When getRawRecords() returns true, this method returns the maximum byte length of a byte[] value which may be stored in a B+Tree leaf (default IndexMetadata.Options.MAX_REC_LEN.
long	getMetadataAddr() Address that can be used to read this metadata record from the store.
String	getName() The name associated with the index -or- null iff the index is not named (internal indices are generally not named while application indices are always named).
IRabaCoder	getNodeKeySerializer() Object used to code (compress) the keys in a node.
IOverflowHandler	getOverflowHandler() An optional object that may be used to inspect, and possibly operate on, each index entry as it is copied into an IndexSegment.
LocalPartitionMetadata	getPartitionMetadata() When non-null, this is the description of the view of this index partition.
protected String	getProperty(IIndexManager indexManager, Properties properties, String namespace, String globalName, String defaultValue)
protected <E> E	getProperty(IIndexManager indexManager, Properties properties, String namespace, String globalName, String defaultValue, IValidator<E> validator)
boolean	getRawRecords() When true the index transparently promote large byte[] values associated with a key to raw records on the backing store.
ScatterSplitConfiguration	getScatterSplitConfiguration() The scatter split configuration for a scale-out index.
ISimpleSplitHandler	getSplitHandler() Object which decides whether and where to split an index partition into 2 or more index partitions.
ITupleSerializer	getTupleSerializer() The object used to form unsigned byte[] keys from Java objects, to (de-)serialize Java object stored in the index, and to (de-)compress the keys and values when stored in a leaf or ResultSet.
boolean	getVersionTimestampFilters() When true the index will maintain the min/max of the per tuple-revision timestamp on each Node of the B+Tree.
boolean	getVersionTimestamps() When true the index will maintain a per-index entry revision timestamp.
int	getWriteRetentionQueueCapacity()
int	getWriteRetentionQueueScan()
boolean	isIsolatable() True iff the index supports transactional isolation (both delete markers and version timestamps are required).
Checkpoint	overflowCheckpoint(Checkpoint oldCheckpoint) Variant used when an index overflows onto a new backing store.
static IndexMetadata	read(IRawStore store, long addr) Read the metadata record from the store.
void	readExternal(ObjectInput in)
void	setBloomFilterFactory(BloomFilterFactory bloomFilterFactory) Set the bloom filter factory.
void	setBranchingFactor(int branchingFactor) The branching factor MAY NOT be changed once an AbstractBTree object has been created.
void	setBTreeClassName(String className)
void	setBtreeRecordCompressorFactory(IRecordCompressorFactory btreeRecordCompressorFactory)
void	setCheckpointClassName(String className)
void	setConflictResolver(IConflictResolver conflictResolver)
void	setDeleteMarkers(boolean deleteMarkers)
void	setIndexSegmentBranchingFactor(int branchingFactor)
void	setIndexSegmentBufferNodes(boolean newValue)
void	setIndexSegmentRecordCompressorFactory(IRecordCompressorFactory segmentRecordCompressorFactory)
void	setInitialDataServiceUUID(UUID uuid)
void	setIsolatable(boolean isolatable) Convenience method sets both setDeleteMarkers(boolean) and setVersionTimestamps(boolean) at the same time.
void	setMaxRecLen(int maxRecLen) Set the maximum length of a byte[] value in a leaf of the index.
void	setNodeKeySerializer(IRabaCoder nodeKeysCoder)
void	setOverflowHandler(IOverflowHandler overflowHandler)
void	setPartitionMetadata(LocalPartitionMetadata pmd)
void	setRawRecords(boolean rawRecords) (Dis|En)able automatic promotion of index byte[] values larger than a configured byte length out of the index leaf and into raw records on the backing persistence store.
void	setScatterSplitConfiguration(ScatterSplitConfiguration newVal)
void	setSplitHandler(ISimpleSplitHandler splitHandler)
void	setTupleSerializer(ITupleSerializer tupleSer)
void	setVersionTimestampFilters(boolean versionTimestampFilters) Sets versionTimestampFilters.
void	setVersionTimestamps(boolean versionTimestamps)
void	setWriteRetentionQueueCapacity(int v)
void	setWriteRetentionQueueScan(int v)
String	toString() A human readable representation of the metadata record.
protected void	toString(StringBuilder sb) Extension hook for toString().
void	write(IRawStore store) Write out the metadata record for the btree on the store and return the address.
void	writeExternal(ObjectOutput out)

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

IndexMetadata

public IndexMetadata()

De-serialization constructor only - DO NOT use this ctor for creating a new instance! It will result in a thrown exception, typically from firstCheckpoint().

IndexMetadata

public IndexMetadata(UUID indexUUID)

Constructor used to configure a new unnamed B+Tree. The index UUID is set to the given value and all other fields are defaulted as explained at #IndexMetadata(Properties, String, UUID). Those defaults may be overridden using the various setter methods, but some values can not be safely overridden after the index is in use.

Parameters:

indexUUID - The indexUUID.

Throws:

IllegalArgumentException - if the indexUUID is null.

IndexMetadata

public IndexMetadata(String name,
                     UUID indexUUID)

Constructor used to configure a new named BTree. The index UUID is set to the given value and all other fields are defaulted as explained at #IndexMetadata(Properties, String, UUID). Those defaults may be overridden using the various setter methods, but some values can not be safely overridden after the index is in use.

Parameters:

name - The index name. When this is a scale-out index, the same name is specified for each index resource. However they will be registered on the journal under different names depending on the index partition to which they belong.

indexUUID - The indexUUID. The same index UUID MUST be used for all component indices in a scale-out index.

Throws:

IllegalArgumentException - if the indexUUID is null.

IndexMetadata

public IndexMetadata(IIndexManager indexManager,
                     Properties properties,
                     String namespace,
                     UUID indexUUID,
                     IndexTypeEnum indexType)

Constructor used to configure a new named B+Tree. The index UUID is set to the given value and all other fields are defaulted as explained at #getProperty(Properties, String, String, String). Those defaults may be overridden using the various setter methods.

Parameters:

indexManager - Optional. When given and when the IIndexManager is a scale-out IBigdataFederation, this object will be used to interpret the IndexMetadata.Options.INITIAL_DATA_SERVICE property.

properties - Properties object used to overridden the default values for this IndexMetadata instance.

namespace - The index name. When this is a scale-out index, the same name is specified for each index resource. However they will be registered on the journal under different names depending on the index partition to which they belong.

indexUUID - The indexUUID. The same index UUID MUST be used for all component indices in a scale-out index.

indexType - Type-safe enumeration specifying the type of the persistence class data structure (historically, this was always a B+Tree).

Throws:

IllegalArgumentException - if properties is null.

IllegalArgumentException - if indexUUID is null.

Method Detail

getMetadataAddr

public final long getMetadataAddr()

Address that can be used to read this metadata record from the store.

Note: This is not a persistent property. However the value is set when the metadata record is read from, or written on, the store. It is zero when you clone() a metadata record until it's been written onto the store.

getInitialDataServiceUUID

public UUID getInitialDataServiceUUID()

The UUID of the DataService on which the first partition of the scale-out index should be created. This is a purely transient property and will be null unless either explicitly set or set using . This property is only set by the ctor(s) that are used to create a new IndexMetadata instance, so no additional lookups are performed during de-serialization.

See Also:

IndexMetadata.Options.INITIAL_DATA_SERVICE, AbstractFederation.registerIndex(IndexMetadata, UUID)

setInitialDataServiceUUID

public void setInitialDataServiceUUID(UUID uuid)

getIndexUUID

public final UUID getIndexUUID()

The unique identifier for the (scale-out) index whose data is stored in this B+Tree data structure.

Note: When using a scale-out index the same indexUUID MUST be assigned to each mutable and immutable B+Tree having data for any partition of that scale-out index. This makes it possible to work backwards from the B+Tree data structures and identify the index to which they belong.

getIndexType

public final IndexTypeEnum getIndexType()

The type of the associated persistence capable data structure.

getName

public final String getName()

The name associated with the index -or- null iff the index is not named (internal indices are generally not named while application indices are always named).

Note: When the index is a scale-out index, this is the name of the scale-out index NOT the name under which an index partition is registered.

Note: When the index is a metadata index, then this is the name of the metadata index itself NOT the name of the managed scale-out index.

getBranchingFactor

public final int getBranchingFactor()

The branching factor for a mutable BTree. The branching factor is the #of children in a node or values in a leaf and must be an integer greater than or equal to three (3). Larger branching factors result in trees with fewer levels. However there is a point of diminishing returns at which the amount of copying performed to move the data around in the nodes and leaves exceeds the performance gain from having fewer levels. The branching factor for the read-only IndexSegments is generally much larger in order to reduce the number of disk seeks.

getIndexSegmentBranchingFactor

public final int getIndexSegmentBranchingFactor()

The branching factor used when building an IndexSegment (default is 4096). Index segments are read-only B+Tree resources. The are built using a bulk index build procedure and typically have a much higher branching factor than the corresponding mutable index on the journal. There are two reasons why it makes sense to use a larger branching factor for an index segment. First, the WORM Journal is used to buffer writes in scale-out and IO on an index on the WORM Journal is driven by node and leaf revisions so the index often uses a smaller branching factor on the WORM. Second, the index segment is laid out in total key order in the file and each node and leaf is a contiguous sequences of bytes on the disk (like the WORM, but unlike the RWStore). Since most of the latency of the disk is the seek, reading larger leaves from an index segment is efficient.

Note: the value of this property will determine the branching factor of the IndexSegment. When the IndexSegment is built, it will be given a clone() of this IndexMetadata and the actual branching factor for the IndexSegment be set on the getBranchingFactor() at that time.

Note: a branching factor of 256 for an index segment and split limits of (1M,5M) imply an average B+Tree height of 1.5 to 1.8. With a 10ms seek time and NO CACHE that is between 15 and 18ms average seek time.

Note: a branching factor of 512 for an index segment and split limits of (1M,5M) imply an average B+Tree height of 1.2 to 1.5. With a 10ms seek time and NO CACHE that is between 12 and 15ms average seek time.

Note: the actual size of the index segment of course depends heavily on (a) whether or now block references are being stored since the referenced blocks are also stored in the index segment; (b) the size of the keys and values stored in the index; and (c) the key, value, and record compression options in use.

getIndexSegmentBufferNodes

public final boolean getIndexSegmentBufferNodes()

Return true iff the nodes region for the IndexSegment should be fully buffered by the IndexSegmentStore.

See Also:

Options#INDEX_DEFAULT_SEGMENT_BUFFER_NODES

setIndexSegmentBufferNodes

public final void setIndexSegmentBufferNodes(boolean newValue)

getBtreeRecordCompressorFactory

public IRecordCompressorFactory getBtreeRecordCompressorFactory()

Return the record-level compression provider for a BTree (may be null, which implies no compression).

setBtreeRecordCompressorFactory

public void setBtreeRecordCompressorFactory(IRecordCompressorFactory btreeRecordCompressorFactory)

getIndexSegmentRecordCompressorFactory

public IRecordCompressorFactory getIndexSegmentRecordCompressorFactory()

Return the record-level compression provider for an IndexSegment (may be null, which implies no compression).

setIndexSegmentRecordCompressorFactory

public void setIndexSegmentRecordCompressorFactory(IRecordCompressorFactory segmentRecordCompressorFactory)

getWriteRetentionQueueCapacity

public final int getWriteRetentionQueueCapacity()

See Also:

IndexMetadata.Options.WRITE_RETENTION_QUEUE_CAPACITY

setWriteRetentionQueueCapacity

public final void setWriteRetentionQueueCapacity(int v)

getWriteRetentionQueueScan

public final int getWriteRetentionQueueScan()

See Also:

IndexMetadata.Options.WRITE_RETENTION_QUEUE_SCAN

setWriteRetentionQueueScan

public final void setWriteRetentionQueueScan(int v)

getPartitionMetadata

public final LocalPartitionMetadata getPartitionMetadata()

When non-null, this is the description of the view of this index partition. This will be null iff the BTree is not part of a scale-out index. This is updated when the view composition for the index partition is changed.

getBTreeClassName

public final String getBTreeClassName()

The name of a class derived from BTree that will be used to re-load the index. Note that index partitions are in general views (of one or more resources). Therefore only unpartitioned indices can be meaningfully specialized solely in terms of the BTree base class.

See Also:

IndexMetadata.Options.BTREE_CLASS_NAME

getCheckpointClassName

public final String getCheckpointClassName()

The name of the Checkpoint class used by the index. This may be overridden to store additional state with each Checkpoint record.

setCheckpointClassName

public final void setCheckpointClassName(String className)

getNodeKeySerializer

public final IRabaCoder getNodeKeySerializer()

Object used to code (compress) the keys in a node.

Note: The keys for nodes are separator keys for the leaves. Since they are chosen to be the minimum length separator keys dynamically when a leaf is split or joined the keys in the node typically DO NOT conform to application expectations and MAY be assigned a different IRabaCoder for that reason.

See Also:

getTupleSerializer()

getTupleSerializer

public final ITupleSerializer getTupleSerializer()

The object used to form unsigned byte[] keys from Java objects, to (de-)serialize Java object stored in the index, and to (de-)compress the keys and values when stored in a leaf or ResultSet.

Note: If you change this value in a manner that is not backward compatible once entries have been written on the index then you may be unable to any read data already written.

getConflictResolver

public final IConflictResolver getConflictResolver()

The optional object for handling write-write conflicts.

The concurrency control strategy detects write-write conflict resolution during backward validation. If a write-write conflict is detected and a conflict resolver is defined, then the conflict resolver is expected to make a best attempt using data type specific rules to reconcile the state for two versions of the same persistent identifier. If the conflict can not be resolved, then validation will fail. State-based conflict resolution when combined with validation (aka optimistic locking) is capable of validating the greatest number of interleavings of transactions (aka serialization orders).

Returns:

The conflict resolver to be applied during validation or null iff no conflict resolution will be performed.

getDeleteMarkers

public final boolean getDeleteMarkers()

When true the index will write a delete marker when an attempt is made to delete the entry under a key. Delete markers will be retained until a compacting merge of an index partition. When false the index entry will be removed from the index immediately.

Delete markers MUST be enabled to use scale-out indices. Index partition views depend on an ordered array of AbstractBTrees. The presence of a delete marker provides an indication of a deleted index entry and is used to prevent reading of index entries for the same key which might exist in an earlier AbstractBTree which is part of the same index partition view.

Delete markers MUST be enabled for transaction support where they play a similar role recording within the write set of the transaction the fact that an index entry has been deleted.

setDeleteMarkers

public final void setDeleteMarkers(boolean deleteMarkers)

getVersionTimestamps

public final boolean getVersionTimestamps()

When true the index will maintain a per-index entry revision timestamp. The primary use of this is in support of transactional isolation. Delete markers MUST be enabled when using revision timestamps.

See Also:

getVersionTimestampFilters()

getVersionTimestampFilters

public final boolean getVersionTimestampFilters()

When true the index will maintain the min/max of the per tuple-revision timestamp on each Node of the B+Tree. This information can be used to perform efficient filtering of iterators such that they visit only nodes and leaves having data for a specified tuple revision timestamp range. This filtering is efficient because it skips any node (and all spanned nodes or leaves) which does not have data for the desired revision timestamp range. In order to find all updates after a given timestamp revision, you specify (fromRevision,Long.MAX_VALUE). In order to visit the delta between two revisions, you specify (fromRevision, toRevision+1).

Tuple revision filtering can be very efficient for some purposes. For example, it can be used to synchronize disconnected clients or compute the write set of a committed transaction. However, it requires more space in the INodeData records since we must store the minimum and maximum timestamp revision for each child of a given node.

Per-tuple timestamp revisions MAY be used without support for per-tuple revision filtering.

See Also:

getVersionTimestamps()

setVersionTimestampFilters

public final void setVersionTimestampFilters(boolean versionTimestampFilters)

Sets versionTimestampFilters. You MUST also use setVersionTimestamps(boolean) to true for version timestamp filtering to be supported.

Parameters:

versionTimestampFilters - true iff version timestamp filtering should be supported.

setVersionTimestamps

public final void setVersionTimestamps(boolean versionTimestamps)

isIsolatable

public final boolean isIsolatable()

True iff the index supports transactional isolation (both delete markers and version timestamps are required).

setIsolatable

public void setIsolatable(boolean isolatable)

Convenience method sets both setDeleteMarkers(boolean) and setVersionTimestamps(boolean) at the same time.

Parameters:

isolatable - true if delete markers and version timestamps will be enabled -or- false if they will be disabled.

getRawRecords

public final boolean getRawRecords()

When true the index transparently promote large byte[] values associated with a key to raw records on the backing store. This feature is disabled by default. Indices which do use large records should enable this option in order to reduce their IO churn and disk footprint.

See Also:

getMaxRecLen()

setRawRecords

public final void setRawRecords(boolean rawRecords)

(Dis|En)able automatic promotion of index byte[] values larger than a configured byte length out of the index leaf and into raw records on the backing persistence store. This option can significicantly reduce the IO churn for indices which do make use of large values. However, the leaves will occupy slightly more space (~ 1 bit per tuple) if this option is enabled and none of the values stored in the index exceed the configured maximum value length. IRabaCoders which rely on a uniform value length generally already use small values and should typically turn this feature off in order to make the leaf as compact as possible.

Parameters:

rawRecords - true if the feature is to be enabled.

See Also:

setMaxRecLen(int)

getMaxRecLen

public final int getMaxRecLen()

When getRawRecords() returns true, this method returns the maximum byte length of a byte[] value which may be stored in a B+Tree leaf (default IndexMetadata.Options.MAX_REC_LEN. Values larger than this will be automatically converted into raw record references.

See Also:

IndexMetadata.Options.MAX_REC_LEN, IndexMetadata.Options.DEFAULT_MAX_REC_LEN

setMaxRecLen

public final void setMaxRecLen(int maxRecLen)

Set the maximum length of a byte[] value in a leaf of the index.

Parameters:

maxRecLen - The maximum length of a byte[] value in a leaf of the index. A value of ZERO (0) may be used to force all values into raw records.

Throws:

IllegalArgumentException - if the argument is negative or greater than Short.MAX_VALUE

See Also:

setRawRecords(boolean)

setPartitionMetadata

public void setPartitionMetadata(LocalPartitionMetadata pmd)

setNodeKeySerializer

public void setNodeKeySerializer(IRabaCoder nodeKeysCoder)

setTupleSerializer

public void setTupleSerializer(ITupleSerializer tupleSer)

setBranchingFactor

public void setBranchingFactor(int branchingFactor)

The branching factor MAY NOT be changed once an AbstractBTree object has been created.

Parameters:

branchingFactor -

setIndexSegmentBranchingFactor

public void setIndexSegmentBranchingFactor(int branchingFactor)

setBTreeClassName

public void setBTreeClassName(String className)

setConflictResolver

public void setConflictResolver(IConflictResolver conflictResolver)

getBloomFilterFactory

public BloomFilterFactory getBloomFilterFactory()

Return the bloom filter factory.

Bloom filters provide fast rejection for point tests in a space efficient manner with a configurable probability of a false positive. Since the bloom filter does not give positive results with 100% certainity, the index is tested iff the bloom filter states that the key exists.

Note: Bloom filters are NOT enabled by default since point tests are not a bottleneck (or even used) for some indices. Also, when multiple indices represent different access paths for the same information, you only need a bloom filter on one of those indices.

Returns:

Return the object that will be used to configure an optional bloom filter for a BTree or IndexSegment. When null the index WILL NOT use a bloom filter.

See Also:

BloomFilterFactory, BloomFilterFactory.DEFAULT

setBloomFilterFactory

public void setBloomFilterFactory(BloomFilterFactory bloomFilterFactory)

Set the bloom filter factory.

Bloom filters provide fast rejection for point tests in a space efficient manner with a configurable probability of a false positive. Since the bloom filter does not give positive results with 100% certainity, the index is tested iff the bloom filter states that the key exists.

Parameters:

bloomFilterFactory - The new value (may be null).

See Also:

BloomFilterFactory.DEFAULT

getOverflowHandler

public IOverflowHandler getOverflowHandler()

An optional object that may be used to inspect, and possibly operate on, each index entry as it is copied into an IndexSegment.

setOverflowHandler

public void setOverflowHandler(IOverflowHandler overflowHandler)

getSplitHandler

public ISimpleSplitHandler getSplitHandler()

Object which decides whether and where to split an index partition into 2 or more index partitions. The default is a null reference. The default behavior when no split handler is specified will work for nearly all use cases and will result in index partitions whose size on the disk is bounded by the parameter specified using OverflowManager.Options#NOMINAL_SHARD_SIZE. Indices which require certain guarantees for atomicity, such as the SparseRowStore, must override this default.

Returns:

The ISimpleSplitHandler -or- null if the application has not imposed any additional constraints on the separator keys when splitting index partitions.

setSplitHandler

public void setSplitHandler(ISimpleSplitHandler splitHandler)

getAsynchronousIndexWriteConfiguration

public AsynchronousIndexWriteConfiguration getAsynchronousIndexWriteConfiguration()

The asynchronous index write API configuration for this index.

getAsynchronousIndexWriteConfiguration

public void getAsynchronousIndexWriteConfiguration(AsynchronousIndexWriteConfiguration newVal)

Set the asynchronous index write API configuration for this index.

getScatterSplitConfiguration

public ScatterSplitConfiguration getScatterSplitConfiguration()

The scatter split configuration for a scale-out index.

setScatterSplitConfiguration

public void setScatterSplitConfiguration(ScatterSplitConfiguration newVal)

write

public void write(IRawStore store)

Write out the metadata record for the btree on the store and return the address.

Parameters:

store - The store on which the metadata record is being written.

Throws:

IllegalStateException - if the record has already been written on the store.

IllegalStateException - if the indexUUID field is null - this generally indicates that you used the de-serialization constructor rather than one of the constructor variants that accept the required UUID parameter.

read

public static IndexMetadata read(IRawStore store,
                                 long addr)

Read the metadata record from the store.

Parameters:

store - the store.

addr - the address of the metadata record.

Returns:

the metadata record. The address from which it was loaded is set on the metadata record as a side-effect.

toString

public String toString()

A human readable representation of the metadata record.

Overrides:

toString in class Object

toString

protected void toString(StringBuilder sb)

Extension hook for toString().

Parameters:

sb - Where to write additional metadata.

readExternal

public void readExternal(ObjectInput in)
                  throws IOException,
                         ClassNotFoundException

Specified by:

readExternal in interface Externalizable

Throws:

IOException

ClassNotFoundException

TODO:

review generated record for compactness.

writeExternal

public void writeExternal(ObjectOutput out)
                   throws IOException

Specified by:

writeExternal in interface Externalizable

Throws:

IOException

clone

public IndexMetadata clone()

Makes a copy of the persistent data, clearing the address of the IndexMetadata record on the cloned copy.

Overrides:

clone in class Object

Returns:

The cloned copy.

firstCheckpoint

public final Checkpoint firstCheckpoint()

Create an initial Checkpoint for a new persistence capable data structure described by this metadata record.

The caller is responsible for writing the Checkpoint record onto the store.

The class identified by getCheckpointClassName() MUST declare a public constructor with the following method signature

  ...( IndexMetadata metadata )
 

Returns:

The Checkpoint.

overflowCheckpoint

public final Checkpoint overflowCheckpoint(Checkpoint oldCheckpoint)

Variant used when an index overflows onto a new backing store.

The caller is responsible for writing the Checkpoint record onto the store.

The class identified by getCheckpointClassName() MUST declare a public constructor with the following method signature

  ...( IndexMetadata metadata, Checkpoint oldCheckpoint )
 

Parameters:

oldCheckpoint - The last checkpoint for the index of the old backing store.

Returns:

The first Checkpoint for the index on the new backing store.

Throws:

IllegalArgumentException - if the oldCheckpoint is null.

getKeyBuilder

public IKeyBuilder getKeyBuilder()

Factory for thread-safe IKeyBuilder objects for use by ITupleSerializer.serializeKey(Object) and possibly others.

Note: A mutable B+Tree is always single-threaded. However, read-only B+Trees allow concurrent readers. Therefore, thread-safety requirement is safe for either a single writers -or- for concurrent readers.

Note: If you change this value in a manner that is not backward compatable once entries have been written on the index then you may be unable to any read data already written.

Note: This method delegates to ITupleSerializer.getKeyBuilder(). This IKeyBuilder SHOULD be used to form all keys for this index. This is critical for indices that have Unicode data in their application keys as the formation of Unicode sort keys from Unicode data depends on the IKeyBuilderFactory. If you use a locally configured IKeyBuilder then your Unicode keys will be encoded based on the Locale configured for the JVM NOT the factory specified for this index.

Specified by:

getKeyBuilder in interface IKeyBuilderFactory

getProperty

protected String getProperty(IIndexManager indexManager,
                             Properties properties,
                             String namespace,
                             String globalName,
                             String defaultValue)

See Also:

Configuration.getProperty(IIndexManager, Properties, String, String, String)

getProperty

protected <E> E getProperty(IIndexManager indexManager,
                            Properties properties,
                            String namespace,
                            String globalName,
                            String defaultValue,
                            IValidator<E> validator)

See Also:

Configuration.getProperty(IIndexManager, Properties, String, String, String, IValidator)