com.bigdata.journal
Interface Options

All Known Subinterfaces:

AbstractTripleStore.Options, BigdataFileSystem.Options, BigdataSail.Options, DataServer.Options, DataService.Options, EmbeddedClient.Options, IndexManager.Options, Journal.Options, LocalDataServiceClient.Options, LocalTripleStore.Options, MetadataServer.Options, MetadataService.Options, OverflowManager.Options, ResourceManager.Options, StoreManager.Options, TempTripleStore.Options

public interface Options

Options for the Journal. Options are specified as property values to the Journal.Journal(Properties) constructor.

Write Cache

Some operating systems, file systems, disk controllers, and disk drives may re-order writes, processing them in a different order from the sequence in which they are issued by the application, and may return from FileChannel.force(boolean) or FileDescriptor.sync() before the data are on stable media. This can have the consequence that the root blocks are laid down on the disk before the application data. In this situation a hard failure during (or after!) the write could result in the loss of application data since the root blocks representing the atomic commit point MIGHT be updated before the application data was successfully made stable on disk. Further, IAtomicStore.commit() MAY return before the data are stable on disk.

Depending on your needs there are several steps which you can take to increase data security, including backups, media replication, etc. In terms of the Journal itself, you can use FORCE_WRITES to request synchronous writes or disable the write cache on your platform. While FORCE_WRITES is trivially specified as a configuration option, please note that the behavior of FORCE_WRITES can vary from platform to platform and that disabling the write cache on your platform may give you a greater assurance that writes are neither being reordered nor returning before the data are on stable storage. Also note that some high end disk systems have battery backup for the on-disk write cache, in which case you may choose to leave the write cache on the disk enabled.

Version:

$Id: Options.java 2265 2009-10-26 12:51:06Z thompsonbry $

Author:

Bryan Thompson

Field Summary

static String	ALTERNATE_ROOT_BLOCK WARNING - The use of this option is dangerous. This option MAY be used if you have a commit point whose root block is valid but where the data associated with that root block is invalid.
static String	BUFFER_MODE The name of the property whose value controls the nature of the backing store (default DEFAULT_BUFFER_MODE).
static String	CREATE An optional boolean property (default is "true").
static String	CREATE_TEMP_FILE This boolean option causes a new file to be created using the File.createTempFile(String, String, File) temporary file mechanism.
static String	CREATE_TIME An optional property allowed iff a new journal is being created whose value is the timestamp to be written in the IRootBlockView.getCreateTime() field of the root block.
static String	DEFAULT_BUFFER_MODE The default for the BUFFER_MODE.
static String	DEFAULT_CREATE The default for the CREATE option.
static String	DEFAULT_CREATE_TEMP_FILE The default for the CREATE_TEMP_FILE option.
static String	DEFAULT_DELETE_ON_CLOSE The default for the DELETE_ON_CLOSE option.
static String	DEFAULT_DELETE_ON_EXIT The default for the DELETE_ON_EXIT option.
static String	DEFAULT_DOUBLE_SYNC The default for the DOUBLE_SYNC option (application data is NOT forced to disk before we write the root blocks).
static String	DEFAULT_FILE_LOCK_ENABLED
static String	DEFAULT_FORCE_ON_COMMIT The default for the FORCE_ON_COMMIT option (file data and metadata are forced).
static String	DEFAULT_FORCE_WRITES The default for the FORCE_WRITES option (writes are not forced).
static String	DEFAULT_HISTORICAL_INDEX_CACHE_CAPACITY The default for the HISTORICAL_INDEX_CACHE_CAPACITY option.
static String	DEFAULT_HISTORICAL_INDEX_CACHE_TIMEOUT
static String	DEFAULT_INITIAL_EXTENT The default initial extent for a new journal.
static String	DEFAULT_LIVE_INDEX_CACHE_CAPACITY The default for the LIVE_INDEX_CACHE_CAPACITY option.
static String	DEFAULT_LIVE_INDEX_CACHE_TIMEOUT
static String	DEFAULT_MAXIMUM_EXTENT The default maximum extent for a new journal before a commit triggers an overflow event (200M).
static String	DEFAULT_READ_CACHE_CAPACITY The default for READ_CACHE_CAPACITY.
static String	DEFAULT_READ_CACHE_MAX_RECORD_SIZE The default for READ_CACHE_MAX_RECORD_SIZE.
static String	DEFAULT_READ_ONLY The default for the READ_ONLY option.
static String	DEFAULT_USE_DIRECT_BUFFERS The default for USE_DIRECT_BUFFERS.
static String	DEFAULT_VALIDATE_CHECKSUM
static String	DEFAULT_WRITE_CACHE_CAPACITY The default for WRITE_CACHE_CAPACITY (1M).
static String	DELETE_ON_CLOSE This boolean option causes the journal file to be deleted when the journal is closed (default false).
static String	DELETE_ON_EXIT This boolean option causes the journal file to be deleted when the VM exits (default false).
static String	DOUBLE_SYNC Deprecated. This option does NOT provide a sufficient guarantee when a write cache is in use by the operating system or the disk if the layered write caches return before all data is safely on disk (or in a battery powered cache). In order to protect against this you MUST disable the write cache layers in the operating system and the disk drive such that FileChannel.force(boolean) will not return until the data are in fact on stable storage. If you disable the OS and disk write cache then you do NOT need to specify this option since writes will be ordered and all data will be on disk before we update the commit blocks.
static String	FILE The name of the file.
static String	FILE_LOCK_ENABLED Deprecated. FileLockUtility is going away since advisory locks are not automatically removed if an application dies.
static String	FORCE_ON_COMMIT A trinary property {no, force, forceMetadata} that determines whether the journal will be forced to stable storage on a commit (default ).
static String	FORCE_WRITES A trinary property {no, force, forceMetadata} that determines whether the journal will be opened in a mode that writes through synchronously to stable storage (default ).
static String	HISTORICAL_INDEX_CACHE_CAPACITY The capacity of the LRU cache for canonicalizing mapping maintained by the AbstractJournal for index Checkpoints.
static String	HISTORICAL_INDEX_CACHE_TIMEOUT The timeout in milliseconds for stale entries in the historical index cache -or- ZERO (0) to disable the timeout (default "60000").
static String	INITIAL_EXTENT The initial extent of the journal (bytes).
static String	JNL The recommended extension for journal files.
static String	LIVE_INDEX_CACHE_CAPACITY The capacity of the LRU cache for the "live" Name2Addr object (default "60").
static String	LIVE_INDEX_CACHE_TIMEOUT The timeout in milliseconds for stale entries in the live index cache -or- ZERO (0) to disable the timeout (default "60000").
static String	MAXIMUM_EXTENT The maximum extent of the journal (bytes).
static long	minimumInitialExtent
static int	minimumWriteCacheCapacity
static String	OFFSET_BITS The #of bits in a 64-bit long integer identifier that are used to encode the byte offset of a record in the store as an unsigned integer.
static String	READ_CACHE_CAPACITY An integer property whose value controls the size of the write cache (in records) (default "0" bytes).
static String	READ_CACHE_MAX_RECORD_SIZE An integer property whose value is the maximum size of a record that will be allowed into the optional read cache.
static String	READ_ONLY When true, the journal must pre-exist and will be read-only (optional, default is "false").
static String	SEG The recommended extension for IndexSegment files.
static String	TMP_DIR The property whose value is the name of the directory in which temporary files will be created.
static String	USE_DIRECT_BUFFERS A boolean property whose value controls whether a direct (native) or heap-based ByteBuffer will be allocated by the selected BufferMode.
static String	VALIDATE_CHECKSUM An optional boolean property (default "true").
static String	WRITE_CACHE_CAPACITY An integer property whose value controls the size of the write cache (in bytes) used by the selected BufferMode (default "1048576" bytes).

Field Detail

FILE

static final String FILE

The name of the file. If the file not found and CREATE is true, then a new journal will be created. The filename extension ".jnl" is recommended.

BUFFER_MODE

static final String BUFFER_MODE

The name of the property whose value controls the nature of the backing store (default DEFAULT_BUFFER_MODE).

See Also:

BufferMode, DEFAULT_BUFFER_MODE

HISTORICAL_INDEX_CACHE_CAPACITY

static final String HISTORICAL_INDEX_CACHE_CAPACITY

The capacity of the LRU cache for canonicalizing mapping maintained by the AbstractJournal for index Checkpoints. The capacity of this cache indirectly controls how many historical indices the journal will hold open. The main reason for keeping a historical index open is to reuse its buffers, including its node and leaf cache, if another request arrives "soon" for that historical commit point of the index.

The effect of this parameter is indirect owning to the semantics of weak references and the control of the JVM over when they are cleared. Once an index becomes weakly reachable, the JVM will eventually GC the index object, thereby effectively closing it (or at least releasing all resources associated with that index). Since indices which are strongly reachable are never "closed" this provides our guarentee that indices are never closed if they are in use.

See Also:

DEFAULT_HISTORICAL_INDEX_CACHE_CAPACITY, #LIVE_INDEX_CACHE_CAPACITY, which governs the retention of clean "live" indices., AbstractJournal.getIndex(long)

HISTORICAL_INDEX_CACHE_TIMEOUT

static final String HISTORICAL_INDEX_CACHE_TIMEOUT

The timeout in milliseconds for stale entries in the historical index cache -or- ZERO (0) to disable the timeout (default "60000"). When this timeout expires, the reference for the entry in the backing HardReferenceQueue will be cleared. Note that the entry will remain in the historical index cache regardless as long as it is strongly reachable.

See Also:

AbstractJournal.getIndex(long)

DEFAULT_HISTORICAL_INDEX_CACHE_TIMEOUT

static final String DEFAULT_HISTORICAL_INDEX_CACHE_TIMEOUT

See Also:

Constant Field Values

LIVE_INDEX_CACHE_CAPACITY

static final String LIVE_INDEX_CACHE_CAPACITY

The capacity of the LRU cache for the "live" Name2Addr object (default "60"). A special Name2Addr instance is maintained for the "live" index objects, gives the most recent Checkpoint address from which the "live" version of the index may be loaded, and provides a canonicalizing cache for the "live" index objects by name.

The capacity of this cache for this special Name2Addr instance indirectly controls how many clean "live" indices the journal will hold open. A "live" index is only clean when it has just been loaded from the store or when it has just been committed, so this really effects how long an index which has been committed will be keep open. The main reason for keeping the index open after a commit is to reuse its buffers, including its node and leaf cache, if another request arrives "soon" for an operation on that index.

The effect of this parameter is indirect owning to the semantics of weak references and the control of the JVM over when they are cleared. Once an index becomes weakly reachable, the JVM will eventually GC the index object, thereby effectively closing it (or at least releasing all resources associated with that index). Dirty indices will be held open until the next commit/abort regardless since a hard reference is placed on the commit list.

See Also:

DEFAULT_LIVE_INDEX_CACHE_CAPACITY, HISTORICAL_INDEX_CACHE_CAPACITY, Name2Addr

LIVE_INDEX_CACHE_TIMEOUT

static final String LIVE_INDEX_CACHE_TIMEOUT

The timeout in milliseconds for stale entries in the live index cache -or- ZERO (0) to disable the timeout (default "60000"). When this timeout expires, the reference for the entry in the backing HardReferenceQueue will be cleared. Note that the entry will remain in the live index cache regardless as long as it is strongly reachable.

See Also:

Name2Addr

DEFAULT_LIVE_INDEX_CACHE_TIMEOUT

static final String DEFAULT_LIVE_INDEX_CACHE_TIMEOUT

See Also:

Constant Field Values

USE_DIRECT_BUFFERS

static final String USE_DIRECT_BUFFERS

A boolean property whose value controls whether a direct (native) or heap-based ByteBuffer will be allocated by the selected BufferMode. Note that this only applies to fully buffered modes, e.g., BufferMode.Transient or BufferMode.Direct. This parameter has no effect for the memory-mapped or disk-only buffer modes.

READ_CACHE_CAPACITY

static final String READ_CACHE_CAPACITY

An integer property whose value controls the size of the write cache (in records) (default "0" bytes). A value of ZERO (0) will disable the read cache.

Note that only the BufferMode.Disk mode supports a read cache - most of the other modes are already fully buffered.

See Also:

DEFAULT_READ_CACHE_CAPACITY

READ_CACHE_MAX_RECORD_SIZE

static final String READ_CACHE_MAX_RECORD_SIZE

An integer property whose value is the maximum size of a record that will be allowed into the optional read cache. Records larger than this size are always read from the backing store.

See Also:

DEFAULT_READ_CACHE_MAX_RECORD_SIZE

WRITE_CACHE_CAPACITY

static final String WRITE_CACHE_CAPACITY

An integer property whose value controls the size of the write cache (in bytes) used by the selected BufferMode (default "1048576" bytes). A value of ZERO (0) will disable the write cache.

Note: This value is ignored by some BufferModes.

Note: The write cache is flushed incrementally each time it becomes full and on each commit. Therefore there is no point having a write cache that is significantly larger then the #of bytes which are written on average during a commit.

See Also:

DEFAULT_WRITE_CACHE_CAPACITY

INITIAL_EXTENT

static final String INITIAL_EXTENT

The initial extent of the journal (bytes). When the journal is backed by a file, this is the initial length of that file. The initial user extent is typically slightly smaller as the head of the file contains some metadata outside of the user space (the root blocks). The initial extent will be transparently extended as necessary when the user space becomes full.

Note: When using a partitioned index strategy the initial extent and the maximum extent should be the same so that the cost of extending the journal may be avoided as much as possible.

See Also:

DEFAULT_INITIAL_EXTENT

minimumInitialExtent

static final long minimumInitialExtent

See Also:

Constant Field Values

minimumWriteCacheCapacity

static final int minimumWriteCacheCapacity

See Also:

Constant Field Values

MAXIMUM_EXTENT

static final String MAXIMUM_EXTENT

The maximum extent of the journal (bytes). The journal will IResourceManager.overflow() once it approaches this limit.

See Also:

DEFAULT_MAXIMUM_EXTENT

OFFSET_BITS

static final String OFFSET_BITS

The #of bits in a 64-bit long integer identifier that are used to encode the byte offset of a record in the store as an unsigned integer. The default is WormAddressManager.SCALE_UP_OFFSET_BITS for a Journal and WormAddressManager.SCALE_OUT_OFFSET_BITS for a StoreManager.ManagedJournal.

See Also:

WormAddressManager.SCALE_UP_OFFSET_BITS, WormAddressManager.SCALE_OUT_OFFSET_BITS

VALIDATE_CHECKSUM

static final String VALIDATE_CHECKSUM

An optional boolean property (default "true"). When true, the checksum stored in the root blocks of an existing store will be validated when the store file is opened. This property may be set to false if you are engaged in a desperate attempt to use a bad root block or to patch a bad root block using a hex editor.

See Also:

DEFAULT_VALIDATE_CHECKSUM

ALTERNATE_ROOT_BLOCK

static final String ALTERNATE_ROOT_BLOCK

WARNING - The use of this option is dangerous. This option MAY be used if you have a commit point whose root block is valid but where the data associated with that root block is invalid. Normally the more current root block is used to (re-)open the store. However, if this option is specified, the previous root block will be used to open the store. This will allow you to access the previous commit point. If you subsequently perform a commit then the most root block will be overwritten and any data associated with the last commit point will be unreachable. This option may be considered in the cases where the application is otherwise unable to proceed. It is strongly recommended that you also specify READ_ONLY so that you do not accidentally trigger a commit and thereby make the data associated with the other root block unreachable. You may of course deliberately allow a commit as an attempt to restore the database to service accepting that you have rolled back the database by one commit point in doing so.

CREATE

static final String CREATE

An optional boolean property (default is "true"). When true and the named file is not found, a new journal will be created.

CREATE_TIME

static final String CREATE_TIME

An optional property allowed iff a new journal is being created whose value is the timestamp to be written in the IRootBlockView.getCreateTime() field of the root block. This is used by the ResourceManager to place consistent createTime timestamps on its managed journal resources.

READ_ONLY

static final String READ_ONLY

When true, the journal must pre-exist and will be read-only (optional, default is "false").

FORCE_WRITES

static final String FORCE_WRITES

A trinary property {no, force, forceMetadata} that determines whether the journal will be opened in a mode that writes through synchronously to stable storage (default ). This option does NOT effect the stability of the data on disk but may be used to tweak the file system buffering (forcing writes is generally MUCH slower and is turned off by default).

See Also:

#FORCE_ON_COMMIT, which controls the stability of the data on disk., DEFAULT_FORCE_WRITES, ForceEnum

FORCE_ON_COMMIT

static final String FORCE_ON_COMMIT

A trinary property {no, force, forceMetadata} that determines whether the journal will be forced to stable storage on a commit (default ).

No

This option is useful when the journal is replicated so that we can always failover to another server having the same data. Unless the file is replicated or transient, this mode can lead to lost data if there is a hardware or software failure.

Force

Force the journal contents, but not the file metadata, to stable storage. The precise semantics of this option are dependent on the OS and hardware platform (some hardware platforms may guarantee that the file metadata is stable using a battery powered disk cache). If there is a crash, the information lost can range from the last modified timestamp on the file to the file length (which can make it impossible to re-open the journal until the file length is corrected with a file system repair utility), to the file allocation structure in the file system (which is always a serious loss, but is normally repairable with a file system repair utility).

ForceMetadata

Force the journal contents and the file metadata to stable storage. This option is the most secure.

Based on some reading online, it appears that not forcing metadata can mean anything from having the lastModified timestamp on the file be incorrect to having the wrong file length in the directory structure, to having the file allocation nodes not be coherent. This appears to be OS and hardware specific (some hardware buffers the power for the disk system so that such writes always complete).

See Also:

DOUBLE_SYNC, IRawStore.force(boolean), FileChannel.force(boolean), ForceEnum

DOUBLE_SYNC

static final String DOUBLE_SYNC

Deprecated. This option does NOT provide a sufficient guarantee when a write cache is in use by the operating system or the disk if the layered write caches return before all data is safely on disk (or in a battery powered cache). In order to protect against this you MUST disable the write cache layers in the operating system and the disk drive such that FileChannel.force(boolean) will not return until the data are in fact on stable storage. If you disable the OS and disk write cache then you do NOT need to specify this option since writes will be ordered and all data will be on disk before we update the commit blocks.

This boolean option causes application data to be forced to stable storage before we update the root blocks. This option seeks to guarantee that the application data is stable on the disk before the atomic commit. Some operating systems and/or file systems may otherwise choose an ordered write or otherwise process the writes in a different order. This could have the consequence that the root blocks are laid down on the disk before the application data. In this situation a hard failure during the write could result in the loss of application data since the updated root blocks represent the atomic commit point but not all application data was successfully made stable on disk.

CREATE_TEMP_FILE

static final String CREATE_TEMP_FILE

This boolean option causes a new file to be created using the File.createTempFile(String, String, File) temporary file mechanism.

Note: If DELETE_ON_EXIT is also specified, then the temporary file will be marked for deletion when the JVM exits.

Note: This option is often used when preparing a journal for a unit test.

Note: The default temporary directory is used unless it is overridden by the TMP_DIR option.

Note: In order to re-open a journal that was created with this option you MUST (a) set this option to false and (b) set FILE to the value returned by AbstractJournal.getFile().

DELETE_ON_CLOSE

static final String DELETE_ON_CLOSE

This boolean option causes the journal file to be deleted when the journal is closed (default false). This option is used by the some test suites (those that do not test restart safety) to keep down the disk burden of the tests and MUST NOT be used with restart-safe data.

DELETE_ON_EXIT

static final String DELETE_ON_EXIT

This boolean option causes the journal file to be deleted when the VM exits (default false). This option is used by the test suites to keep down the disk burden of the tests and MUST NOT be used with restart-safe data.

TMP_DIR

static final String TMP_DIR

The property whose value is the name of the directory in which temporary files will be created. When not specified the default is governed by the value of the System property named java.io.tmpdir. There are several kinds of temporary files that can be created, including temporary journals, intermediate files from an index merge process, etc.

FILE_LOCK_ENABLED

static final String FILE_LOCK_ENABLED

Deprecated. FileLockUtility is going away since advisory locks are not automatically removed if an application dies.

When true (default "true") a FileLock will be sought for the journal by default. When false only an advisory lock will be sought. Note that FileLock is never used for memory-mapped (causes problems) or read-only files (does not work) and MAY fail on NFS mounts. If FileLock fails then we always back down to an advisory lock.

See Also:

FileLockUtility.openFile(File, String, boolean)

DEFAULT_FILE_LOCK_ENABLED

static final String DEFAULT_FILE_LOCK_ENABLED

See Also:

Constant Field Values

DEFAULT_BUFFER_MODE

static final String DEFAULT_BUFFER_MODE

The default for the BUFFER_MODE.

DEFAULT_USE_DIRECT_BUFFERS

static final String DEFAULT_USE_DIRECT_BUFFERS

The default for USE_DIRECT_BUFFERS.

See Also:

Constant Field Values

DEFAULT_WRITE_CACHE_CAPACITY

static final String DEFAULT_WRITE_CACHE_CAPACITY

The default for WRITE_CACHE_CAPACITY (1M).

See Also:

Constant Field Values

DEFAULT_READ_CACHE_CAPACITY

static final String DEFAULT_READ_CACHE_CAPACITY

The default for READ_CACHE_CAPACITY.

See Also:

Constant Field Values

DEFAULT_READ_CACHE_MAX_RECORD_SIZE

static final String DEFAULT_READ_CACHE_MAX_RECORD_SIZE

The default for READ_CACHE_MAX_RECORD_SIZE.

See Also:

Constant Field Values

DEFAULT_INITIAL_EXTENT

static final String DEFAULT_INITIAL_EXTENT

The default initial extent for a new journal.

See Also:

INITIAL_EXTENT, Constant Field Values

DEFAULT_MAXIMUM_EXTENT

static final String DEFAULT_MAXIMUM_EXTENT

The default maximum extent for a new journal before a commit triggers an overflow event (200M).

See Also:

MAXIMUM_EXTENT, Constant Field Values

DEFAULT_VALIDATE_CHECKSUM

static final String DEFAULT_VALIDATE_CHECKSUM

See Also:

Constant Field Values

DEFAULT_CREATE

static final String DEFAULT_CREATE

The default for the CREATE option.

See Also:

Constant Field Values

DEFAULT_READ_ONLY

static final String DEFAULT_READ_ONLY

The default for the READ_ONLY option.

See Also:

Constant Field Values

DEFAULT_FORCE_WRITES

static final String DEFAULT_FORCE_WRITES

The default for the FORCE_WRITES option (writes are not forced).

TODO:

consider changing this default so that we normally request synchronous writes.

DEFAULT_FORCE_ON_COMMIT

static final String DEFAULT_FORCE_ON_COMMIT

The default for the FORCE_ON_COMMIT option (file data and metadata are forced).

DEFAULT_DOUBLE_SYNC

static final String DEFAULT_DOUBLE_SYNC

The default for the DOUBLE_SYNC option (application data is NOT forced to disk before we write the root blocks).

See Also:

Constant Field Values

DEFAULT_DELETE_ON_CLOSE

static final String DEFAULT_DELETE_ON_CLOSE

The default for the DELETE_ON_CLOSE option.

See Also:

Constant Field Values

DEFAULT_DELETE_ON_EXIT

static final String DEFAULT_DELETE_ON_EXIT

The default for the DELETE_ON_EXIT option.

See Also:

Constant Field Values

DEFAULT_CREATE_TEMP_FILE

static final String DEFAULT_CREATE_TEMP_FILE

The default for the CREATE_TEMP_FILE option.

See Also:

Constant Field Values

DEFAULT_HISTORICAL_INDEX_CACHE_CAPACITY

static final String DEFAULT_HISTORICAL_INDEX_CACHE_CAPACITY

The default for the HISTORICAL_INDEX_CACHE_CAPACITY option.

See Also:

Constant Field Values

DEFAULT_LIVE_INDEX_CACHE_CAPACITY

static final String DEFAULT_LIVE_INDEX_CACHE_CAPACITY

The default for the LIVE_INDEX_CACHE_CAPACITY option.

See Also:

Constant Field Values

JNL

static final String JNL

The recommended extension for journal files.

See Also:

Constant Field Values

SEG

static final String SEG

The recommended extension for IndexSegment files.

See Also:

Constant Field Values