com.bigdata.journal
Class CompactTask

java.lang.Object
  com.bigdata.journal.CompactTask

All Implemented Interfaces:

Callable<Journal>

public class CompactTask
extends Object
implements Callable<Journal>

Task compacts the journal state onto a caller specified file. This may be used to compact a journal, to create backups, or to convert an in-memory journal into a disk-based journal. The task reads the state of each named index as of the selected commit point, writing the index entries in index order onto the output journal. This process will typically both reduce the the space on disk required by the (new) backing store and improve locality in the (new) backing store.

Note: The new Journal WILL NOT include any historical commit points other than the one selected by the caller specified commitTime.

Note: If any indices use references to raw records then they MUST define an IOverflowHandler in order for the raw records to be copied to the new store and those references updated in the index to point to the records in the new store. For example, the BigdataFileSystem uses such references and defines an IOverflowHandler so that the raw file blocks will not be lost on overflow.

Version:

$Id: CompactTask.java 2265 2009-10-26 12:51:06Z thompsonbry $

Author:

Bryan Thompson

See Also:

Journal.compact(File)

TODO:

it would be easy enough to change the branching factor during this task., add listener api for start/end notice (and perhaps error)., add export task that builds index segments from the indices on a journal and work this into CompactJournalUtility or a similar utility class.

Nested Class Summary

protected class

CompactTask.CopyIndexTask Copy an index to the new journal.

Field Summary

protected long	commitTime The caller specified commit time.
protected static boolean	DEBUG
protected AtomicInteger	doneCount
protected AtomicReference<Throwable>	firstCause
protected static boolean	INFO
protected static org.apache.log4j.Logger	log Logger.
protected Journal	oldJournal The source Journal.
protected File	outFile The output File.
protected AtomicInteger	startCount

Constructor Summary

CompactTask(Journal src, File outFile, long commitTime)
The task reads the state of each named index as of the given commitTime and writes the index data in order on the output journal.

Method Summary

Journal	call() Compact the oldJournal journal onto the outFile file.
protected void	copyIndices(Journal newJournal) Copy all named indices from the oldJournal to the new journal.
protected Journal	createJournal() Create the output journal.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected static final org.apache.log4j.Logger log

Logger.

INFO

protected static final boolean INFO

DEBUG

protected static final boolean DEBUG

oldJournal

protected final Journal oldJournal

The source Journal.

outFile

protected final File outFile

The output File.

commitTime

protected final long commitTime

The caller specified commit time.

firstCause

protected final AtomicReference<Throwable> firstCause

startCount

protected final AtomicInteger startCount

doneCount

protected final AtomicInteger doneCount

Constructor Detail

CompactTask

public CompactTask(Journal src,
                   File outFile,
                   long commitTime)

The task reads the state of each named index as of the given commitTime and writes the index data in order on the output journal.

Note: Unlike the IndexSegmentBuilder, this does not produce a perfect read-optimized index. However, in many cases this task does significantly improve the locality of reference for the BTrees and will discard any deleted data or data which has been overrwritten.

Parameters:

src - The source journal.

outFile - The output file.

commitTime - The commit time whose state will be compacted onto the output file (the first commit point whose commit time is LTE to the given commit time will be used).

Method Detail

call

public Journal call()
             throws Exception

Compact the oldJournal journal onto the outFile file.

Specified by:

call in interface Callable<Journal>

Returns:

The already open Journal iff this task succeeds. If the task fails, then the Journal (if created) will have been closed. If you are backing up data, then be sure to shutdown the returned Journal so that it can release its resources.

Throws:

Exception

createJournal

protected Journal createJournal()

Create the output journal.

Returns:

The output journal.

copyIndices

protected void copyIndices(Journal newJournal)

Copy all named indices from the oldJournal to the new journal.

Note: This code is similar to that found in the OverflowManager.

Parameters:

newJournal - The new journal.