com.bigdata.btree
Class BTreeCounters

java.lang.Object
  com.bigdata.btree.BTreeCounters

All Implemented Interfaces:

Cloneable

public final class BTreeCounters
extends Object
implements Cloneable

A helper class that collects statistics on an AbstractBTree.

Note: This class DOES NOT have a hard reference to the AbstractBTree. Holding an instance of this class WILL NOT force the AbstractBTree to remain strongly reachable.

Note: Counters for mutation are plain fields. Counters for read-only operations are AtomicLong or AtomicInteger objects since read operations may involve high concurrently and could otherwise lead to lost counter updates.

Version:

$Id: BTreeCounters.java 5809 2011-12-19 16:56:48Z thompsonbry $

Author:

Bryan Thompson

TODO:

add(BTreeCounters), subtract(BTreeCounters) and the copy ctor are not atomic. I have not seen any problems resulting from this., instrument the key search time, instrument bloom filter time when the bloom filter is used (this should be done separately on the bloom filter object)., (I should recheck these costs now that that refactor has been done). At present, deserialization is much slower than serialization, primarily because of object creation. Changing to a raw record format for nodes and leaves would likely reduce the deserialization costs significantly. The time could also be reduced by lazy decompression of the values and lazy deserialization of the values such that tuple scans for keys only do not require value deserialization.

Field Summary

AtomicLong	bytesOnStore_nodesAndLeaves The #of bytes in node and leaf) records on the backing store for the unisolated view of the index.
AtomicLong	bytesOnStore_rawRecords The #of bytes in the unisolated view of the index which are being used to store raw records.
CAT	bytesRead
long	bytesReleased
long	bytesWritten
CAT	deserializeNanos De-serialization time for nodes and leaves.
int	headSplit
int	leavesCopyOnWrite
int	leavesJoined
CAT	leavesRead #of leaf read operations.
int	leavesSplit
int	leavesWritten
CAT	ngetKey
CAT	ngetValue
CAT	nindexOf
AtomicLong	ninserts #of keys looked up in the tree by contains/lookup(key) (does not count those rejected by the bloom filter before they are tested against the B+Tree).
int	nodesCopyOnWrite
int	nodesJoined
CAT	nodesRead #of node read operations.
int	nodesSplit
int	nodesWritten
CAT	nrangeCount
CAT	nrangeIterator
AtomicLong	nremoves
long	ntupleInsertDelete #of deleted tuples that were inserted into the B+Tree (rather than deleting the value for an existing tuple).
long	ntupleInsertValue #of non-deleted tuples that were inserted into the B+Tree (rather than updating the value for an existing tuple).
long	ntupleRemove #of pre-existing tuples that were removed from the B+Tree (only non-zero when the B+Tree does not support delete markers).
long	ntupleUpdateDelete #of pre-existing un-deleted tuples whose delete marker was set (we don't count re-deletes of an already deleted tuple).
long	ntupleUpdateValue #of pre-existing tuples whose value was updated to a non-deleted value (includes update of a deleted tuple to a non-deleted tuple by overwrite of the tuple).
CAT	rawRecordsBytesRead Total bytes read for raw records.
long	rawRecordsBytesWritten
CAT	rawRecordsRead The #of raw record read operations.
long	rawRecordsWritten
CAT	readNanos Read time for nodes and leaves (but not raw records).
int	rootsJoined
int	rootsSplit
long	serializeNanos
int	tailSplit
long	writeNanos

Constructor Summary

BTreeCounters()

BTreeCounters(BTreeCounters c)
Copy constructor.

Method Summary

void	add(BTreeCounters o) Adds the values from another BTreeCounters object to this one.
BTreeCounters	clone()
double	computeRawReadScore() Return a score whose increasing value is correlated with the amount of read activity on an index as reflected in these BTreeCounters.
double	computeRawReadWriteScore() Return a score whose increasing value is correlated with the amount of read/write activity on an index as reflected in these BTreeCounters.
double	computeRawWriteScore() Return a score whose increasing value is correlated with the amount of write activity on an index as reflected in these BTreeCounters.
boolean	equals(Object o) Equal iff they are the same instance.
long	getBytesRead() The number of bytes read from the backing store.
long	getBytesReleased() The number of bytes released from the backing store.
long	getBytesWritten() The number of bytes written onto the backing store.
ICounterSet	getCounters() Return a CounterSet reporting on the various counters tracked in the instance fields of this class.
int	getLeavesWritten() The #of leaves written on the backing store.
int	getNodesWritten() The #of nodes written on the backing store.
static double	normalize(double rawScore, double totalRawScore) Normalizes a raw score in the context of totals for some data service.
BTreeCounters	subtract(BTreeCounters o) Subtracts the given counters from the current counters, returning a new counter object containing their difference.
String	toString()

Methods inherited from class java.lang.Object

finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

ninserts

public final AtomicLong ninserts

#of keys looked up in the tree by contains/lookup(key) (does not count those rejected by the bloom filter before they are tested against the B+Tree).

nremoves

public final AtomicLong nremoves

nindexOf

public final CAT nindexOf

ngetKey

public final CAT ngetKey

ngetValue

public final CAT ngetValue

nrangeCount

public final CAT nrangeCount

nrangeIterator

public final CAT nrangeIterator

rootsSplit

public int rootsSplit

rootsJoined

public int rootsJoined

nodesSplit

public int nodesSplit

nodesJoined

public int nodesJoined

leavesSplit

public int leavesSplit

leavesJoined

public int leavesJoined

tailSplit

public int tailSplit

headSplit

public int headSplit

nodesCopyOnWrite

public int nodesCopyOnWrite

leavesCopyOnWrite

public int leavesCopyOnWrite

ntupleInsertValue

public long ntupleInsertValue

#of non-deleted tuples that were inserted into the B+Tree (rather than updating the value for an existing tuple).

ntupleInsertDelete

public long ntupleInsertDelete

#of deleted tuples that were inserted into the B+Tree (rather than deleting the value for an existing tuple). Note that delete markers MAY be written into a B+Tree when there is no tuple for that key in the BTree. Those cases arise when the B+Tree is used to isolate a transaction's write set and are counted here.

ntupleUpdateValue

public long ntupleUpdateValue

#of pre-existing tuples whose value was updated to a non-deleted value (includes update of a deleted tuple to a non-deleted tuple by overwrite of the tuple).

ntupleUpdateDelete

public long ntupleUpdateDelete

#of pre-existing un-deleted tuples whose delete marker was set (we don't count re-deletes of an already deleted tuple).

ntupleRemove

public long ntupleRemove

#of pre-existing tuples that were removed from the B+Tree (only non-zero when the B+Tree does not support delete markers).

nodesRead

public final CAT nodesRead

#of node read operations.

leavesRead

public final CAT leavesRead

#of leaf read operations.

bytesRead

public final CAT bytesRead

readNanos

public final CAT readNanos

Read time for nodes and leaves (but not raw records).

deserializeNanos

public final CAT deserializeNanos

De-serialization time for nodes and leaves.

rawRecordsRead

public final CAT rawRecordsRead

The #of raw record read operations.

rawRecordsBytesRead

public final CAT rawRecordsBytesRead

Total bytes read for raw records.

nodesWritten

public int nodesWritten

leavesWritten

public int leavesWritten

bytesWritten

public long bytesWritten

bytesReleased

public long bytesReleased

writeNanos

public long writeNanos

serializeNanos

public long serializeNanos

rawRecordsWritten

public long rawRecordsWritten

rawRecordsBytesWritten

public long rawRecordsBytesWritten

bytesOnStore_rawRecords

public final AtomicLong bytesOnStore_rawRecords

The #of bytes in the unisolated view of the index which are being used to store raw records.

TODO:

not correctly tracked in scale-out (overflow handling).

bytesOnStore_nodesAndLeaves

public final AtomicLong bytesOnStore_nodesAndLeaves

The #of bytes in node and leaf) records on the backing store for the unisolated view of the index. This value grows and shrinks as nodes (leaves) are added to and removed from the index. The actual space occupied by the index on the backing store depends on whether and when persistence store reclaims the storage associated with deleted nodes and leaves.

TODO:

break out for nodes and leaves separately., not correctly tracked in scale-out (overflow handling).

Constructor Detail

BTreeCounters

public BTreeCounters()

BTreeCounters

public BTreeCounters(BTreeCounters c)

Copy constructor.

Parameters:

c -

Method Detail

equals

public boolean equals(Object o)

Equal iff they are the same instance.

Overrides:

equals in class Object

clone

public BTreeCounters clone()

Overrides:

clone in class Object

add

public void add(BTreeCounters o)

Adds the values from another BTreeCounters object to this one.

Parameters:

o -

subtract

public BTreeCounters subtract(BTreeCounters o)

Subtracts the given counters from the current counters, returning a new counter object containing their difference.

Parameters:

o -

computeRawReadWriteScore

public double computeRawReadWriteScore()

Return a score whose increasing value is correlated with the amount of read/write activity on an index as reflected in these BTreeCounters.

The score is the serialization / deserialization time plus the read / write time. Time was chosen since it is a common unit and since it reflects the combination of CPU time, memory time (for allocations and garbage collection - the latter can be quite significant), and the disk wait time. The other main component of time is key search, but that is not instrumented right now.

Serialization and deserialization are basically a CPU activity and drive memory to the extent that allocations are made, especially during deserialization.

The read/write time is strongly dominated by actual DISK IO and by garbage collection time (garbage collection can cause threads to be suspended at any time). For deep B+Trees, DISK READ time dominates DISK WRITE time since increasing numbers of random reads are required to materialize any given leaf.

The total read-write cost (in seconds) for a BTree is one of the factors that is considered when choosing which index partition to move. Comparing the total read/write cost for BTrees across a database can help to reveal which index partitions have the heaviest load. If only a few index partitions for a given scale-out index have a heavy load, then those index partitions are hotspots for that index.

Returns:

The computed score.

computeRawReadScore

public double computeRawReadScore()

Return a score whose increasing value is correlated with the amount of read activity on an index as reflected in these BTreeCounters. The score is deserializeNanos + readNanos.

See Also:

computeRawReadWriteScore()

computeRawWriteScore

public double computeRawWriteScore()

Return a score whose increasing value is correlated with the amount of write activity on an index as reflected in these BTreeCounters. The score is serializeNanos plus writeNanos.

See Also:

computeRawReadWriteScore()

normalize

public static double normalize(double rawScore,
                               double totalRawScore)

Normalizes a raw score in the context of totals for some data service.

Parameters:

rawScore - The raw score.

totalRawScore - The raw score computed from the totals.

Returns:

The normalized score.

getNodesWritten

public final int getNodesWritten()

The #of nodes written on the backing store.

getLeavesWritten

public final int getLeavesWritten()

The #of leaves written on the backing store.

getBytesRead

public final long getBytesRead()

The number of bytes read from the backing store.

getBytesWritten

public final long getBytesWritten()

The number of bytes written onto the backing store.

getBytesReleased

public final long getBytesReleased()

The number of bytes released from the backing store.

getCounters

public ICounterSet getCounters()

Return a CounterSet reporting on the various counters tracked in the instance fields of this class.

toString

public String toString()

Overrides:

toString in class Object