com.bigdata.rwstore
Class StorageStats

java.lang.Object
  com.bigdata.rwstore.StorageStats

public class StorageStats
extends Object

Maintains stats on the RWStore allocations, useful for tuning Allocator sizes and tracking store efficiency. It can also track reallocation patterns that are lost in static snapshots of current usage. Stats are retained on external requests and also on internal allocator use. totalSlots represents the total reserved slots in all the AllocationBlocks for the associated FixedAllocators. slotAllocations is the total of all allocations made slotDeletes is the total of all slot deletions Therefore the total of currently allocated slots is slotAllocations - slotDeletes sizeAllocations is the total in bytes of the actual data stored in the slots sizeDeletes is the total in bytes of the actual data that has been deleted Therefore the size of the total in bytes stored in currently allocated slots is sizeAllocations - sizeDeletes

Author:

Martyn Cutcher

Nested Class Summary

class	StorageStats.BlobBucket
class	StorageStats.Bucket

Constructor Summary

StorageStats(DataInputStream instr)

StorageStats(int[] buckets)

Method Summary

void	allocateBlob(int sze)
void	deleteBlob(int sze)
byte[]	getData()
void	register(FixedAllocator alloc)
void	register(FixedAllocator alloc, boolean init)
void	register(SectorAllocator allocator, boolean init)
void	showStats(StringBuilder str) Collected statistics are against each Allocation Block size: AllocatorSizeThe #of bytes in the allocated slots issued by this allocator. AllocatorCountThe #of fixed allocators for that slot size. SlotsInUseThe difference between the two previous columns (net slots in use for this slot size). SlotsReservedThe #of slots in this slot size which have had storage reserved for them. SlotsAllocatedCumulative allocation of slots to date in this slot size (regardless of the transaction outcome). SlotsRecycledCumulative recycled slots to date in this slot size (regardless of the transaction outcome). SlotsChurnHow frequently slots of this size are re-allocated (SlotsInUse/SlotsAllocated). %SlotsUnusedThe percentage of slots of this size which are not in use (1-(SlotsInUse/SlotsReserved)). BytesReservedThe space reserved on the backing file for those allocation slots BytesAppDataThe #of bytes in the allocated slots which are used by application data (including the record checksum). %SlotWasteHow well the application data fits in the slots (BytesAppData/(SlotsInUse*AllocatorSize)). %AppDataHow much of your data is stored by each allocator (BytesAppData/Sum(BytesAppData)). %StoreFileHow much of the backing file is reserved for each allocator (BytesReserved/Sum(BytesReserved)). %StoreWasteHow much of the total waste on the store is waste for this allocator size ((BytesReserved-BytesAppData)/(Sum(BytesReserved)-Sum(BytesAppData))).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StorageStats

public StorageStats(int[] buckets)

Parameters:

buckets - - the slot sizes used by the FixedAllocators

StorageStats

public StorageStats(DataInputStream instr)
             throws IOException

Parameters:

instr - restore from reopen

Throws:

IOException

Method Detail

getData

public byte[] getData()
               throws IOException

Throws:

IOException

allocateBlob

public void allocateBlob(int sze)

deleteBlob

public void deleteBlob(int sze)

register

public void register(FixedAllocator alloc,
                     boolean init)

register

public void register(FixedAllocator alloc)

showStats

public void showStats(StringBuilder str)

Collected statistics are against each Allocation Block size:

AllocatorSize

The #of bytes in the allocated slots issued by this allocator.

AllocatorCount

The #of fixed allocators for that slot size.

SlotsInUse

The difference between the two previous columns (net slots in use for this slot size).

SlotsReserved

The #of slots in this slot size which have had storage reserved for them.

SlotsAllocated

Cumulative allocation of slots to date in this slot size (regardless of the transaction outcome).

SlotsRecycled

Cumulative recycled slots to date in this slot size (regardless of the transaction outcome).

SlotsChurn

How frequently slots of this size are re-allocated (SlotsInUse/SlotsAllocated).

%SlotsUnused

The percentage of slots of this size which are not in use (1-(SlotsInUse/SlotsReserved)).

BytesReserved

The space reserved on the backing file for those allocation slots

BytesAppData

The #of bytes in the allocated slots which are used by application data (including the record checksum).

%SlotWaste

How well the application data fits in the slots (BytesAppData/(SlotsInUse*AllocatorSize)).

%AppData

How much of your data is stored by each allocator (BytesAppData/Sum(BytesAppData)).

%StoreFile

How much of the backing file is reserved for each allocator (BytesReserved/Sum(BytesReserved)).

%StoreWaste

How much of the total waste on the store is waste for this allocator size ((BytesReserved-BytesAppData)/(Sum(BytesReserved)-Sum(BytesAppData))).

Parameters:

str - FIXME Javadoc edit - this has diverged from the comments above. Also, there is also a divideByZero which can appear (this has been fixed).

     [java] Exception in thread "main" java.lang.ArithmeticException: / by zero
     [java]     at java.math.BigDecimal.divideAndRound(BigDecimal.java:1407)
     [java]     at java.math.BigDecimal.divide(BigDecimal.java:1381)
     [java]     at java.math.BigDecimal.divide(BigDecimal.java:1491)
     [java]     at com.bigdata.rwstore.StorageStats$Bucket.slotsUnused(StorageStats.java:240)
     [java]     at com.bigdata.rwstore.StorageStats.showStats(StorageStats.java:448)
     [java]     at com.bigdata.rwstore.RWStore.showAllocators(RWStore.java:2620)
     [java]     at com.bigdata.rdf.store.DataLoader.main(DataLoader.java:1415)
     

register

public void register(SectorAllocator allocator,
                     boolean init)