com.bigdata.io
Class ByteArrayBuffer

java.lang.Object
  java.io.OutputStream
      com.bigdata.io.ByteArrayBuffer

All Implemented Interfaces:

IByteArrayBuffer, IByteArraySlice, IDataRecord, it.unimi.dsi.fastutil.io.RepositionableStream, Closeable, Flushable

Direct Known Subclasses:

DataOutputBuffer

public class ByteArrayBuffer
extends OutputStream
implements IByteArrayBuffer, it.unimi.dsi.fastutil.io.RepositionableStream

A view on a mutable byte[] that may be extended.

Note: The backing byte[] slice always has an offset of ZERO (0) and a length equal to the capacity of the backing byte[]. The length is automatically extended iff the backing buffer is extended.

Note: This class implements OutputStream so that it may be wrapped by an OutputBitStream. Likewise it implements RepositionableStream so that you can rewind the OutputBitStream.

Note: The concept of limit() and capacity() are slightly different than for ByteBuffer since the limit and the capacity may be transparently extended on write. However, neither the limit nor the capacity will be extended on read (this prevents you from reading bad data). The relative put operations always set the limit to the position as a post-condition of the operation. The absolute get/put operations ignore the limit entirely.

This class is NOT thread-safe for mutation. Not only is the underlying FixedByteArrayBuffer not thread-safe for mutation, but the operation which replaces the slice(int, int) when the capacity of the backing buffer must be extended is not atomic.

Version:

$Id: ByteArrayBuffer.java 2265 2009-10-26 12:51:06Z thompsonbry $

Author:

Bryan Thompson

Field Summary

static int	DEFAULT_INITIAL_CAPACITY The default capacity of the buffer.
protected static boolean	INFO
protected static org.apache.log4j.Logger	log

Constructor Summary

ByteArrayBuffer()
Creates a buffer with an initial capacity of 1024 bytes.

ByteArrayBuffer(int initialCapacity)
Creates a buffer with the specified initial capacity.

ByteArrayBuffer(int pos, int limit, byte[] buf)
Create a new buffer backed by the given array.

Method Summary

byte[]	array() The backing byte[] WILL be transparently replaced if the buffer capacity is extended.
ByteBuffer	asByteBuffer() Wraps up a reference to the data in a ByteBuffer between the position and the limit.
protected static int	assertNonNegative(String msg, int v) Throws exception unless the value is non-negative.
int	capacity() The capacity of the buffer.
int	copy(ByteBuffer src) Relative copy of data into this buffer.
int	copyAll(ByteArrayBuffer src) Relative copy data from the origin (offset ZERO) of the source buffer up to its read limit into this buffer.
int	copyRest(ByteArrayBuffer src) Relative copy data from the current position of the source buffer up to its read limit into this buffer.
void	ensureCapacity(int capacity) Ensure that the buffer capacity is a least capacity total bytes.
void	ensureFree(int len) Ensure that at least len bytes are free in the buffer.
void	ensureFree(int pos, int len) Ensure that at least len bytes are free in the buffer starting at pos.
protected int	extend(int required) Return the new capacity for the buffer (default is always large enough and will normally double the buffer capacity each time it overflows).
void	flip() Sets the position to zero but leaves the read limit at the old position.
void	get(int srcoff, byte[] dst) Absolute bulk get copies dst.length bytes from the specified offset into the slice defined by this buffer into the caller's array.
void	get(int srcoff, byte[] dst, int dstoff, int dstlen) Absolute bulk get copies the specified slice of bytes from this buffer into the specified slice of the caller's array.
boolean	getBit(long bitIndex) Get the value of a bit.
byte	getByte() Relative get method for reading a byte value.
byte	getByte(int pos) Absolute get for reading a byte value.
double	getDouble()
double	getDouble(int pos)
float	getFloat()
float	getFloat(int pos)
int	getInt()
int	getInt(int pos)
long	getLong()
long	getLong(int pos)
protected static int	getNibbleLength(long v) Return the #of non-zero nibbles, counting from the first non-zero nibble in the long value.
it.unimi.dsi.io.OutputBitStream	getOutputBitStream() Return a bit stream which will write on this buffer.
short	getShort()
short	getShort(int pos)
int	len() The length of the slice is always the capacity of the backing byte[].
int	limit() The read limit (there is no write limit on the buffer since the capacity will be automatically extended on overflow).
int	mark() Sets the mark.
int	off() The offset of the slice into the backing byte[] is always zero.
int	packLong(long v) Packs a non-negative long value into the minimum #of bytes in which the value can be represented and writes those bytes onto the buffer.
int	packShort(short v) Packs a non-negative short value into one or two bytes and writes them on the buffer.
int	pos() The current position in the buffer.
int	pos(int pos) Set the position in the buffer (does not change the limit).
long	position()
void	position(long v)
void	put(byte[] b) Relative put method for writing a byte[] on the buffer.
void	put(byte[] b, int off, int len) Relative put method for writing a byte[] on the buffer.
void	put(int pos, byte[] b) Absolute bulk put copies all bytes in the caller's array into this buffer starting at the specified position within the slice defined by this buffer.
void	put(int pos, byte[] b, int off, int len) Absolute bulk put copies the specified slice of bytes from the caller's array into this buffer starting at the specified position within the slice defined by this buffer.
void	putByte(byte v) Relative put method for writing a byte value.
void	putByte(int pos, byte v) Absolute put method for writing a byte value.
void	putDouble(double v)
void	putDouble(int pos, double d)
void	putFloat(float v)
void	putFloat(int pos, float f)
void	putInt(int v)
void	putInt(int pos, int v)
void	putLong(int pos, long v)
void	putLong(long v)
void	putShort(int pos, short v)
void	putShort(short v)
int	remaining() The #of bytes remaining in the buffer for relative read operations (limit - pos).
ByteArrayBuffer	reset() Prepares the buffer for new data by resetting the position and limit to zero.
int	rewind() Rewinds the buffer to the mark.
boolean	setBit(long bitIndex, boolean value) Set the value of a bit.
int	skip(int nbytes) Skip forward or backward the specified number of bytes.
AbstractFixedByteArrayBuffer	slice(int off, int len) Return a slice of the backing buffer.
byte[]	toByteArray() Return a copy of the data written on the buffer (the bytes in [0:pos]).
byte[]	trim() Trims the backing byte[] to an exact fit by making a copy of the data in the buffer and returns the old byte[].
void	write(byte[] b)
void	write(byte[] b, int off, int len)
void	write(int b)

Methods inherited from class java.io.OutputStream

close, flush

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected static final org.apache.log4j.Logger log

INFO

protected static final boolean INFO

DEFAULT_INITIAL_CAPACITY

public static final int DEFAULT_INITIAL_CAPACITY

The default capacity of the buffer.

See Also:

Constant Field Values

Constructor Detail

ByteArrayBuffer

public ByteArrayBuffer()

Creates a buffer with an initial capacity of 1024 bytes. The position and the read limit will zero. The capacity of the buffer will be automatically extended as required.

ByteArrayBuffer

public ByteArrayBuffer(int initialCapacity)

Creates a buffer with the specified initial capacity. The position and the read limit will be zero. The capacity of the buffer will be automatically extended as required.

Parameters:

initialCapacity - The initial capacity.

ByteArrayBuffer

public ByteArrayBuffer(int pos,
                       int limit,
                       byte[] buf)

Create a new buffer backed by the given array. The initial capacity will be the size of the given byte[]. The mark will be zero. The capacity of the buffer will be automatically extended as required.

Note: The buffer reference is used directly rather than making a copy of the data.

Parameters:

pos - The initial position.

limit - The initial limit().

buf - The backing byte[].

Method Detail

array

public final byte[] array()

The backing byte[] WILL be transparently replaced if the buffer capacity is extended. The backing byte[]. This method DOES NOT guarantee that the backing array reference will remain constant. Some implementations use an extensible backing byte[] and will replace the reference when the backing buffer is extended. This is re-allocated whenever the capacity of the buffer is too small and reused otherwise.

Specified by:

array in interface IByteArrayBuffer

Specified by:

array in interface IByteArraySlice

off

public final int off()

The offset of the slice into the backing byte[] is always zero.

Specified by:

off in interface IByteArrayBuffer

Specified by:

off in interface IByteArraySlice

len

public final int len()

The length of the slice is always the capacity of the backing byte[].

Specified by:

len in interface IByteArrayBuffer

Specified by:

len in interface IByteArraySlice

assertNonNegative

protected static int assertNonNegative(String msg,
                                       int v)

Throws exception unless the value is non-negative.

Parameters:

msg - The exception message.

v - The value.

Returns:

The value.

Throws:

IllegalArgumentException - unless the value is non-negative.

ensureFree

public final void ensureFree(int pos,
                             int len)

Ensure that at least len bytes are free in the buffer starting at pos. The buffer may be grown by this operation but it will not be truncated.

This operation is equivilent to

 ensureCapacity(pos + len)
 

and the latter is often used as an optimization.

Parameters:

pos - The position in the buffer.

len - The minimum #of free bytes.

ensureCapacity

public void ensureCapacity(int capacity)

Ensure that the buffer capacity is a least capacity total bytes. The buffer may be grown by this operation but it will not be truncated.

Parameters:

capacity - The minimum #of bytes in the buffer.

TODO:

this can be potentially overridden in a derived class to only copy those bytes up to the current position, which would be somewhat faster.

capacity

public final int capacity()

Description copied from interface: IByteArrayBuffer

The capacity of the buffer.

Specified by:

capacity in interface IByteArrayBuffer

extend

protected int extend(int required)

Return the new capacity for the buffer (default is always large enough and will normally double the buffer capacity each time it overflows).

Parameters:

required - The minimum required capacity.

Returns:

The new capacity.

TODO:

this does not need to be final. also, caller's could set the policy including a policy that refuses to extend the capacity.

trim

public final byte[] trim()

Trims the backing byte[] to an exact fit by making a copy of the data in the buffer and returns the old byte[]. All bytes between off() and limit() are copied into the exact fit byte[].

Note: A slice(int, int) with a view of this ByteArrayBuffer will continue to have a view onto the backing buffer associated with this instance. This works because SliceImpl#array() delegates to the outer array() method. This makes it possible to trim() a ByteArrayBuffer on which you have slice(int, int)s, while maintaining the validity of those slices.

Returns:

The old byte[].

put

public final void put(int pos,
                      byte[] b)

Description copied from interface: IDataRecord

Absolute bulk put copies all bytes in the caller's array into this buffer starting at the specified position within the slice defined by this buffer.

Specified by:

put in interface IDataRecord

Parameters:

pos - The starting position within the slice defined by this buffer.

b - The source data.

put

public final void put(int pos,
                      byte[] b,
                      int off,
                      int len)

Description copied from interface: IDataRecord

Absolute bulk put copies the specified slice of bytes from the caller's array into this buffer starting at the specified position within the slice defined by this buffer.

Specified by:

put in interface IDataRecord

Parameters:

pos - The offset into the slice to which the data will be copied.

b - The source data.

off - The offset of the 1st byte in the source data to be copied.

len - The #of bytes to be copied.

get

public final void get(int srcoff,
                      byte[] dst)

Description copied from interface: IDataRecord

Absolute bulk get copies dst.length bytes from the specified offset into the slice defined by this buffer into the caller's array.

Specified by:

get in interface IDataRecord

Parameters:

srcoff - The offset into the slice of the first byte to be copied.

dst - The array into which the data will be copied.

get

public final void get(int srcoff,
                      byte[] dst,
                      int dstoff,
                      int dstlen)

Description copied from interface: IDataRecord

Absolute bulk get copies the specified slice of bytes from this buffer into the specified slice of the caller's array.

Specified by:

get in interface IDataRecord

Parameters:

srcoff - The offset into the slice defined by this buffer of the first byte to be copied.

dst - The array into which the data will be copied.

dstoff - The offset of the first byte in that array onto which the data will be copied.

dstlen - The #of bytes to be copied.

putByte

public final void putByte(int pos,
                          byte v)

Description copied from interface: IDataRecord

Absolute put method for writing a byte value.

Specified by:

putByte in interface IDataRecord

Parameters:

pos - The offset into the slice.

v - The value.

getByte

public final byte getByte(int pos)

Description copied from interface: IDataRecord

Absolute get for reading a byte value.

Specified by:

getByte in interface IDataRecord

Parameters:

pos - The offset into the slice.

Returns:

The byte value at that offset.

putShort

public final void putShort(int pos,
                           short v)

Specified by:

putShort in interface IDataRecord

getShort

public final short getShort(int pos)

Specified by:

getShort in interface IDataRecord

putInt

public final void putInt(int pos,
                         int v)

Specified by:

putInt in interface IDataRecord

getInt

public final int getInt(int pos)

Specified by:

getInt in interface IDataRecord

putFloat

public final void putFloat(int pos,
                           float f)

Specified by:

putFloat in interface IDataRecord

getFloat

public final float getFloat(int pos)

Specified by:

getFloat in interface IDataRecord

putLong

public final void putLong(int pos,
                          long v)

Specified by:

putLong in interface IDataRecord

putDouble

public final void putDouble(int pos,
                            double d)

Specified by:

putDouble in interface IDataRecord

getLong

public final long getLong(int pos)

Specified by:

getLong in interface IDataRecord

getDouble

public final double getDouble(int pos)

Specified by:

getDouble in interface IDataRecord

toByteArray

public final byte[] toByteArray()

Return a copy of the data written on the buffer (the bytes in [0:pos]).

Specified by:

toByteArray in interface IDataRecord

Returns:

A new array containing data in the buffer.

See Also:

asByteBuffer()

TODO:

this returns the data in [0:pos], which is essentially the data written on the buffer using relative put operations but without requiring a flip() to set the pos to zero while leaving the read limit alone. that is at odds with the rest of the relative api but there is a lot of use of this method already. perhaps it can be moved to the DataOutputBuffer where it would share semantics with ByteArrayOutputStream.toByteArray()?

remaining

public final int remaining()

The #of bytes remaining in the buffer for relative read operations (limit - pos).

Specified by:

remaining in interface IByteArrayBuffer

pos

public final int pos()

The current position in the buffer.

Specified by:

pos in interface IByteArrayBuffer

pos

public final int pos(int pos)

Set the position in the buffer (does not change the limit).

Parameters:

pos - The new position, must be in [0:capacity()).

Returns:

The old position.

Throws:

IllegalArgumentException - if pos is less than ZERO (0).

IllegalArgumentException - if pos is greater than or equal to the current capacity of the buffer.

limit

public final int limit()

The read limit (there is no write limit on the buffer since the capacity will be automatically extended on overflow).

Specified by:

limit in interface IByteArrayBuffer

flip

public final void flip()

Sets the position to zero but leaves the read limit at the old position. After invoking this method you can use relative get methods to read all data up to the read limit.

reset

public ByteArrayBuffer reset()

Prepares the buffer for new data by resetting the position and limit to zero.

Returns:

This buffer.

ensureFree

public final void ensureFree(int len)

Ensure that at least len bytes are free in the buffer. The buffer may be grown by this operation but it will not be truncated.

This operation is equivalent to

 ensureCapacity(this.len + len)
 

and the latter is often used as an optimization.

Parameters:

len - The minimum #of free bytes.

mark

public final int mark()

Sets the mark.

Returns:

the old mark (initially zero).

rewind

public final int rewind()

Rewinds the buffer to the mark. Does not change the limit.

Returns:

The new pos().

copy

public final int copy(ByteBuffer src)

Relative copy of data into this buffer.

Parameters:

src - The source.

Returns:

The #of bytes copied.

copyRest

public final int copyRest(ByteArrayBuffer src)

Relative copy data from the current position of the source buffer up to its read limit into this buffer.

Parameters:

src - The source buffer.

Returns:

The #of bytes copied.

See Also:

copyAll(ByteArrayBuffer)

copyAll

public final int copyAll(ByteArrayBuffer src)

Relative copy data from the origin (offset ZERO) of the source buffer up to its read limit into this buffer.

Parameters:

src - The source buffer.

Returns:

The #of bytes copied.

See Also:

copyRest(ByteArrayBuffer)

asByteBuffer

public final ByteBuffer asByteBuffer()

Wraps up a reference to the data in a ByteBuffer between the position and the limit.

Specified by:

asByteBuffer in interface IDataRecord

Returns:

A ByteBuffer encapsulating a reference to the data in the current buffer. The data will be overwritten if reset() is invoked followed by any operations that write on the buffer.

put

public final void put(byte[] b)

Relative put method for writing a byte[] on the buffer.

Parameters:

b - The byte[].

put

public final void put(byte[] b,
                      int off,
                      int len)

Relative put method for writing a byte[] on the buffer.

Parameters:

b - The byte[].

off - The offset of the first byte in b to be written on the buffer.

len - The #of bytes in b to be written on the buffer.

putByte

public final void putByte(byte v)

Relative put method for writing a byte value.

Parameters:

v - The value.

TODO:

rename as put(byte) to conform with ByteBuffer?

getByte

public final byte getByte()

Relative get method for reading a byte value.

Returns:

The value.

Throws:

IndexOutOfBoundsException - if the position is greater than or equal to the limit.

TODO:

rename as get(byte) to conform with ByteBuffer?

putShort

public final void putShort(short v)

getShort

public final short getShort()

putInt

public final void putInt(int v)

getInt

public final int getInt()

putFloat

public final void putFloat(float v)

getFloat

public final float getFloat()

putLong

public final void putLong(long v)

getLong

public final long getLong()

putDouble

public final void putDouble(double v)

getDouble

public final double getDouble()

packLong

public final int packLong(long v)

Packs a non-negative long value into the minimum #of bytes in which the value can be represented and writes those bytes onto the buffer. The first byte determines whether or not the long value was packed and, if packed, how many bytes were required to represent the packed long value. When the high bit of the first byte is a one (1), then the long value could not be packed and the long value is found by clearing the high bit and interpreting the first byte plus the next seven (7) bytes as a long. Otherwise the next three (3) bits are interpreted as an unsigned integer giving the #of bytes (nbytes) required to represent the packed long value. To recover the long value the high nibble is cleared and the first byte together with the next nbytes are interpreted as an unsigned long value whose leading zero bytes were not written.

    
 [0|1|2|3|4|5|6|7]
  1 - - -   nbytes = 8, clear high bit and interpret this plus the next 7 bytes as a long.
  0 1 1 1   nbytes = 7, clear high nibble and interpret this plus the next 6 bytes as a long. 
  0 1 1 0   nbytes = 6, clear high nibble and interpret this plus the next 5 bytes as a long. 
  0 1 0 1   nbytes = 5, clear high nibble and interpret this plus the next 4 bytes as a long.
  0 1 0 0   nbytes = 4, clear high nibble and interpret this plus the next 3 bytes as a long.
  0 0 1 1   nbytes = 3, clear high nibble and interpret this plus the next 3 bytes as a long.
  0 0 1 0   nbytes = 2, clear high nibble and interpret this plus the next byte as a long.
  0 0 0 1   nbytes = 1, clear high nibble.  value is the low nibble.
  
 

Parameters:

v - The unsigned long value.

Returns:

The #of bytes onto which the unsigned long value was packed.

getNibbleLength

protected static final int getNibbleLength(long v)

Return the #of non-zero nibbles, counting from the first non-zero nibble in the long value. A value of 0L is considered to be one nibble for our purposes.

Parameters:

v - The long value.

Returns:

The #of nibbles in [1:16].

packShort

public final int packShort(short v)

Packs a non-negative short value into one or two bytes and writes them on the buffer. A short in [0:127] is packed into one byte. Larger values are packed into two bytes. The high bit of the first byte is set if the value was packed into two bytes. If the bit is set, clear the high bit, read the next byte, and interpret the two bytes as a short value. Otherwise interpret the byte as a short value.

Parameters:

v - The unsigned short integer.

Returns:

The #of bytes into which the value was packed.

write

public final void write(int b)
                 throws IOException

Specified by:

write in class OutputStream

Throws:

IOException

write

public final void write(byte[] b)
                 throws IOException

Overrides:

write in class OutputStream

Throws:

IOException

write

public final void write(byte[] b,
                        int off,
                        int len)
                 throws IOException

Overrides:

write in class OutputStream

Throws:

IOException

position

public long position()
              throws IOException

Specified by:

position in interface it.unimi.dsi.fastutil.io.RepositionableStream

Throws:

IOException

position

public void position(long v)
              throws IOException

Specified by:

position in interface it.unimi.dsi.fastutil.io.RepositionableStream

Throws:

IOException

getBit

public final boolean getBit(long bitIndex)

Description copied from interface: IDataRecord

Get the value of a bit.

Specified by:

getBit in interface IDataRecord

Parameters:

bitIndex - The index of the bit, counting from the first bit position in the slice.

Returns:

The value of the bit.

setBit

public final boolean setBit(long bitIndex,
                            boolean value)

Description copied from interface: IDataRecord

Set the value of a bit.

Specified by:

setBit in interface IDataRecord

Parameters:

bitIndex - The index of the bit, counting from the first bit position in the slice.

Returns:

The old value of the bit.

skip

public int skip(int nbytes)

Skip forward or backward the specified number of bytes.

Parameters:

nbytes - The #of bytes to skip (MAY be negative).

Returns:

The new position.

slice

public AbstractFixedByteArrayBuffer slice(int off,
                                          int len)

Return a slice of the backing buffer. The slice will always reference the current backing array(), even when the buffer is extended and the array reference is replaced. The pos() and limit() are ignored by this method.

Specified by:

slice in interface IDataRecord

Parameters:

off - The starting offset into the backing buffer of the slice.

len - The length of that slice.

Returns:

The slice.

getOutputBitStream

public it.unimi.dsi.io.OutputBitStream getOutputBitStream()

Return a bit stream which will write on this buffer. The stream will begin at the current pos().

Returns:

The bit stream.

TODO:

Add OBS(byte[],off,len) ctor for faster operations. However, you MUST pre-extend the buffer to have sufficient capacity since an OutputBitStream wrapping a byte[] will not auto-extend.