org.cdlib.xtf.util
Class PackedByteBuf

Object
  extended by PackedByteBuf

public class PackedByteBuf
extends Object

Packs ints and strings into a byte buffer, using an efficient variable-size int format. Transparently compresses and decompresses large buffers.

Author:
Martin Haye

Nested Class Summary
private  class PackedByteBuf.CompressInfo
          Keeps tracks of inflate/deflate stuff on a thread-local basis.
 
Field Summary
private  byte[] bytes
          Byte buffer to read from or write to
private  boolean compressed
          Tells whether compression succeeded
private static ThreadLocal compressInfo
          Used to compress/decompress data
private static int compressLimitMax
          Maximum buffer size to compress (above this, we wouldn't have enough bytes to store the length.)
private static int compressLimitMin
          Minimum buffer size to compress (below this, it's probably not worth even trying).
private static byte compressMarker
          Special marker used to denote a compressed buffer
private  boolean compressTried
          Tells whether we've attempted to compress the buffer
private  int pos
          Current position within the byte buffer, bytes
private  char[] strChars
          Temporary buffer used when decoding strings
private  int uncompLen
          Original (uncompressed) length of the buffer
 
Constructor Summary
PackedByteBuf(byte[] rawBytes)
          Construct a byte buffer from packed data that has been read somehow.
PackedByteBuf(DataInput in, int len)
          Construct a byte buffer for reading from.
PackedByteBuf(int initialSize)
          Construct a byte buffer for writing into.
PackedByteBuf(SubStoreReader in, int len)
          Construct a byte buffer for reading from.
 
Method Summary
 Object clone()
          Obtain a copy of this buffer (with only the valid bytes)
 void compact()
          Makes the buffer as small as possible to hold the existing data; after this operation, no more data may be added.
private  void compress()
          If the buffer hasn't been compressed yet, do so.
private  void decompress()
          Given a raw buffer, this method determines if it is compressed, and if so, decompresses it.
private  void ensureSize(int nBytes)
          Make sure that the buffer has room to hold at least nBytes bytes.
 int length()
          Returns the number of bytes currently in the buffer.
 void output(DataOutput out)
          Copy the entire contents of the buffer to an output sink.
 void output(DataOutput out, int len)
          Copy some or all of the buffer to an output sink.
 void output(SubStoreWriter out)
          Copy the entire contents of the buffer to an output sink.
 void output(SubStoreWriter out, int len)
          Copy some or all of the buffer to an output sink.
 PackedByteBuf readBuffer()
          Read a buffer that was previously packed into this one with writeBuffer().
 byte readByte()
          Read in a single byte from the buffer.
 void readBytes(byte[] bytes)
          Read in a bunch of bytes.
 void readBytes(byte[] outBytes, int start, int length)
          Read in a bunch of bytes.
 int readInt()
          Read an integer from a buffer that was previously made with writeInt().
 String readString()
          Read a string from a buffer that was previously made with writeString().
 void reset()
          Resets the buffer so that reads/writes occur at the start.
 void setBytes(byte[] rawBytes)
          Take a chunk of raw data for unpacking.
 void skipBuffer()
          Skip over a buffer that was written with writeBuffer()
 void skipBytes(int num)
          Skip a bunch of bytes
 void skipInt()
          Skip over an integer made with writeInt()
 void skipString()
          Skip over a string that was written with writeString()
 void writeBuffer(PackedByteBuf b)
          Write another buffer into this one.
 void writeByte(byte b)
          Write out a single byte.
 void writeBytes(byte[] b)
          Write out a bunch of bytes
 void writeBytes(byte[] b, int offset, int length)
          Write out a bunch of bytes
 void writeCharSequence(CharSequence s)
          Write a general character sequence to the buffer, using an efficient format.
 void writeInt(int n)
          Write a (non-negative) integer to the buffer.
 void writeString(String s)
          Write a string to the buffer, using an efficient format.
 
Methods inherited from class Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

bytes

private byte[] bytes
Byte buffer to read from or write to


pos

private int pos
Current position within the byte buffer, bytes


strChars

private char[] strChars
Temporary buffer used when decoding strings


compressTried

private boolean compressTried
Tells whether we've attempted to compress the buffer


compressed

private boolean compressed
Tells whether compression succeeded


uncompLen

private int uncompLen
Original (uncompressed) length of the buffer


compressInfo

private static ThreadLocal compressInfo
Used to compress/decompress data


compressMarker

private static final byte compressMarker
Special marker used to denote a compressed buffer

See Also:
Constant Field Values

compressLimitMin

private static final int compressLimitMin
Minimum buffer size to compress (below this, it's probably not worth even trying). Initial value: 100.

See Also:
Constant Field Values

compressLimitMax

private static final int compressLimitMax
Maximum buffer size to compress (above this, we wouldn't have enough bytes to store the length.) Initial value: 65000

See Also:
Constant Field Values
Constructor Detail

PackedByteBuf

public PackedByteBuf(int initialSize)
Construct a byte buffer for writing into.

Parameters:
initialSize - Hint as to the likely maximum amount of data bytes. If this is exceeded, the buffer will expand itself automatically.

PackedByteBuf

public PackedByteBuf(SubStoreReader in,
                     int len)
              throws IOException
Construct a byte buffer for reading from. This constructor reads a chunk of data from 'in', starting at its current position.

Parameters:
in - Source for data
len - How many bytes to read
Throws:
IOException

PackedByteBuf

public PackedByteBuf(DataInput in,
                     int len)
              throws IOException
Construct a byte buffer for reading from. This constructor reads a chunk of data from 'in', starting at its current position.

Parameters:
in - Source for data
len - How many bytes to read
Throws:
IOException

PackedByteBuf

public PackedByteBuf(byte[] rawBytes)
Construct a byte buffer from packed data that has been read somehow.

Parameters:
rawBytes - The raw data
Method Detail

setBytes

public void setBytes(byte[] rawBytes)
Take a chunk of raw data for unpacking.

Parameters:
rawBytes - The raw data

decompress

private void decompress()
Given a raw buffer, this method determines if it is compressed, and if so, decompresses it. Note that 'pos' may come out non-zero; reading should start at 'pos', not zero.


clone

public Object clone()
Obtain a copy of this buffer (with only the valid bytes)

Overrides:
clone in class Object

writeByte

public void writeByte(byte b)
Write out a single byte.


writeBytes

public void writeBytes(byte[] b)
Write out a bunch of bytes


writeBytes

public void writeBytes(byte[] b,
                       int offset,
                       int length)
Write out a bunch of bytes


writeInt

public void writeInt(int n)
Write a (non-negative) integer to the buffer. Writes between 1 and 5 bytes, depending on the size of the number.

Parameters:
n - The number to write

writeString

public void writeString(String s)
Write a string to the buffer, using an efficient format.

Parameters:
s - The string to write.

writeCharSequence

public void writeCharSequence(CharSequence s)
Write a general character sequence to the buffer, using an efficient format.

Parameters:
s - The sequence to write.

writeBuffer

public void writeBuffer(PackedByteBuf b)
Write another buffer into this one.

Parameters:
b - The buffer to write.

ensureSize

private void ensureSize(int nBytes)
Make sure that the buffer has room to hold at least nBytes bytes. If not, it's expanded to make room.


reset

public void reset()
Resets the buffer so that reads/writes occur at the start. Also sets length() to zero (though reads can occur past length()).


compress

private void compress()
If the buffer hasn't been compressed yet, do so. After compression, no more bytes may be added.


compact

public void compact()
Makes the buffer as small as possible to hold the existing data; after this operation, no more data may be added.


length

public int length()
Returns the number of bytes currently in the buffer. After this operation, no more data may be added.


output

public void output(SubStoreWriter out)
            throws IOException
Copy the entire contents of the buffer to an output sink.

Parameters:
out - Where to write the data to.
Throws:
IOException

output

public void output(DataOutput out)
            throws IOException
Copy the entire contents of the buffer to an output sink.

Parameters:
out - Where to write the data to.
Throws:
IOException

output

public void output(SubStoreWriter out,
                   int len)
            throws IOException
Copy some or all of the buffer to an output sink. Note that if 'len' is greater than the buffer's size, the output will be padded at the end with zeros.

Parameters:
out - Where to write the data to
len - How many bytes to write (okay to exceed buffer length)
Throws:
IOException

output

public void output(DataOutput out,
                   int len)
            throws IOException
Copy some or all of the buffer to an output sink. Note that if 'len' is greater than the buffer's size, the output will be padded at the end with zeros.

Parameters:
out - Where to write the data to
len - How many bytes to write (okay to exceed buffer length)
Throws:
IOException

readByte

public byte readByte()
Read in a single byte from the buffer.


readBytes

public void readBytes(byte[] bytes)
Read in a bunch of bytes.


readBytes

public void readBytes(byte[] outBytes,
                      int start,
                      int length)
Read in a bunch of bytes.


skipBytes

public void skipBytes(int num)
Skip a bunch of bytes


readInt

public int readInt()
Read an integer from a buffer that was previously made with writeInt().


skipInt

public void skipInt()
Skip over an integer made with writeInt()


readString

public String readString()
Read a string from a buffer that was previously made with writeString().


skipString

public void skipString()
Skip over a string that was written with writeString()


readBuffer

public PackedByteBuf readBuffer()
Read a buffer that was previously packed into this one with writeBuffer().


skipBuffer

public void skipBuffer()
Skip over a buffer that was written with writeBuffer()