public abstract class DistinctFilter<E> extends Object implements IChunkConverter<E,E>
ISolutions generated by an
IRule. The filter is optimized if only a single chunk is visited by the source iterator. Otherwise, the filter is implemented using a
BTreebacked by a
When more than one chunk is processed,
ISolutions are transformed
into unsigned byte keys. The
BTree is tested for each such key. If
the key is NOT found, then it is inserted into the
BTree and the
solution is passed by the filter. Otherwise the solution is rejected by the
filter. The backing
BTree is closed when the filter is finalized, but
it will hold a hard reference to the
TemporaryStore until then.
Solutions are processed in chunks for efficient ordered reads and writes on
BTreebut the bloom filter parameters MUST be chosen so as to make the possibility of a false positive sufficiently unlikely to satisfy the application criteria. However, such a filter will always have a non-zero chance of incorrectly rejecting a solution when that solution has NOT been seen by the filter. Since the bloom filter can under-generate, it could only be applied in very specialized circumstances, e.g., it might be OK for text search.
|Constructor and Description|
|Modifier and Type||Method and Description|
Convert the next chunk of element(s) from the source iterator into target element(s).
Return an unsigned byte key that is a representation of the visited element.
public E convert(IChunkedOrderedIterator<E> src)
Note: This method will only be invoked if
for the source iterator.
Note: Iterators are single-threaded so the implementation of this method does not need to be thread-safe.
protected abstract byte getSortKey(E e)
e- The visited element.
Copyright © 2006-2012 SYSTAP, LLC. All Rights Reserved.