com.bigdata.relation.rule.eval.AbstractJoinNexus
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Object
public abstract class AbstractJoinNexus
Base implementation for IJoinNexus
| Field Summary | |
|---|---|
protected int |
chunkCapacity
|
protected int |
chunkOfChunksCapacity
|
protected IElementFilter<?> |
filter
|
protected IIndexManager |
indexManager
|
protected IEvaluationPlanFactory |
planFactory
The factory for rule evaluation plans. |
protected long |
readTimestamp
|
protected IResourceLocator<?> |
resourceLocator
Note: cached. |
protected int |
solutionFlags
|
| Fields inherited from interface com.bigdata.relation.rule.eval.IJoinNexus |
|---|
ALL, BINDINGS, ELEMENT, RULE |
| Constructor Summary | |
|---|---|
protected |
AbstractJoinNexus(IJoinNexusFactory joinNexusFactory,
IIndexManager indexManager)
|
| Method Summary | ||
|---|---|---|
boolean |
bind(IPredicate<?> pred,
IConstraint[] constraints,
Object e,
IBindingSet bindings)
Binds variables from a visited element. |
|
boolean |
bind(IRule rule,
int index,
Object e,
IBindingSet bindings)
Binds variables from a visited element. |
|
boolean |
forceSerialExecution()
When true, rule level parallelism is disabled and the
ISolution buffers are flushed after after every IStep. |
|
ActionEnum |
getAction()
The kind of operation that is being executed (Query, Insert, or Delete). |
|
IStreamSerializer<IBindingSet[]> |
getBindingSetSerializer()
FIXME Custom serialization for binding sets, especially since there tends to be a lot of redundancy in the data arising from how bindings are propagated during JOINs. |
|
int |
getChunkCapacity()
The #of elements in a chunk for query or mutation. |
|
int |
getChunkOfChunksCapacity()
The #of chunks that can be held by an IBuffer that is the target
or one or more UnsynchronizedArrayBuffers. |
|
int |
getFullyBufferedReadThreshold()
The #of elements that will be materialized in a fully buffered read by an IAccessPath. |
|
IRelation |
getHeadRelationView(IPredicate pred)
The head relation is what we write on for mutation operations and is also responsible for minting new elements from computed ISolutions. |
|
IIndexManager |
getIndexManager()
Used to locate indices, relations and relation containers. |
|
IJoinNexusFactory |
getJoinNexusFactory()
The factory object is used to materialize appropriate IJoinNexus
instances when the rule execution crosses an RMI boundary. |
|
int |
getMaxParallelSubqueries()
The maximum #of subqueries for the first join dimension that will be issued in parallel. |
|
IEvaluationPlanFactory |
getPlanFactory()
Return the factory for IEvaluationPlans. |
|
String |
getProperty(String name,
String defaultValue)
Resolve the property value using the IIndexManager, the namespace
of the resource, and the Properties instance to be tested as
hidden parameters. |
|
|
getProperty(String name,
String defaultValue,
IValidator<T> validator)
Resolves, parses, and validates the property value. |
|
IRangeCountFactory |
getRangeCountFactory()
The factory object for range counts used by IEvaluationPlans. |
|
long |
getReadTimestamp()
Equivalent to IJoinNexusFactory.getReadTimestamp(). |
|
IResourceLocator |
getRelationLocator()
|
|
IRuleStatisticsFactory |
getRuleStatisticsFactory()
The factory for rule statistics objects. |
|
IRuleTaskFactory |
getRuleTaskFactory(boolean parallel,
IRule rule)
Return the effective IRuleTaskFactory for the rule. |
|
IElementFilter<ISolution> |
getSolutionFilter()
Return the IElementFilter that will be used to reject solutions
based on the bindings for the head of the rule -or- null
if no filter will be imposed. |
|
IStreamSerializer<ISolution[]> |
getSolutionSerializer()
FIXME Custom serialization for solution sets, especially since there tends to be a lot of redundancy in the data arising from how bindings are propagated during JOINs. |
|
IAccessPath |
getTailAccessPath(IRelation relation,
IPredicate predicate)
Obtain an access path reading from relation for the specified predicate (from the tail of some rule). |
|
IRelation |
getTailRelationView(IPredicate pred)
Locate and return the view of the relation(s) identified by the IPredicate. |
|
long |
getWriteTimestamp()
Equivalent to IJoinNexusFactory.getWriteTimestamp(). |
|
protected boolean |
isEmptyProgram(IStep step)
Return true iff the step is an empty IProgram. |
|
Iterator<PartitionLocator> |
locatorScan(AbstractScaleOutFederation<?> fed,
IPredicate<?> predicate)
Return an iterator visiting the PartitionLocator for the index
partitions from which an IAccessPath must read in order to
materialize all elements which would be visited for that predicate. |
|
IBindingSet |
newBindingSet(IRule rule)
Factory for IBindingSet implementations. |
|
IBuffer<ISolution[]> |
newDeleteBuffer(IMutableRelation relation)
Return a thread-safe buffer onto which chunks of computed ISolutions will be written. |
|
IBuffer<ISolution[]> |
newInsertBuffer(IMutableRelation relation)
Return a thread-safe buffer onto which chunks of computed ISolutions will be written. |
|
IBlockingBuffer<ISolution[]> |
newQueryBuffer()
Note: ISolution (not relation elements) will be written on the
buffer concurrently by different rules so there is no natural order for
the elements in the buffer. |
|
ISolution |
newSolution(IRule rule,
IBindingSet bindingSet)
Create a new ISolution. |
|
protected abstract ISortKeyBuilder<?> |
newSortKeyBuilder(IPredicate<?> head)
Return the ISortKeyBuilder used to impose DISTINCT on the
solutions generated by a query. |
|
IBuffer<ISolution> |
newUnsynchronizedBuffer(IBuffer<ISolution[]> targetBuffer,
int chunkCapacity)
Return a buffer suitable for a single-threaded writer that flushes onto the specified targetBuffer. |
|
protected Object |
runDistributedProgram(IBigdataFederation<?> fed,
ActionEnum action,
IStep step)
Runs a distributed IProgram (key-range partitioned indices, RMI,
and multi-machine). |
|
protected Object |
runLocalProgram(ActionEnum action,
IStep step)
This variant handles both local indices on a TemporaryStore or
Journal WITHOUT concurrency controls (fast). |
|
long |
runMutation(IStep step)
Run as mutation operation (it will write any solutions onto the relations named in the head of the various IRules). |
|
protected Object |
runProgram(ActionEnum action,
IStep step)
Core impl. |
|
IChunkedOrderedIterator<ISolution> |
runQuery(IStep step)
Run as a query. |
|
int |
solutionFlags()
The flags that effect the behavior of IJoinNexus.newSolution(IRule, IBindingSet). |
|
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Methods inherited from interface com.bigdata.relation.rule.eval.IJoinNexus |
|---|
fakeBinding, newBindingSetSortKeyBuilder |
| Field Detail |
|---|
protected final IIndexManager indexManager
protected final IResourceLocator<?> resourceLocator
protected final long readTimestamp
protected final int chunkCapacity
protected final int chunkOfChunksCapacity
protected final int solutionFlags
protected final IElementFilter<?> filter
protected final IEvaluationPlanFactory planFactory
| Constructor Detail |
|---|
protected AbstractJoinNexus(IJoinNexusFactory joinNexusFactory,
IIndexManager indexManager)
joinNexusFactory - The object used to create this instance and which can be used
to create other instances as necessary for distributed rule
execution.indexManager - The object used to resolve indices, relations, etc.| Method Detail |
|---|
public final int getChunkOfChunksCapacity()
IJoinNexusIBuffer that is the target
or one or more UnsynchronizedArrayBuffers. This is generally a
small value on the order of the #of parallel producers that might be
writing on the IBuffer since the capacity of the
UnsynchronizedArrayBuffers is already quite large (10k or better
elements, defining a single "chunk" from a single producer).
getChunkOfChunksCapacity in interface IJoinNexusIJoinNexus.getMaxParallelSubqueries(),
IJoinNexus.getChunkCapacity()public final int getChunkCapacity()
IJoinNexus10,000 or better.
getChunkCapacity in interface IJoinNexuspublic final int getFullyBufferedReadThreshold()
IJoinNexusIAccessPath. When this threshold is exceeded the
IAccessPath will use an IAsynchronousIterator instead.
This value should on the order of IJoinNexus.getChunkCapacity().
getFullyBufferedReadThreshold in interface IJoinNexusIAccessPath.iterator(int, int),
AbstractResource.Options#FULLY_BUFFERED_READ_THRESHOLD
public final String getProperty(String name,
String defaultValue)
IJoinNexusIIndexManager, the namespace
of the resource, and the Properties instance to be tested as
hidden parameters.
getProperty in interface IJoinNexusname - The property name.defaultValue - The default.
Configuration
public final <T> T getProperty(String name,
String defaultValue,
IValidator<T> validator)
IJoinNexus
getProperty in interface IJoinNexusname - The property name.defaultValue - The default value.
public IElementFilter<ISolution> getSolutionFilter()
IJoinNexusIElementFilter that will be used to reject solutions
based on the bindings for the head of the rule -or- null
if no filter will be imposed. This may be used for query or mutation.
getSolutionFilter in interface IJoinNexuspublic IRuleStatisticsFactory getRuleStatisticsFactory()
IJoinNexus
getRuleStatisticsFactory in interface IJoinNexuspublic IJoinNexusFactory getJoinNexusFactory()
IJoinNexusIJoinNexus
instances when the rule execution crosses an RMI boundary.
getJoinNexusFactory in interface IJoinNexuspublic IRangeCountFactory getRangeCountFactory()
IJoinNexusIEvaluationPlans.
getRangeCountFactory in interface IJoinNexuspublic final boolean forceSerialExecution()
IJoinNexustrue, rule level parallelism is disabled and the
ISolution buffers are flushed after after every IStep.
This can be enabled if you are exploring apparent concurrency problems
with the rules. It should normally be false for better
performance.
forceSerialExecution in interface IJoinNexuspublic final int getMaxParallelSubqueries()
IJoinNexusExecutorService entirely and ONE (1) to submit a single task at a
time to the ExecutorService.
getMaxParallelSubqueries in interface IJoinNexuspublic final ActionEnum getAction()
IJoinNexus
getAction in interface IJoinNexuspublic final long getWriteTimestamp()
IJoinNexusIJoinNexusFactory.getWriteTimestamp().
getWriteTimestamp in interface IJoinNexuspublic final long getReadTimestamp()
IJoinNexusIJoinNexusFactory.getReadTimestamp().
getReadTimestamp in interface IJoinNexuspublic IRelation getHeadRelationView(IPredicate pred)
ISolutions.
This method depends solely on the name of the head relation and the
timestamp of interest for the view.
getHeadRelationView in interface IJoinNexuspred - The IPredicate, which MUST be the head of some
IRule.
IRelation, which will never be a fused view and
which will accept writes iff the rules are being executed as a
mutation operation.public IRelation getTailRelationView(IPredicate pred)
IJoinNexusIPredicate.
Note: This method is responsible for returning a fused view when more
than one relation name was specified for the IPredicate. It
SHOULD be used whenever the IRelation is selected based on a
predicate in the tail of an IRule and could therefore be a fused
view of more than one relation instance. (The head of the IRule
must be a simple IRelation and not a view.)
Note: The implementation should choose the read timestamp for each
relation in the view using #getReadTimestamp(String).
getTailRelationView in interface IJoinNexuspred - The IPredicate, which MUST be a tail from some IRule.
IRelation, which might be a
RelationFusedView.
public IAccessPath getTailAccessPath(IRelation relation,
IPredicate predicate)
IJoinNexus
Note that passing in the IRelation is important since it
otherwise must be discovered using the IResourceLocator. By
requiring the caller to resolve it before hand and pass it into this
method the contention and demand on the IResourceLocator cache is
reduced.
getTailAccessPath in interface IJoinNexusrelation - The relation.predicate - The predicate. When IPredicate.getPartitionId() is
set, the returned IAccessPath MUST read on the
identified local index partition (directly, not via RMI).
public Iterator<PartitionLocator> locatorScan(AbstractScaleOutFederation<?> fed,
IPredicate<?> predicate)
IJoinNexusPartitionLocator for the index
partitions from which an IAccessPath must read in order to
materialize all elements which would be visited for that predicate.
Note: You can use an IDataServiceCallable to obtain the
reference of the IDataService and pass that into your
AbstractTask in order to have the federation reference available
when running under the ConcurrencyManager.
locatorScan in interface IJoinNexusfed - The federation, which is required in order to access the
IMetadataIndex for a scale-out index.predicate - The predicate, with whatever bindings already applied.
public final IIndexManager getIndexManager()
IJoinNexus
getIndexManager in interface IJoinNexus
public final boolean bind(IRule rule,
int index,
Object e,
IBindingSet bindings)
IJoinNexusNote: The bindings are propagated before the constraints are verified so this method will have a side-effect on the bindings even if the constraints were not satisfied. Therefore you should clone the bindings before calling this method.
bind in interface IJoinNexusrule - The rule.index - The index of the IPredicate in the body of the
Rule.e - An element materialized by the IAccessPath for that
IPredicate.
true unless the new bindings would violate any of
the IConstraints declared for the Rule).
public final boolean bind(IPredicate<?> pred,
IConstraint[] constraints,
Object e,
IBindingSet bindings)
IJoinNexusNote: The bindings are propagated before the constraints are verified so this method will have a side-effect on the bindings even if the constraints were not satisfied. Therefore you should clone the bindings before calling this method.
bind in interface IJoinNexuspred - The IPredicate from which the element was read.constraints - An array of constraints which must be satisfied (optional).e - An element materialized by the IAccessPath for that
IPredicate.
true unless the new bindings would violate any of
the optional IConstraint.
public final ISolution newSolution(IRule rule,
IBindingSet bindingSet)
IJoinNexusISolution. The behavior of this method generally
depends on bit flags specified when the IJoinNexus was created.
Note: For many purposes, it is only the computed IJoinNexus.ELEMENTs that
are of interest. For high-level query, you will generally specify only
the IJoinNexus.BINDINGS. The IJoinNexus.BINDINGS are also useful for some
truth maintenance applications. The IJoinNexus.RULE is generally only of
interest for inspecting the behavior of some rule set.
newSolution in interface IJoinNexusrule - The rule.bindingSet - The bindings (the implementation MUST clone the bindings if
they will be saved with the ISolution).
ISolution.IJoinNexus.ELEMENT,
IJoinNexus.BINDINGS,
IJoinNexus.RULE,
IJoinNexus.solutionFlags(),
Solutionpublic final int solutionFlags()
IJoinNexusIJoinNexus.newSolution(IRule, IBindingSet).
solutionFlags in interface IJoinNexuspublic IStreamSerializer<ISolution[]> getSolutionSerializer()
getSolutionSerializer in interface IJoinNexus(needs to be written).ISolutions much like we already do for
DISTINCT or intend to do for SORT and use the equivalent of leading
key compression to reduce IO costs (or when they are SORTed we
could leverage that to produce a more compact serialization).public IStreamSerializer<IBindingSet[]> getBindingSetSerializer()
getBindingSetSerializer in interface IJoinNexusSPOBindingSetSerializer, which has not been finished.ISolutions much like we already do for
DISTINCT or intend to do for SORT and use the equivalent of leading
key compression to reduce IO costs (or when they are SORTed we
could leverage that to produce a more compact serialization).public final IBindingSet newBindingSet(IRule rule)
IJoinNexusIBindingSet implementations.
Note: The factory MUST apply any bound
constants for the IRule before
returning the IBindingSet.
newBindingSet in interface IJoinNexusrule - The rule whose bindings will be stored in the binding set.
public final IRuleTaskFactory getRuleTaskFactory(boolean parallel,
IRule rule)
IJoinNexusIRuleTaskFactory for the rule. When the rule
is a step of a sequential program writing on one or more
IMutableRelations, then the returned IStepTask must
automatically flush the buffer after the rule executes in order to ensure
that the state of the IMutableRelation(s) are updated before the
next IRule is executed.
getRuleTaskFactory in interface IJoinNexusparallel - true unless the rule is a step is a sequential
IProgram. Note that a sequential step MUST flush its
buffer since steps are run in sequence precisely because they
have a dependency!rule - A rule that is a step in some program. If the program is just
a rule then the value of parallel does not matter. The
buffer will is cleared when it flushed so a re-flushed is
always a NOP.
IStepTask to execute for that rule.RunRuleAndFlushBufferTaskFactory,
RunRuleAndFlushBufferTaskpublic final IEvaluationPlanFactory getPlanFactory()
IJoinNexusIEvaluationPlans.
getPlanFactory in interface IJoinNexuspublic final IResourceLocator getRelationLocator()
public final IBuffer<ISolution> newUnsynchronizedBuffer(IBuffer<ISolution[]> targetBuffer,
int chunkCapacity)
IJoinNexus
The returned buffer MUST apply the optional filter value returned by
IJoinNexus.getSolutionFilter() in order to keep individual
ISolutions out of the buffer. Filtering is done at this level
since the targetBuffer contains chunks of solutions.
newUnsynchronizedBuffer in interface IJoinNexustargetBuffer - A thread-safe buffer for chunks of ISolutions that
was allocated with IJoinNexus.newQueryBuffer(),
IJoinNexus.newInsertBuffer(IMutableRelation), or
IJoinNexus.newDeleteBuffer(IMutableRelation).chunkCapacity - The capacity of the new buffer. This should be maximum chunk
size that will be produced or IJoinNexus.getChunkCapacity() if
you do not have better information.
public final IBlockingBuffer<ISolution[]> newQueryBuffer()
ISolution (not relation elements) will be written on the
buffer concurrently by different rules so there is no natural order for
the elements in the buffer.
newQueryBuffer in interface IJoinNexuspublic IBuffer<ISolution[]> newInsertBuffer(IMutableRelation relation)
ISolutions will be written. When the buffer is
flushed the chunked ISolutions will be
inserted into the IMutableRelation.
Note: getSolutionFilter() is applied by
newUnsynchronizedBuffer(IBuffer, int) and NOT by the buffer
returned by this method.
newInsertBuffer in interface IJoinNexusrelation - The relation.
public IBuffer<ISolution[]> newDeleteBuffer(IMutableRelation relation)
ISolutions will be written. When the buffer is
flushed the chunks of ISolutions will be
deleted from the IMutableRelation.
Note: getSolutionFilter() is applied by
newUnsynchronizedBuffer(IBuffer, int) and NOT by the buffer
returned by this method.
newDeleteBuffer in interface IJoinNexusrelation - The relation.
protected abstract ISortKeyBuilder<?> newSortKeyBuilder(IPredicate<?> head)
ISortKeyBuilder used to impose DISTINCT on the
solutions generated by a query.
head - The head of the rule.
ISortKeyBuilder.
public IChunkedOrderedIterator<ISolution> runQuery(IStep step)
throws Exception
IJoinNexus
runQuery in interface IJoinNexusstep - The IRule or IProgram.
IllegalStateException - unless this is an ActionEnum.Query.
IllegalArgumentException - if either argument is null.
Exception
public final long runMutation(IStep step)
throws Exception
IJoinNexusIRules).
runMutation in interface IJoinNexusstep - The IRule or IProgram.
IllegalArgumentException - unless ActionEnum.isMutation() is true.
Exceptionprotected final boolean isEmptyProgram(IStep step)
IProgram.
step - The step.
protected final Object runProgram(ActionEnum action,
IStep step)
throws Exception
DataService (highly efficient) or within the client
using the IClientIndex to submit operations to the appropriate
DataService(s) (not very efficient, even w/o RMI).
IChunkedOrderedIterator (query) or Long
(mutation count).
Exception
protected final Object runLocalProgram(ActionEnum action,
IStep step)
throws Exception
TemporaryStore or
Journal WITHOUT concurrency controls (fast).
Exception
protected final Object runDistributedProgram(IBigdataFederation<?> fed,
ActionEnum action,
IStep step)
throws Exception
IProgram (key-range partitioned indices, RMI,
and multi-machine).
Exception
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||