|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectcom.bigdata.rdf.sparql.ast.StaticAnalysisBase
com.bigdata.rdf.sparql.ast.StaticAnalysis_CanJoin
com.bigdata.rdf.sparql.ast.StaticAnalysis
public class StaticAnalysis
Methods for static analysis of a query. There is one method which looks "up". This corresponds to how we actually evaluation things (left to right in the query plan). There are two methods which look "down". This corresponds to the bottom-up evaluation semantics of SPARQL.
When determining the "known" bound variables on entry to a node we have to look "up" the tree until we reach the outer most group. Note that named subqueries DO NOT receive bindings from the places where they are INCLUDEd into the query.
QueryBaseSubqueryRootNamedSubqueryRootNamedSubqueryIncludeServiceNodeStatementPatternNodeStatementPatternNode.isOptional() is true.
Note: we sometimes attach a simple optional join to the parent group for
efficiency, at which point it becomes an "optional" statement pattern. An
optional statement pattern may also have zero or more FilterNodes
associated with it.
JoinGroupNodeUnionNodeAssignmentNodeNote: BIND() in a PROJECTION is handled differently as it is non-optional (if the value expression results in an error the solution is dropped). Projections are handled when we do the analysis of a QueryBase node since we can see both the WHERE clause and the PROJECTION clauses at the same time.
IF()IF semantics : If evaluating the first argument raises an
error, then an error is raised for the evaluation of the IF expression. (This
greatly simplifies the analysis of the EBV of the IF value expressions, but
there is still uncertainty concerning whether the THEN or the ELSE is
executed for a given solution.) However, IF is not allowed to
conditionally bind a variable in the THEN/ELSE expressions so we do not have
to consider it here.BOUND(var)true or false in for the
BOUND() expression.
We could also attach the StaticAnalysis as an annotation on the
QueryRoot and provide a factory method for accessing it. That way we
would have reuse of the cached static analysis data. Each AST optimizer (or
the ASTOptimizerList) would have to clear the cached
StaticAnalysis when producing a new QueryRoot. Do this when
we add an ASTContainer to provide a better home for the queryStr, the parse
tree, the original AST, and the optimized AST.
| Field Summary |
|---|
| Fields inherited from class com.bigdata.rdf.sparql.ast.StaticAnalysisBase |
|---|
evaluationContext, queryRoot |
| Constructor Summary | |
|---|---|
StaticAnalysis(QueryRoot queryRoot,
IEvaluationContext evaluationContext)
|
|
| Method Summary | |
|---|---|
IQueryNode |
findParent(GraphPatternGroup<?> group)
Return the parent of the GraphPatternGroup. |
JoinGroupNode |
findParentJoinGroup(GraphPatternGroup<?> group)
Find and return the parent JoinGroupNode which is the lowest such
JoinGroupNode dominating the given GraphPatternGroup. |
static INeedsMaterialization.Requirement |
gatherVarsToMaterialize(BOp c,
Set<IVariable<IV>> terms)
Static helper used to determine materialization requirements. |
Set<IVariable<?>> |
getAfterVars(IGroupMemberNode node,
Set<IVariable<?>> vars)
Return any variables which are used after the given node in the current ordering of its parent JoinGroupNode but DOES NOT consider the
parent or the PROJECTION for the query in which this group appears. |
Set<IVariable<?>> |
getDefinitelyIncomingBindings(IGroupMemberNode node,
Set<IVariable<?>> vars)
Return the set of variables which MUST be bound coming into this group during top-down, left-to-right evaluation. |
Set<IVariable<?>> |
getDefinitelyProducedBindings(IBindingProducerNode node,
Set<IVariable<?>> vars,
boolean recursive)
Return the set of variables which MUST be bound for solutions after the evaluation of this group. |
Set<IVariable<?>> |
getDefinitelyProducedBindings(QueryBase queryBase)
Report "MUST" bound bindings projected by the query. |
Set<IVariable<?>> |
getDefinitelyProducedBindings(ServiceNode node)
Report "MUST" bound bindings projected by the SERVICE. |
Set<IVariable<?>> |
getDefinitelyProducedBindingsAndFilterVariables(IGroupNode<? extends IGroupMemberNode> group,
Set<IVariable<?>> vars)
Collect all variables appearing in the group. |
List<FilterNode> |
getJoinFilters(JoinGroupNode group)
Return only the filter child nodes in this group whose variables were not fully bound on entry into the join group but which will be fully bound no later than once we have run the required joins in this group. |
Set<IVariable<?>> |
getJoinVars(NamedSubqueryInclude nsi,
String solutionSet,
Set<IVariable<?>> vars)
Return the join variables for an INCLUDE of a pre-existing named solution set. |
Set<IVariable<?>> |
getJoinVars(NamedSubqueryRoot aNamedSubquery,
NamedSubqueryInclude anInclude,
Set<IVariable<?>> vars)
Identify the join variables for the specified INCLUDE for the position within the query in which it appears. |
Set<IVariable<?>> |
getJoinVars(ServiceNode serviceNode,
Set<IVariable<?>> vars)
Return the join variables for a SERVICE. |
Set<IVariable<?>> |
getJoinVars(SubqueryRoot subquery,
Set<IVariable<?>> vars)
Identify the join variables for the specified subquery for the position within the query in which it appears. |
Set<IVariable<?>> |
getMaybeIncomingBindings(IGroupMemberNode node,
Set<IVariable<?>> vars)
Return the set of variables which MIGHT be bound coming into this group during top-down, left-to-right evaluation. |
Set<IVariable<?>> |
getMaybeProducedBindings(IBindingProducerNode node,
Set<IVariable<?>> vars,
boolean recursive)
Return the set of variables which MUST or MIGHT be bound after the evaluation of this join group. |
Set<IVariable<?>> |
getMaybeProducedBindings(QueryBase node)
Report the "MUST" and "MAYBE" bound bindings projected by the query. |
Set<IVariable<?>> |
getMaybeProducedBindings(ServiceNode node)
Report the "MUST" and "MAYBE" bound variables projected by the service. |
List<FilterNode> |
getPostFilters(JoinGroupNode group)
Return only the filter child nodes in this group that will not be fully bound even after running the required joins in this group. |
List<FilterNode> |
getPreFilters(JoinGroupNode group)
Return only the filter child nodes in this group that will be fully bound before running any of the joins in this group. |
Set<IVariable<?>> |
getProjectedVars(IGroupMemberNode proxy,
GraphPatternGroup<?> groupToLift,
QueryBase query,
Set<IVariable<?>> exogenousVars,
Set<IVariable<?>> projectedVars)
Return the set of variables which must be projected if the group is to be converted into a sub-query. |
List<FilterNode> |
getPruneFilters(JoinGroupNode group)
Deprecated. This is now handled by ASTBottomUpOptimizer. I think
that we will not need this method (it is only invoked from
the test suite at this point). |
static boolean |
isAggregate(ProjectionNode projection,
GroupByNode groupBy,
HavingNode having)
Return true if any of the ProjectionNode,
GroupByNode, or HavingNode indicate that this is an
aggregation query. |
static boolean |
isAggregate(QueryBase query)
Return true if any of the ProjectionNode,
GroupByNode, or HavingNode indicate that this is an
aggregation query. |
static boolean |
requiresMaterialization(IConstraint c)
Use the INeedsMaterialization interface to find and collect
variables that need to be materialized for this constraint. |
| Methods inherited from class com.bigdata.rdf.sparql.ast.StaticAnalysis_CanJoin |
|---|
canJoin, canJoinUsingConstraints, getJoinGraphConstraints |
| Methods inherited from class com.bigdata.rdf.sparql.ast.StaticAnalysisBase |
|---|
addAll, getNamedSubqueryRoot, getQueryRoot, getRequiredNamedSubqueryRoot, getSolutionSetStats, getSpannedVariables, getSpannedVariables, isFullyBound |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public StaticAnalysis(QueryRoot queryRoot,
IEvaluationContext evaluationContext)
queryRoot - The root of the query. We need to have this on hand in order
to resolve NamedSubqueryIncludes during static
analysis.evaluationContext - The evaluation context provides access to the
ISolutionSetStats and the ISparqlCache for
named solution sets.(StaticAnalysis#getDefinitelyBound() ignores exogenous variables.)| Method Detail |
|---|
public JoinGroupNode findParentJoinGroup(GraphPatternGroup<?> group)
JoinGroupNode which is the lowest such
JoinGroupNode dominating the given GraphPatternGroup.
This will search the tree to locate the parent when the
GraphPatternGroup appears as the annotation of a
QueryBase, ServiceNode, or a FilterNode having a
ExistsNode or NotExistsNode.
group - The given group.
JoinGroupNode above that group.public IQueryNode findParent(GraphPatternGroup<?> group)
GraphPatternGroup. When the group has an
explicit parent reference, that reference is returned immediately.
Otherwise the QueryRoot is searched for a node having the given
group as an annotation. This makes it possible to locate a
QueryBase, ServiceNode, ExistsNode, or
NotExistsNode given its GraphPatternGroup.
Note: The parent of a SubqueryRoot is obtained by
SubqueryBase.getParent() and is simply the JoinGroupNode
in which the SubqueryRoot appears.
group - The group.
GraphPatternGroup, QueryBase, ServiceNode
, or a FilterNode. This will be null iff the
group does not appear anywhere in the QueryRoot.
TODO The parent of a NamedSubqueryRoot is less well
defined. A NamedSubqueryRoot may be included in multiple
positions within the AST. Each of those could be considered a
parent of the NamedSubqueryRoot in the sense that it
provides a context within which the result of the query may be
included. However, for the purposes of bottom up analysis, there
is no parent of a NamedSubqueryRoot. It runs as if it
were a top-level query (except that it might not have visibility
into exogenous variables?).
public Set<IVariable<?>> getDefinitelyIncomingBindings(IGroupMemberNode node,
Set<IVariable<?>> vars)
This method DOES NOT pay attention to bottom up variable scoping rules. Queries which are badly designed MUST be rewritten (by lifting out named subqueries) such that they become well designed and adhere to bottom-up evaluation semantics.
vars - Where to store the "MUST" bound variables.
getMaybeIncomingBindings(IGroupMemberNode, Set) need to
consider the exogenous variables. Perhaps modify the
StaticAnalysis constructor to pass in the exogenous
IBindingSet[]?
FIXME For some purposes we need to consider the top-down,
left-to-right evaluation order. However, for others, such as when
considering whether a variable appearing in a filter will be in
scope, we need to consider whether there exists some evaluation
order for which the variable would be in scope.(StaticAnalysis#getDefinitelyBound() ignores exogenous variables.)
public Set<IVariable<?>> getMaybeIncomingBindings(IGroupMemberNode node,
Set<IVariable<?>> vars)
This method DOES NOT pay attention to bottom up variable scoping rules. Queries which are badly designed MUST be rewritten (by lifting out named subqueries) such that they become well designed and adhere to bottom-up evaluation semantics.
vars - Where to store the "maybe" bound variables. This includes ANY
variable which MIGHT or MUST be bound.
getDefinitelyIncomingBindings(IGroupMemberNode, Set)
need to consider the exogenous variables. Perhaps modify the
StaticAnalysis constructor to pass in the exogenous
IBindingSet[]?
FIXME This is unable to look upwards when the group is the graph
pattern of a subquery, a service, or a (NOT) EXISTS filter.https://sourceforge.net/apps/trac/bigdata/ticket/412
public Set<IVariable<?>> getDefinitelyProducedBindings(IBindingProducerNode node,
Set<IVariable<?>> vars,
boolean recursive)
The returned collection reflects "bottom-up" evaluation semantics. This method does NOT consider variables which are already bound on entry to the group.
Note: When invoked for an OPTIONAL or MINUS join group, the variables which would become bound during the evaluation of the join group are reported. Caller's who wish to NOT have variables reported for OPTIONAL or MINUS groups MUST NOT invoke this method for those groups.
Note: The recursive analysis does not throw out variables when part of the tree will provably fail to bind anything. It is the role of query optimizers to identify those situations and prune the AST appropriately.
node - The node to be analyzed.vars - Where to store the "MUST" bound variables.recursive - When true, the child groups will be recursively
analyzed. When false, only this group will
be analyzed.
public Set<IVariable<?>> getDefinitelyProducedBindingsAndFilterVariables(IGroupNode<? extends IGroupMemberNode> group,
Set<IVariable<?>> vars)
This has the same behavior as a non-recursive call obtain the definitely bound variables PLUS the variables used by the filters in the group.
vars - The variables are added to this set.group - The group whose variables will be reported.includeFilters - When true, variables appearing in FILTERs are
also reported.
public Set<IVariable<?>> getMaybeProducedBindings(IBindingProducerNode node,
Set<IVariable<?>> vars,
boolean recursive)
The returned collection reflects "bottom-up" evaluation semantics. This method does NOT consider variables which are already bound on entry to the group.
vars - Where to store the "MUST" and "MIGHT" be bound variables.recursive - When true, the child groups will be recursively
analyzed. When false, only this group will
be analyzed.
public Set<IVariable<?>> getDefinitelyProducedBindings(QueryBase queryBase)
ProjectionNode for the query.
Note that the projection can rename variables. It can also bind a
constant on a variable. Variables which are not projected by the query
will NOT be reported.
FIXME For a top-level query, any exogenously bound variables are also
definitely bound (in a subquery they are definitely bound if they are
projected into the subquery).
(StaticAnalysis#getDefinitelyBound() ignores exogenous variables.),
(StaticAnalysis
does not follow renames of projected variables)public Set<IVariable<?>> getMaybeProducedBindings(QueryBase node)
public Set<IVariable<?>> getDefinitelyProducedBindings(ServiceNode node)
ServiceNode.getGraphPattern() .
public Set<IVariable<?>> getMaybeProducedBindings(ServiceNode node)
ServiceNode.getGraphPattern(). A SERVICE does NOT have an
explicit PROJECTION so it can not rename the projected bindings.
public List<FilterNode> getPreFilters(JoinGroupNode group)
Note: Anything returned by this method should be lifted into the parent group since it can be run before this group is evaluated. By lifting the pre-filters into the parent group we can avoid issuing as many as-bound subqueries for this group since those which fail the filter will not be issued.
group - The JoinGroupNode.
ASTLiftPreFiltersOptimizerpublic List<FilterNode> getJoinFilters(JoinGroupNode group)
group - The JoinGroupNode.
public List<FilterNode> getPostFilters(JoinGroupNode group)
Note: It is possible that some of these filters will be fully bound due to nested optionals and unions.
Note: This will report any filters which are not pre-filters and are not-join filters, including filters which are prune-filters. An AST optimizer is responsible for identifying and removing filters which should be pruned. Until they have been pruned, they will continue to be reported by this method.
group - The JoinGroupNode.
public List<FilterNode> getPruneFilters(JoinGroupNode group)
ASTBottomUpOptimizer. I think
that we will not need this method (it is only invoked from
the test suite at this point).
Note: Filters containing a FunctionNode for
FunctionRegistry.BOUND MUST NOT be pruned and are NOT reported by
this method.
group - The JoinGroupNode.
?x simply does not appear in the group or
any child of that group, then BOUND(?x) can be replaced by
false and NOT BOUND(?x) by true.
However, in order to do this we must also look at any exogenous solution(s) (those supplied with the query when it is being evaluated). If the variable is bound in some exogenous solutions then it could be bound when the FILTER is run and the filter can not be pruned.
public static boolean requiresMaterialization(IConstraint c)
INeedsMaterialization interface to find and collect
variables that need to be materialized for this constraint.
public static INeedsMaterialization.Requirement gatherVarsToMaterialize(BOp c,
Set<IVariable<IV>> terms)
FullyInlineTypedLiteralIV
or a specific numeric data type, then some operators may be able to
operate directly on that IV. This is especially interesting for
aggregates.
public Set<IVariable<?>> getJoinVars(NamedSubqueryRoot aNamedSubquery,
NamedSubqueryInclude anInclude,
Set<IVariable<?>> vars)
aNamedSubquery - The named subquery.anInclude - An include for that subquery.
public Set<IVariable<?>> getJoinVars(SubqueryRoot subquery,
Set<IVariable<?>> vars)
aSubquery - The subquery.vars -
public Set<IVariable<?>> getJoinVars(ServiceNode serviceNode,
Set<IVariable<?>> vars)
serviceNode - vars -
public Set<IVariable<?>> getJoinVars(NamedSubqueryInclude nsi,
String solutionSet,
Set<IVariable<?>> vars)
nsi - The NamedSubqueryIncludesolutionSet - The name of a pre-existing solution set.vars - The caller's collection.
public Set<IVariable<?>> getAfterVars(IGroupMemberNode node,
Set<IVariable<?>> vars)
JoinGroupNode but DOES NOT consider the
parent or the PROJECTION for the query in which this group appears.
node - A node which is a direct child of some JoinGroupNode.vars - Where to store the variables.
IllegalArgumentException - if the node is not the direct child of some
JoinGroupNode.
public Set<IVariable<?>> getProjectedVars(IGroupMemberNode proxy,
GraphPatternGroup<?> groupToLift,
QueryBase query,
Set<IVariable<?>> exogenousVars,
Set<IVariable<?>> projectedVars)
When considering the projection of the (sub-)query in which the group appears, the SELECT EXPRESSIONS are consulted to identify variables which we need to project out of the group.
proxy - The join group which will be replaced by a sub-query. This is
used to decide which variables are known bound (and hence
should be projected into the WHERE clause if they are used
within that WHERE clause). It is also used to decide which
variables which become bound in the WHERE clause will be used
outside of its scope and hence must be projected out of the
WHERE clause. (The parent of this proxy MUST be a
JoinGroupNode, not a UnionNode and not
null. This condition is readily satisified if the
rewrite is considering the children of some join group node as
the parent of the proxy will be that join group node.)groupToLift - The group which is being lifted out and whose projection will
be computed.query - The query (or sub-query) in which that proxy node exists. This
is used to identify anything which is PROJECTed out of the
query.exogenousVars - Any variables which are bound outside of the query AND known
to be in scope (exogenous variables in a sub-select are only
in scope if they are projected into the sub-select).projectedVars - The variables which must be projected will be added to this
collection.
public static boolean isAggregate(QueryBase query)
true if any of the ProjectionNode,
GroupByNode, or HavingNode indicate that this is an
aggregation query.
query - The query.
trueif it is an aggregation query.
public static boolean isAggregate(ProjectionNode projection,
GroupByNode groupBy,
HavingNode having)
true if any of the ProjectionNode,
GroupByNode, or HavingNode indicate that this is an
aggregation query. All arguments are optional.
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||