-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54680/
-----------------------------------------------------------
(Updated Dec. 13, 2016, 12:09 a.m.)
Review request for atlas and David Kantor.
Bugs: ATLAS-1369
https://issues.apache.org/jira/browse/ATLAS-1369
Repository: atlas
Description (updated)
-------
The way the queries are generated now, in many cases the indices are not used
when executing queries. The main problem is that gremlin cannot currently use
"has" expressions within an "and" or an "or". This JIRA introduces a
GremlinQueryOptimizer that attempts to extract "has" expressions out of "and"
expressions and turn "or" expressions into unions.
The code being submitted has only been tested with TP 3 at this point. There
are some issues that need to be fixed before this can be used with TP 2:
* FunctionGenerator - generates functions that take GraphTraversal as an
argument. That should be changed to GreminPipeline when using TP 2. The return
type should probably come from a new method in GremlinExpressionFactory
* loop support - for TP2, we will need to add logic to make sure that the
loop call is not separated from the alias it references
* alias handling - the alias safe/restore logic in
ExpandOrsOptimization.getBaseResultExpression() may be need to adjusted for TP2
* possibly other issues, TBD
Major changes:
* Updated version of Guava used by Atlas, so that could use
Lists.cartesianProduct
* New package - org.apache.atlas.gremlin.optimizer - contains optimization
framework
* Refactoring of Groovy Expression classes
* new - AbstractFunctionExpression - super class of anything with a
caller
* added 4 new methods to GroovyExpression
* now, each GroovyExpression is a TraversalStepType associated with it
These are used by the optimization framework
* Added feature toggle : atlas.query.gremlinOptimizerEnabled. The feature
turns optimization on/off when using TP 3. It is always off in TP2.
* Most of the new logic is in GremlinQueryOptimizer,
ExpandAndsOptimization, and ExpandOrsOptimization
Diffs
-----
common/src/main/java/org/apache/atlas/groovy/AbstractFunctionExpression.java
PRE-CREATION
common/src/main/java/org/apache/atlas/groovy/AbstractGroovyExpression.java
49eaae87e593d4e350ecb9a51a83e967d95b514b
common/src/main/java/org/apache/atlas/groovy/ArithmeticExpression.java
0aec5d03ea6e2a108129b20003d390e826f76bb5
common/src/main/java/org/apache/atlas/groovy/BinaryExpression.java
ccc9204cdc9613efeb2b6548d8d426bee7cfec06
common/src/main/java/org/apache/atlas/groovy/CastExpression.java
963724c7cefcd871918d340c25c197143d155d56
common/src/main/java/org/apache/atlas/groovy/ClosureExpression.java
2d702093937b69a7d82e8d5f58fb528144377de5
common/src/main/java/org/apache/atlas/groovy/CodeBlockExpression.java
9a726f2f82764b0a93c523467b3fa316e2dbf4e0
common/src/main/java/org/apache/atlas/groovy/ComparisonExpression.java
345f838635f62457b0fd42efed83fd5b96b5c16c
common/src/main/java/org/apache/atlas/groovy/ComparisonOperatorExpression.java
63638b73b048ed301f5a172bf7cec0e2f618d019
common/src/main/java/org/apache/atlas/groovy/FieldExpression.java
f6d06bd194e91d332fd7418da2a9ff24342b4647
common/src/main/java/org/apache/atlas/groovy/FunctionCallExpression.java
b60edef998b4c8361b4f194539577014428483db
common/src/main/java/org/apache/atlas/groovy/GroovyExpression.java
493bd3df1274fe67f47abe6606a2f447af8fa470
common/src/main/java/org/apache/atlas/groovy/IdentifierExpression.java
6abdbf02ed54389f6bd7d9516fa14dad5d791b8d
common/src/main/java/org/apache/atlas/groovy/ListExpression.java
f7acaace9c4e5c9f87c596f7c11c4c41c1b13db5
common/src/main/java/org/apache/atlas/groovy/LiteralExpression.java
008c885686985ff38e0d1ad28a5e0f736e700a39
common/src/main/java/org/apache/atlas/groovy/LogicalExpression.java
ee5829b39e8c4bd6c80942328100259d2762c506
common/src/main/java/org/apache/atlas/groovy/RangeExpression.java
7322f697474f05630398c6379ec0aa4c3ba178d6
common/src/main/java/org/apache/atlas/groovy/StatementListExpression.java
PRE-CREATION
common/src/main/java/org/apache/atlas/groovy/TernaryOperatorExpression.java
75a2f8682f62d1d7bf173a6d704c7accb0c27808
common/src/main/java/org/apache/atlas/groovy/TraversalStepType.java
PRE-CREATION
common/src/main/java/org/apache/atlas/groovy/TypeCoersionExpression.java
4a6105234fe9ca33830516558a40f459ad7082f9
common/src/main/java/org/apache/atlas/groovy/VariableAssignmentExpression.java
7e018f1dcffed654b958c59d70378d6178db4367
distro/src/conf/atlas-application.properties
eea46a6abf3e8f0e1c81cd62e765d8c72e8fa623
pom.xml bdfd785194e35239db5feab93a5fa70ef20f84fb
repository/src/main/java/org/apache/atlas/gremlin/Gremlin2ExpressionFactory.java
6b6896150efe185cf4f2cb4ae005bd5aaa0b359a
repository/src/main/java/org/apache/atlas/gremlin/Gremlin3ExpressionFactory.java
ca1ad5df9fc01b5c50c2dcc31098beca1cdfb76d
repository/src/main/java/org/apache/atlas/gremlin/GremlinExpressionFactory.java
3677544d73183116cd0bea4a1b04be11f19b5c1a
repository/src/main/java/org/apache/atlas/gremlin/optimizer/AliasFinder.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/CallHierarchyVisitor.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/ExpandAndsOptimization.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/ExpandOrsOptimization.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/ExpressionFinder.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/FunctionGenerator.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/GremlinOptimization.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/GremlinQueryOptimizer.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/HasForbiddenType.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/IsOr.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/IsOrParent.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/OptimizationContext.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/SplitPointFinder.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/gremlin/optimizer/UpdatedExpressions.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
1a3faf778f01582e476792780a9dbaa8a4010d29
repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
84608d9db54058ae59595acc42710b23c0c9b918
repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java
a270b9789d67cb124e7929d495da04878f2ea35d
repository/src/main/scala/org/apache/atlas/query/GremlinQuery.scala
a61ff982915684b72fe7067eea2fc2ac75d31baf
repository/src/test/java/org/apache/atlas/repository/graph/GremlinQueryOptimizerTest.java
PRE-CREATION
Diff: https://reviews.apache.org/r/54680/diff/
Testing
-------
Ran repository and webapp tests, no regressions found.
Thanks,
Jeff Hagelberg
