----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55813/#review162725 -----------------------------------------------------------
Fix it, then Ship it! repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java (line 127) <https://reviews.apache.org/r/55813/#comment234051> Please change log level to 'debug'. repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java (line 153) <https://reviews.apache.org/r/55813/#comment234052> Please change log level to 'debug'. - Madhan Neethiraj On Jan. 23, 2017, 10:09 p.m., Sarath Subramanian wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55813/ > ----------------------------------------------------------- > > (Updated Jan. 23, 2017, 10:09 p.m.) > > > Review request for atlas, Madhan Neethiraj and Suma Shivaprasad. > > > Bugs: ATLAS-1403 > https://issues.apache.org/jira/browse/ATLAS-1403 > > > Repository: atlas > > > Description > ------- > > Currently DSL uses a fill function during Gremlin Translation to merge > results by typeName and superTypeName and fill function loads the resulting > vertices in memory. This causes significant memory usage and ATLAS server > spends lot of time doing GC instead of useful work resulting in OOO sometimes > ( when GC is not able to recover and search queries are run in parallel) > The proposal is to replace this with typeName checks along by finding all the > subtypes for a given type and using an IN clause in the filter. > For eg: > Query = Person where (birthday < "1950-01-01T02:35:58.440Z") limit 40 offset 0 > Optimized query > Gremlin Query = L: > {g.V.has("__typeName", T.in, > ['Person','Manager']).and(_().has("Person.birthday", T.lt, -631142641560)) > [0..<40].toList()} > > > Diffs > ----- > > > repository/src/main/java/org/apache/atlas/discovery/DataSetLineageService.java > fd5dba7 > > repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java > 266f27c > > repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java > b637f90 > > repository/src/main/java/org/apache/atlas/gremlin/Gremlin2ExpressionFactory.java > 41dc65f > > repository/src/main/java/org/apache/atlas/gremlin/GremlinExpressionFactory.java > 3677544 > repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java > 889236c > repository/src/main/scala/org/apache/atlas/query/ClosureQuery.scala daef582 > > repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala > a9dcdff > repository/src/main/scala/org/apache/atlas/query/GremlinEvaluator.scala > ade4176 > repository/src/test/scala/org/apache/atlas/query/GremlinTest2.scala 33513c5 > > Diff: https://reviews.apache.org/r/55813/diff/ > > > Testing > ------- > > Ran Unit Tests and was successful. > > > Thanks, > > Sarath Subramanian > >
