[
https://issues.apache.org/jira/browse/TINKERPOP-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16882008#comment-16882008
]
Wang Xiyu edited comment on TINKERPOP-2256 at 7/10/19 12:47 PM:
----------------------------------------------------------------
A quick modification:
[https://github.com/wangxiyu191/tinkerpop/commit/954247db3a147ac90c7d2df49df0eb88e514e122]
Machine: 2 * Intel(R) Xeon(R) CPU E5-2630
original verison (06601d9a0e61bf01797bd68443798b97b586f114):
{code:java}
==>Traversal Metrics
Step Count
Traversers Time (ms) % Dur
=============================================================================================================
TinkerGraphStep(vertex,[1]) 1
1 233.972 9.54
VertexStep(OUT,vertex) 7
7 268.718 10.96
VertexStep(OUT,vertex) 190
190 261.400 10.66
VertexStep(OUT,vertex) 7846
7846 259.702 10.59
VertexStep(OUT,vertex) 344093
344093 378.445 15.44
AggregateStep(a) 344093
343721 872.621 35.59
PathStep 344093
343721 176.785 7.21
>TOTAL -
- 2451.646 -
{code}
call processNextStart only once (954247db3a147ac90c7d2df49df0eb88e514e122) :
{code:java}
==>Traversal Metrics
Step Count
Traversers Time (ms) % Dur
=============================================================================================================
TinkerGraphStep(vertex,[1]) 1
1 0.099 0.01
VertexStep(OUT,vertex) 7
7 0.049 0.00
VertexStep(OUT,vertex) 190
190 0.098 0.01
VertexStep(OUT,vertex) 7846
7846 1.912 0.18
VertexStep(OUT,vertex) 344093
344093 95.345 9.22
AggregateStep(a) 344093
343721 816.591 78.95
PathStep 344093
343721 120.165 11.62
>TOTAL -
- 1034.262 -
{code}
was (Author: wangxiyu191):
A quick modification:
[https://github.com/wangxiyu191/tinkerpop/commit/954247db3a147ac90c7d2df49df0eb88e514e122]
Machine: 2 * Intel(R) Xeon(R) CPU E5-2630
call processNextStart only once ( 06601d9a0e61bf01797bd68443798b97b586f114 ) :
{code:java}
==>Traversal Metrics
Step Count
Traversers Time (ms) % Dur
=============================================================================================================
TinkerGraphStep(vertex,[1]) 1
1 233.972 9.54
VertexStep(OUT,vertex) 7
7 268.718 10.96
VertexStep(OUT,vertex) 190
190 261.400 10.66
VertexStep(OUT,vertex) 7846
7846 259.702 10.59
VertexStep(OUT,vertex) 344093
344093 378.445 15.44
AggregateStep(a) 344093
343721 872.621 35.59
PathStep 344093
343721 176.785 7.21
>TOTAL -
- 2451.646 -
{code}
original verison (954247db3a147ac90c7d2df49df0eb88e514e122) :
{code:java}
==>Traversal Metrics
Step Count
Traversers Time (ms) % Dur
=============================================================================================================
TinkerGraphStep(vertex,[1]) 1
1 0.099 0.01
VertexStep(OUT,vertex) 7
7 0.049 0.00
VertexStep(OUT,vertex) 190
190 0.098 0.01
VertexStep(OUT,vertex) 7846
7846 1.912 0.18
VertexStep(OUT,vertex) 344093
344093 95.345 9.22
AggregateStep(a) 344093
343721 816.591 78.95
PathStep 344093
343721 120.165 11.62
>TOTAL -
- 1034.262 -
{code}
> processAllStarts of AggregateStep should be called only once
> ------------------------------------------------------------
>
> Key: TINKERPOP-2256
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2256
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
> Reporter: Wang Xiyu
> Priority: Minor
>
> Currently the function processNextStart , hasNextBarrier, nextBarrier of
> AggregateStep all call AggregateStep.processAllStarts:
> {code:java}
> protected Traverser.Admin<S> processNextStart() {
> this.processAllStarts();
> return this.barrier.remove();
> }{code}
> Every time we get a traverser from AggregateStep,
> AggregateStep.processAllStarts will be called. Then processAllStarts calls
> this.starts.hasNext().
> {code:java}
> @Override
> public void processAllStarts() {
> if (this.starts.hasNext()) {
> final BulkSet<Object> bulkSet = new BulkSet<>();
> while (this.starts.hasNext()) {
> final Traverser.Admin<S> traverser = this.starts.next();
> bulkSet.add(TraversalUtil.applyNullable(traverser,
> this.aggregateTraversal), traverser.bulk());
> traverser.setStepId(this.getNextStep().getId()); // when
> barrier is reloaded, the traversers should be at the next step
> this.barrier.add(traverser);
> }
> this.getTraversal().getSideEffects().add(this.sideEffectKey,
> bulkSet);
> }
> }
> {code}
> It results in a lot of hasNext call.
> As document says "The step uses [eager
> evaluation|http://en.wikipedia.org/wiki/Eager_evaluation] in that no objects
> continue on until all previous objects have been fully aggregated." maybe we
> can limit the AggregateStep.processAllStarts only be called once.
>
>
>
> We found this when we run DSL like this :
> {code:java}
> g.V().has('name','wxy').repeat(both("knows").simplePath()).emit().times(2).aggregate("friends"){code}
> and the plan is like this :
> {code:java}
> GraphStep(vertex,[name.eq(wxy)]),
> RepeatStep([VertexStep(BOTH,[knows],vertex), PathFilterStep(simple),
> RepeatEndStep],until(loops(2)),emit(true)), AggregateStep(friends)
> {code}
> Then we found thousands of calls to GraphStep(vertex,[name.eq(wxy)]).hasNext.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
