[
https://issues.apache.org/jira/browse/TINKERPOP3-702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695675#comment-14695675
]
Ran Magen commented on TINKERPOP3-702:
--------------------------------------
Hey Matt,
I don't really follow, I might have not explained myself well.
I've got a specialized {{VertexStep}} that basically batches up the incoming
vertices like so:
{code:java}
while(this.starts.hasNext() && count < REASONABLE_BULK_SIZE) {
addToBulk(this.starts.next()); count++; }
{code}
After this I query the bulk of the vertices, temporarily cache them in the
step, and release them one by one in a flatMap fashion.
Now, this works great for traversals of the form {{g.V().out()}} (as an
example). But any time my {{VertexStep}} is "trapped" inside an inner traversal
(as is the case in {{RepeatStep}}, {{CoalesceStep}}, etc), I can't utilize this
bulking mechanism, since only one traverser is "released" at a time (see
[CoalesceStep|https://github.com/apache/incubator-tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/CoalesceStep.java#L50],
[RepeatStep|https://github.com/apache/incubator-tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/branch/RepeatStep.java#L156]).
Basically only enabling a local scope behavior.
This behavior makes sense in {{LocalStep}}, but I still don't understand why
these other steps act like this. Can't someone needing a local scope behavior
just use the {{LocalStep}} in the inner traversal? Why limit these steps from
working at a global scope?
> Buffer input to inner traversals
> --------------------------------
>
> Key: TINKERPOP3-702
> URL: https://issues.apache.org/jira/browse/TINKERPOP3-702
> Project: TinkerPop 3
> Issue Type: Improvement
> Components: process
> Reporter: Ran Magen
> Assignee: Marko A. Rodriguez
> Fix For: 3.0.0-incubating
>
>
> In elastic-gremlin we implement an optimized VertexStep. Part of its job is
> to batch/buffer/bulk different traversers and query them together in-order to
> minizmize the number of queries.
> You can see the implementation here:
> https://github.com/rmagen/elastic-gremlin/blob/master/src/main/java/org/elasticgremlin/process/optimize/ElasticVertexStep.java#L36
> This works great in regular traversals, the "starts" iterator returns as many
> traversers as the previous step gave out.
> But when the step is in an innerTraversal (e.g.
> g.V().repeat(__.out()).times(8)), the "starts" iterator only returns one
> traverser, and will return the next traverser only in the next call to
> processNextStart. Thus, there is no way to run a bulk query.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
