[
https://issues.apache.org/jira/browse/TINKERPOP-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17163579#comment-17163579
]
Stephen Mallette commented on TINKERPOP-2393:
---------------------------------------------
Interesting idea and it's nice to see someone looking for optimizations like
this one. If I'm not mistaken the only {{Barrier}} with a configurable size is
{{NoOpBarrier}}. If so, it would basically mean that we'd only see a value less
than "exhaust the stream" for that {{Step}}, is that right?
Offhand I'm not sure if I can think of anything particularly wrong with what
you are suggesting so I think it's worth trying to put into practice. In
addition to normal tests it would be nice to see some profiling that
demonstrates the improvement in memory here. Please submit your pull request to
the {{3.4-dev}} branch assuming this change only ends up improving resource
usage. If it ends up producing some breaking change then it should go to the
{{master}} branch. Thanks.
> BranshStep should respect the barrier size of BarrierStep when calling
> applyCurrentTraverser
> --------------------------------------------------------------------------------------------
>
> Key: TINKERPOP-2393
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2393
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
> Reporter: Norio Akagi
> Priority: Minor
>
> Right now in BranchStep, if children contain a BarrierStep it keeps calling
> applyCurrentTraverser until this.starts is exhausted.
> [https://github.com/apache/tinkerpop/blob/534746b6019d0775e628375a3dfeba8a18df90fe/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/branch/BranchStep.java#L139-L140]
> This means the child step may hold all upstream solutions in memory at once,
> and potentially memory usage is bloated.
> I think here we can change so that BarrierStep can have a method like
> getMaxBarrierSize, and BranchStep examines the size of barrier that child
> steps can handle at once then break the while loop either when
> `this.starts.hasNext` becomes false or we exceed the barrier size to save the
> potential memory consumption.
>
> When there are multiple barrier steps as children, we may pick either max or
> min among them.
> I am happy to implement this, but please let me know if you have any comments
> or concerns on this change.
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
