[
https://issues.apache.org/jira/browse/TINKERPOP-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Norio Akagi updated TINKERPOP-2393:
-----------------------------------
Description:
Right now in BranchStep, if children contain a BarrierStep it keeps calling
applyCurrentTraverser until this.starts is exhausted.
[https://github.com/apache/tinkerpop/blob/534746b6019d0775e628375a3dfeba8a18df90fe/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/branch/BranchStep.java#L139-L140]
This means the child step may hold all upstream solutions in memory at once,
and potentially memoyy usage is bloated.
I think here we can change so that BarrierStep can have a method like
`getMaxBarrierSize`, and BranchStep examines the size of barrier that child
steps can handle at once then breaks the while loop either when
`this.starts.hasNext` becomes false or we exceed the barrier size to save the
potential memory consumption.
When there are multiple barrier steps as children, we may pick either max or
min among them.
I am happy to implement this, but please let me know if you have any comments
or concerns on this change.
Thanks
was:
Right now in BranchStep, if children contain a BarrierStep it keeps calling
applyCurrentTraverser until this.starts is exhausted.
[https://github.com/apache/tinkerpop/blob/534746b6019d0775e628375a3dfeba8a18df90fe/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/branch/BranchStep.java#L139-L140]
This means child step may hold all upstream solutions in memory and potentially
bloated.
I think here we can change so that BarrierStep can have a method like
`getMaxBarrierSize`, then BranchStep examines the size and break the while loop
either when `this.starts.hasNext` becomes false or we exceed the barrier size
to save the potential memory consumption.
When there are multiple barrier steps as children, we may pick either max or
min among them.
I am happy to implement this, but please let me know if you have any comments
or concerns on this change.
Thanks
> BranshStep should respect the barrier size of BarrierStep when calling
> applyCurrentTraverser
> --------------------------------------------------------------------------------------------
>
> Key: TINKERPOP-2393
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2393
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
> Reporter: Norio Akagi
> Priority: Minor
>
> Right now in BranchStep, if children contain a BarrierStep it keeps calling
> applyCurrentTraverser until this.starts is exhausted.
> [https://github.com/apache/tinkerpop/blob/534746b6019d0775e628375a3dfeba8a18df90fe/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/branch/BranchStep.java#L139-L140]
> This means the child step may hold all upstream solutions in memory at once,
> and potentially memoyy usage is bloated.
> I think here we can change so that BarrierStep can have a method like
> `getMaxBarrierSize`, and BranchStep examines the size of barrier that child
> steps can handle at once then breaks the while loop either when
> `this.starts.hasNext` becomes false or we exceed the barrier size to save the
> potential memory consumption.
>
> When there are multiple barrier steps as children, we may pick either max or
> min among them.
> I am happy to implement this, but please let me know if you have any comments
> or concerns on this change.
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
