[
https://issues.apache.org/jira/browse/TINKERPOP-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152432#comment-15152432
]
Marko A. Rodriguez commented on TINKERPOP-1164:
-----------------------------------------------
I just tested this and it works. It was literally 5 lines of code I needed to
change. Its "hardcoded" to only work for {{CountGlobalStep}}, but can easily be
generalized to all {{ReducingBarrierSteps}}. This is insane. This will greatly
speed up reducing steps and will allow for reducing steps to be mid-traversal.
Here are the steps that benefit from this:
{code}
sum()
max()
min()
count()
mean()
group()
groupCount()
tree()
fold()
{code}
What is crazy is that besides {{aggregate()}} and {{store()}}, those are the
only steps in Gremlin that have {{MapReduce}}
implementations................................... this might be the argument
for the death of {{MapReduce}} (in the future). For now, we can simply just
change these steps to use {{Memory}} and its all backwards compatible and no
one is the wiser.
> ReducingBarriersSteps should use ComputerMemory, not MapReduce.
> ---------------------------------------------------------------
>
> Key: TINKERPOP-1164
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1164
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
> Affects Versions: 3.1.0-incubating
> Reporter: Marko A. Rodriguez
>
> This just hit me like a ton of bricks. Check this:
> {code}
> g.V().count()
> {code}
> That is:
> {code}
> TraversalVertexProgram + CountMapReduce
> {code}
> Thats stupid. Just use the "reduction" aspects of {{Memory}}. Replace
> {{CountMapReduce}} with:
> {code}
> memory.incr("~reducing", traverser.bulk())
> {code}
> Thats it. Likewise for all the other reducing barriers! Now, not only do we
> don't have to do a {{MapReduce}} job, we don't even have to break out of the
> {{TraversalVertexProgram}} (no more "No mid-barrier steps."). Why? Well,
> because its in memory, its computed on that iteration and then accessible!
> {code}
> g.V().group().by('lang').select('java').values("name")
> {code}
> That would be one TraversalVertexProgram!
> .....................why do we even have MapReduce..........................
> is Memory all we really need.
> Crazy.............................................. thats craZy
> talk................................. but still, think about it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
