[
https://issues.apache.org/jira/browse/TINKERPOP-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15159644#comment-15159644
]
Marko A. Rodriguez commented on TINKERPOP-1166:
-----------------------------------------------
The new Reducing model took the Friendster g.V().count() from 6.8 minutes to
5.6 minutes w/ 3.4 of those minutes spent loading the graph
(ScriptInputFormat). The gain is because count(), group(), groupCount(), min(),
max(), etc. no longer have subsequent MapReduce jobs and thus, one less scan of
the full graph. Another benefit of this model is that there are "parallel
reductions." {{groupCount("m")....count()}} is not two subsequent MapReduce
jobs, but embedded in the {{VertexProgram}} execution.
> Add Memory.reduce() as option to Memory implementations.
> --------------------------------------------------------
>
> Key: TINKERPOP-1166
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1166
> Project: TinkerPop
> Issue Type: Improvement
> Components: hadoop, process, tinkergraph
> Affects Versions: 3.1.2-incubating
> Reporter: Marko A. Rodriguez
> Labels: breaking
>
> Currently {{Memory}} supports {{incr}}, {{and}}, {{or}}, ... These are great
> and what people will typically use. However, we should also provide the
> generalization which is simply {{Memory.reduce}}. In this situation,
> {{incr}}, {{or}}, {{and}}, etc. are just specifications of {{Memory.reduce}}.
> How would it work?
> When memory is initialized in a {{VertexProgram}}, it would be like this:
> {code}
> memory.set("myReduction", new MyReducingFunction(0))
> {code}
> Then {{ReducingFunction}} would look like this:
> {code}
> public class ReducingFunction implements UnaryOperator<A> {
> public A getInitialValue();
> public A apply(A first, A second);
> }
> {code}
> Easy peasy. Note that both Spark and Giraph support such types of
> function-based reduction in their respective "memory engines."
> TinkerGraphComputer will, of course, be easy to add this functionality too.
> Why do this? For two reasons:
> 1. We get extra flexibility in {{Memory}}.
> 2. https://issues.apache.org/jira/browse/TINKERPOP-1164
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
