[
https://issues.apache.org/jira/browse/TINKERPOP-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153272#comment-15153272
]
Marko A. Rodriguez commented on TINKERPOP-1166:
-----------------------------------------------
More thinking on this. I think we might need to make this a breaking change for
both providers and users. Its not too brutal though.
{code}
VertexProgram.getComputeKeys() -> Set<ComputeKey>
VertexProgram.getMemoryKeys() -> Set<MemoryKey>
ComputeKey [String key, Persistence.STATIC/TRANSIENT]
MemoryKey [String key, UnaryOperator reducer, Persistence.STATIC/TRANSIENT]
{code}
This will solve TINKERPOP-1162 too.
Next, in {{VertexProgram.setup(Memory memory)}} user will do:
{code}
memory.register("vote.to.halt", true, {a,b -> a || b}) // we can have
OrOperator.instance() all nice and static.
memory.register("globalCount", 0, {a,b -> a + b})
{code}
Then, when someone wants to update a memory item in the
{{VertexProgram.execute()}}-method, they do:
{code}
memory.add("vote.to.halt", false)
memory.add("globalCount", 134345)
{code}
Finally, in the {{VertexProgram.terminate(Memory memory)}} method, memory can
be hard set (like how it is now).
{code}
memory.set("vote.to.halt",true)
memory.set("globalCount",0)
{code}
There is nothing "crazy here," its just an API change. Why this won't be
toooooo bad.
* Users typically don't write VertexPrograms so I bet not many exist. And if
they do, they are built by "power users" who will appreciate why this is better.
* GraphComputer providers are few and far between (seems most just use
SparkGraphComputer). However, for those that do exist, the code is still "the
same", its primarily an API rework. Not tooooooooo crazy, but yea, they will
have to do something.
The benefits:
1. We get transient keys finally settled. This is huge for data reduction. (see
TINKERPOP-1162)
2. We are on our way to gutting the {{MapReduce}} infrastructure. (perhaps a
Deprecation in 3.3.0)
3. We get super efficient/fast {{ReducingBarrierStep}} OLAP implementations.
To add to (3), its crazy cause now Traversal/SideEffects and
GraphComputer/Memory are in one-to-one correspondence. Its very elegant.
> Add Memory.reduce() as option to Memory implementations.
> --------------------------------------------------------
>
> Key: TINKERPOP-1166
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1166
> Project: TinkerPop
> Issue Type: Improvement
> Components: hadoop, process, tinkergraph
> Affects Versions: 3.1.2-incubating
> Reporter: Marko A. Rodriguez
> Labels: breaking
>
> Currently {{Memory}} supports {{incr}}, {{and}}, {{or}}, ... These are great
> and what people will typically use. However, we should also provide the
> generalization which is simply {{Memory.reduce}}. In this situation,
> {{incr}}, {{or}}, {{and}}, etc. are just specifications of {{Memory.reduce}}.
> How would it work?
> When memory is initialized in a {{VertexProgram}}, it would be like this:
> {code}
> memory.set("myReduction", new MyReducingFunction(0))
> {code}
> Then {{ReducingFunction}} would look like this:
> {code}
> public class ReducingFunction implements UnaryOperator<A> {
> public A getInitialValue();
> public A apply(A first, A second);
> }
> {code}
> Easy peasy. Note that both Spark and Giraph support such types of
> function-based reduction in their respective "memory engines."
> TinkerGraphComputer will, of course, be easy to add this functionality too.
> Why do this? For two reasons:
> 1. We get extra flexibility in {{Memory}}.
> 2. https://issues.apache.org/jira/browse/TINKERPOP-1164
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
