Hello,
So Distributed OLTP is done. I was able to get barriers, side-effects,
auto-halting, stalling traversers for bulking, etc. implemented since this
morning and well, tada.
There are more classes now, so here is a link to the package root. Again, this
is just a test/ package in TinkerGraph for now to make it easy for me to play.
https://github.com/apache/tinkerpop/tree/TINKERPOP-1564/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/akka
<https://github.com/apache/tinkerpop/tree/TINKERPOP-1564/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/akka>
In TinkerActorSystem, you will see 3 traversals that executed in the public
static main(). Here is the output.
EXECUTING: [[withStrategies(VertexProgramStrategy)], [V(), match([[], [as(a),
out(created), as(b)]], [[], [as(b), in(created), as(c)]], [[], [as(b),
has(name, eq(lop))]]), where(a, neq(c)), select(a, b, c), by(name)]]
master[created]: akka://traversal-1552478766/user/master
worker[created]: akka://traversal-1552478766/user/master/worker-765740875
worker[created]: akka://traversal-1552478766/user/master/worker-458575643
worker[created]: akka://traversal-1552478766/user/master/worker-2134136508
master[result]: {a=peter, b=lop, c=josh}
master[result]: {a=marko, b=lop, c=peter}
master[result]: {a=josh, b=lop, c=marko}
master[result]: {a=josh, b=lop, c=peter}
master[result]: {a=peter, b=lop, c=marko}
master[result]: {a=marko, b=lop, c=josh}
//////////////////////////////////
EXECUTING: [[withStrategies(VertexProgramStrategy)], [V(), repeat([[],
[both()]]), times(2), groupCount(a), by(name), cap(a), select(keys), unfold(),
limit(3)]]
master[created]: akka://traversal--116529363/user/master
worker[created]: akka://traversal--116529363/user/master/worker-97122649
worker[created]: akka://traversal--116529363/user/master/worker-32927032
worker[created]: akka://traversal--116529363/user/master/worker-1156903778
master[result]: ripple
master[result]: peter
master[result]: vadas
//////////////////////////////////
EXECUTING: [[withStrategies(VertexProgramStrategy)], [V(), repeat([[],
[both()]]), times(2), hasLabel(person), group(), by(name), by([[],
[out(created), values(name), dedup(), fold()]])]]
master[created]: akka://traversal--558368687/user/master
worker[created]: akka://traversal--558368687/user/master/worker-704359101
worker[created]: akka://traversal--558368687/user/master/worker-767830874
worker[created]: akka://traversal--558368687/user/master/worker-1757847686
master[result]: {peter=[lop], vadas=[], josh=[ripple, lop], marko=[lop]}
//////////////////////////////////
Why are these traversals cool?
1. match() works.
2. side-effects work.
3. reducing barriers work and check it, you can traverser beyond the
local star graph!
What really opened everything up was realizing I could implement my own Mailbox.
https://github.com/apache/tinkerpop/blob/TINKERPOP-1564/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/akka/TraverserMailbox.java
<https://github.com/apache/tinkerpop/blob/TINKERPOP-1564/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/akka/TraverserMailbox.java>
This allows me to order how certain types of messages are read by the actors
and most importantly, allows me to back Traverser messages by a TraverserSet
and thus, we now have the bulking optimization.
So freakin’ cool.
Enjoy!,
Marko.
http://markorodriguez.com
> On Dec 7, 2016, at 5:13 AM, Marko Rodriguez <[email protected]> wrote:
>
> Hello,
>
> I have been studying Akka lately as a way to implement distributed
> OLTP/dynamic query routing. I have a working implementation.
>
> This is a “toy implementation” currently in the TINKERPOP-1564 test/ of
> tinkergraph-gremlin.
>
> There are 3 classes:
>
> TinkerActorSystem
>
> https://github.com/apache/tinkerpop/blob/0462411f1d1ad6fab9d3e6e58e5e158c9bb3f207/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/akka/TinkerActorSystem.java
>
> <https://github.com/apache/tinkerpop/blob/0462411f1d1ad6fab9d3e6e58e5e158c9bb3f207/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/akka/TinkerActorSystem.java>
> This class creates an Akka ActorSystem for a submitted traversal. It then
> spawns a MasterTraversalActor given the traversal and a partitioner. The
> partitioner is currently hardcoded.
>
> MasterTraversalActor
>
> https://github.com/apache/tinkerpop/blob/0462411f1d1ad6fab9d3e6e58e5e158c9bb3f207/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/akka/MasterTraversalActor.java
>
> <https://github.com/apache/tinkerpop/blob/0462411f1d1ad6fab9d3e6e58e5e158c9bb3f207/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/akka/MasterTraversalActor.java>
> This class spawns a WorkerTraversalActor for each partition in the
> partitioner. It then “starts” each worker and awaits halted traversers
> (results).
>
> WorkerTraversalActor
>
> https://github.com/apache/tinkerpop/blob/0462411f1d1ad6fab9d3e6e58e5e158c9bb3f207/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/akka/WorkerTraversalActor.java
>
> <https://github.com/apache/tinkerpop/blob/0462411f1d1ad6fab9d3e6e58e5e158c9bb3f207/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/akka/WorkerTraversalActor.java>
> This class streams out its partition's start step traversers. If the
> traverser references elements local to its partition, it process it. Else, it
> messages the traverser to the respective worker partition. It also receives
> traversers from remote workers and processes them accordingly. If it yields a
> halted traverser, the worker sends the halted traverser to the master
> traversal.
>
> Here is the System.out data when using:
>
> g.V().match(
> as("a").out("created").as("b"),
> as("b").in("created").as("c"),
> as("b").has("name", eq("lop"))).
> where("a", neq("c")).
> select("a", "b", "c").by("name")
>
> master[created]: akka://traversal-1552478766/user/master
> <akka://traversal-1552478766/user/master>
> worker[created]: akka://traversal-1552478766/user/master/worker-1211806485
> <akka://traversal-1552478766/user/master/worker-1211806485>
> worker[created]: akka://traversal-1552478766/user/master/worker-1261612621
> <akka://traversal-1552478766/user/master/worker-1261612621>
> worker[created]: akka://traversal-1552478766/user/master/worker-1864420351
> <akka://traversal-1552478766/user/master/worker-1864420351>
> worker[created]: akka://traversal-1552478766/user/master/worker-1054674616
> <akka://traversal-1552478766/user/master/worker-1054674616>
> worker[created]: akka://traversal-1552478766/user/master/worker-593666157
> <akka://traversal-1552478766/user/master/worker-593666157>
> master[result]: {a=marko, b=lop, c=peter}
> master[result]: {a=josh, b=lop, c=marko}
> master[result]: {a=marko, b=lop, c=josh}
> master[result]: {a=peter, b=lop, c=josh}
> master[result]: {a=peter, b=lop, c=marko}
> master[result]: {a=josh, b=lop, c=peter}
>
>
> This implementation currently does not support traversals with barriers or
> side-effects. Moreover, it does not “stall the workers” to build up traverser
> bulks. In other words, it currently messages one traverser at a time.
>
> I hope you can appreciate the simplicity of implementation. Using
> akka-remote, the code stays the same, save the URIs of the Akka actors
> changes. Its all pretty basic in fact.
>
> Enjoy,
> Marko.
>
> http://markorodriguez.com <http://markorodriguez.com/>
