Hi, Ok thanks for that. I am trying some trickery. Will let you know where the trickery leads to. CheersPieter
On Thu, 2017-04-20 at 13:59 -0600, Marko Rodriguez wrote: > Pieter…. you have a fascinating problem. Because this touches exactly > at one of the fundamental concepts of Gremlin — local vs. global > traversals. > g.V().out().count() == g.V().local(out()).count() > However, > > g.V().out().count() != g.V().local(out().count()) > /////////////// > > gremlin> g.V().out().count() > ==>6 > gremlin> g.V().local(out()).count() > ==>6 > gremlin> g.V().out().count() > ==>6 > gremlin> g.V().local(out().count()) > ==>3 > ==>0 > ==>0 > ==>2 > ==>0 > ==>1 > gremlin> > So, your ability to grab the "global stream” and not just a single > object from it within a local traversal will require some trickery. > > Look at: > > g.V().local(out().count()) > How would we do this so you pull all the V()’s into the Sqlg[out()] > but still single stream post process? Thats a good question. Hmmmm… > > > g.V().aggregate(‘x’).cap(‘x’).local(sqlgOut().count()) > Now, Sqlg[out()] would do this: > > 1. Is the input a list? If yes, then execute all in batch. > 2. Then pop off the first mapping as the output to count() > only! > 3. then Sqlg[out()].reset() method doesn’t clear….. > > Wait… no, it can’t do that………………………………cause it will not next() > correctly. > > Hm. Wow…. mind blown. > > Owwww…………. > > Marko. > > > > > > > On Apr 20, 2017, at 10:47 AM, pieter <[email protected]> > > wrote: > > > > Sorry, forwarding was not a good idea either, > > > > Here is an example with global children and the batching works > > well. > > Sqlg does not currently optimize the 'where' (TraversalFilterStep). > > > > @Test > > public void testBatchingIncomingTraversersOnVertexStep() { > > int count = 10_000; > > for (int i = 0; i < count; i++) { > > Vertex a1 = this.sqlgGraph.addVertex(T.label, "A"); > > Vertex b1 = this.sqlgGraph.addVertex(T.label, "B"); > > a1.addEdge("ab", b1); > > } > > this.sqlgGraph.tx().commit(); > > > > GraphTraversal traversal = this.sqlgGraph.traversal() > > .V().where(__.hasLabel("A")) > > .out(); > > printTraversalForm(traversal); > > List<Vertex> vertices = traversal.toList(); > > assertEquals(count, vertices.size()); > > } > > > > This prints out, > > > > pre-strategy:[GraphStep(vertex,[]), > > TraversalFilterStep([HasStep([~label.eq(A)])]), > > VertexStep(OUT,vertex)] > > post- > > strategy:[SqlgGraphStepCompiled(vertex,[])@[sqlgPathFakeLabel], > > HasStep([~label.eq(A)]), > > SqlgVertexStepCompiled@[sqlgPathFakeLabel]] > > > > The SqlgVertexStepCompiled is able to iterate all 10 000 incoming > > traversers and execute one query for the out(). > > > > This reduced the query time from 12 seconds to 0.4 seconds. > > Happiness!! > > > > An example with a local traversal. > > > > @Test > > public void testBatchingIncomingTraversersOnLocalVertexStep() { > > int count = 10_000; > > for (int i = 0; i < count; i++) { > > Vertex a1 = this.sqlgGraph.addVertex(T.label, "A"); > > Vertex b1 = this.sqlgGraph.addVertex(T.label, "B"); > > a1.addEdge("ab", b1); > > } > > this.sqlgGraph.tx().commit(); > > > > GraphTraversal traversal = this.sqlgGraph.traversal() > > .V().hasLabel("A") > > .local( > > __.out() > > ); > > printTraversalForm(traversal); > > List<Vertex> vertices = traversal.toList(); > > Assert.assertEquals(count, vertices.size()); > > } > > > > This prints out, > > > > pre-strategy:[GraphStep(vertex,[]), HasStep([~label.eq(A)]), > > LocalStep([VertexStep(OUT,vertex)])] > > post- > > strategy:[SqlgGraphStepCompiled(vertex,[])@[sqlgPathFakeLabel], > > LocalStep([SqlgVertexStepCompiled@[sqlgPathFakeLabel]])] > > > > In this case SqlgVertexStepCompiled is a local traversal of the > > LocalStep. > > > > Iterating the starts only returns one traverser as the LocalStep > > only > > puts one on the traversal at a time. > > > > I suppose I can replace LocalStep with a custom one but there are > > many > > steps with local children which will make things > > fragile if I were to replace so many steps in a copy paste fashion. > > > > Thanks > > Pieter > > > > > > On Thu, 2017-04-20 at 09:10 -0600, Marko Rodriguez wrote: > > > Hello, > > > > > > > I have started optimizing Sqlg to do a bulk/barrier for its > > > > VertexStep > > > > optimizations. > > > > > > Cool. > > > > > > > Sqlg has two optimization strategies. > > > > > > > > GraphStepStrategy and VertexStepStrategy. GraphStepStrategy > > > > executes > > > > first and then VertexStepStrategy. > > > > > > > > GraphStepStrategy starts at the beginning of the traversal > > > > optimizing > > > > from left to right till it reaches a step that it can not > > > > optimize > > > > and > > > > terminates. > > > > > > Makes sense. > > > > > > > After that VertexStepStrategy tries to optimize what remains. > > > > It ultimately replaces optimizable sequential steps with a > > > > SqlgVertexStep. > > > > > > Okay... > > > > > > > Thus far the SqlgVertexStep always has one incoming traverser > > > > from > > > > where it continues the traversal. Basically it translated to a > > > > sql > > > > where clause with the incoming traversal element's id. > > > > > > > > The current optimization is to bulk the incoming traversers and > > > > execute > > > > the traversal for all incoming traversers in one go. This > > > > reduces > > > > latency and has a drastic performance improvement. > > > > > > > > I do the same as the existing BarrierSteps and iterate the > > > > `starts` > > > > to > > > > collect all the left incoming traversers and from there I > > > > continue > > > > and > > > > all is well. > > > > > > Smart. You got chops. > > > > > > > However for local traversals there is only one start on the > > > > traversal > > > > so the barrier idea is not working. > > > > > > > > Is there a way barrier all incoming left traversers on local > > > > traversals? > > > > > > Eeeeeeeeeeeeeeee…… huuuuuuuhhhhhhhh……………. > > > > > > There is a “easy” and there is a “hard.” Give me an example > > > traversal > > > and lets discuss from a more specific standpoint before > > > generalizing... > > > > > > Thanks, > > > Marko. > > > > > > http://markorodriguez.com
