> > would this notion of a "mutation tags" be useful in > BulkLoaderVertexProgram?
Yes. I've used this approach in DSEG's BulkUpdateVertexProgram, which allows to add properties and edges. IMO we should also create a new VP in TinkerPop, one that is optimized to handle all update scenarios. 1. How do we deal with ID generation? Do we need to care about IDs for edges and properties? I don't think so. Vertices would be tricky though, but... 2. How do we add vertices in OLAP? We never allowed that in the past iirc, but that would be a "great to have". The question is, do we then have to care about IDs? The user could provide an ID or the underlying Graph DB could provide one at write time. 3. How do we deal with updates that already occurred at the graph database > while OLAP was processing? It depends. Some scenarios would probably be solvable, others would lead to failure. Maybe we can even make some of the behaviors configurable through the BulkLoaderVP. Cheers, Daniel On Tue, Apr 12, 2016 at 7:57 PM, Marko Rodriguez <okramma...@gmail.com> wrote: > Hello, > > One of the big outstanding features in TinkerPop3 is OLAP-based graph > mutations. That is: > > graph database --read--> graph processor --write--> graph database > > Daniel Kuppitz has provided BulkLoaderVertexProgram with "from scratch" or > incremental loading capabilities. It would be nice to be able to piggy back > on this work for OLAP mutations. Here is an idea that borrows from the OLAP > mutation model developed by Matthias Broecheler in Faunus. > > 1. We create a MutatingProgram interface. > 2. That interface will have a collection of public static final String > compute keys. > - gremlin.mutatingProgram.mutation > - gremlin.mutatingProgram.droppedProperties > - gremlin.mutatingProgram.addedProperties > 3. Any VertexProgram can implement MutatingProgram and in doing so, it > will use the respective compute keys. > 4. If that VertexProgram deletes a vertex, it does not delete the vertex, > it simple adds the property "gremlin.mutatingProgram.mutation=dropped." > 5. If that VertexProgram adds an edge, it adds the edge with the property > "gremlin.mutatingProgram.mutation=added." > 6. If that VertexProgram adds a vertex property, it adds the vertex > property with the property "gremlin.mutatingProgram.mutation=added." > 7. If that VertexProgram deletes a property, it adds the property to the > element "gremlin.mutatingProgram.droppedProperties=[x,y,z]" > 8. ... > 9. It is up to the VertexProgram to be smart about consistency on > mutations: > * If an edge is added, the next iteration should copy that edge to > incoming/outgoing vertex's incident edge set. > * If an edge property is added, the next iteration should update > the property on the incoming/outgoing vertex's incident edge's property. > * We can provide various static helper methods in MutatingProgram > to make this easy. > 10. When the VertexProgram has completed its computation (terminated), > BulkLoaderVertexProgram will be able to read the resultant graph and use > the MutatingProgram compute keys as necessary to do the respective updates > to the source graph (i.e. the graph database). > > --------------------- > > Problems: > > 1. How do we deal with ID generation? > 2. How do we add vertices in OLAP? > 3. How do we deal with updates that already occurred at the graph > database while OLAP was processing? > > @kuppitz -- would this notion of a "mutation tags" be useful in > BulkLoaderVertexProgram? I assume it would make your life much easier as > you don't have to do a "diff" -- the diff is provided to you. > > Thoughts on this matter would be much appreciated. > > Thank you, > Marko. > > http://markorodriguez.com > >
