> Now when bytecode (containing instructions for execution) is submitted to LocalMachine, it will lookup the registered UUID and if it exists, use the pre-compiled source code.
maybe i didn't follow properly but is this for the purpose of caching traversals to avoid the costs of traversal to bytecode compilation? in other words, is this describing a general way to cache compiled bytecode so that it doesn't have to go through strategy application more than once? On Mon, Mar 25, 2019 at 8:48 AM Marko Rodriguez <[email protected]> wrote: > Hi, > > Here is how the TP4 bytecode submission infrastructure is looking. > > In TP3, TraversalSource maintained the “pre-compilation” of strategies, > database connectivity, etc. This was not smart for the following reasons: > > 1. It assumed the traversal would execute on the same machine that > it was created on. > 2. We had to make an explicit distinction between local and remote > execution via RemoteStrategy. > 3. RemoteStrategy passes an excessive amount of data over the wire > on each traversal submission (the source instructions!). > 4. RemoteStrategy is bug prone with traversal inspection and > RemoteStep, etc. > > In TP4, we are now going to assume that Bytecode (a traversal) is always > submitted somewhere and this “somewhere" could be local or remote. This > “somewhere” must implement the Machine interface. > > > https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/Machine.java > > Machine makes explicit the TP4 communication protocol. The only objects > being transmitted are either Bytecode or Traversers. Simple. > > Here is an example using LocalMachine: > > Machine machine = LocalMachine.open(); > TraversalSource g = > Gremlin.traversal(machine).withProcessor(…).withStructure(…).withStrategy(…) > > The first time a traversal is generated from g, the Bytecode source > instructions are registered with the machine. > > > https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/language/gremlin/src/main/java/org/apache/tinkerpop/language/gremlin/TraversalSource.java#L99-L104 > < > https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/language/gremlin/src/main/java/org/apache/tinkerpop/language/gremlin/TraversalSource.java#L99-L104 > > > > The intention is that, on registration, the Machine will pre-compile the > source instructions (sort strategies, ensure processor and structure > setup/connectivity). Machine.register() returns a new Bytecode which > contains the registration information for future lookup. This registration > information is Machine-specific and can even be nothing! > > > https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/BasicMachine.java#L37-L39 > < > https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/BasicMachine.java#L37-L39 > > > > However, more intelligently, LocalMachine maintains a > Map<UUID,SourceCompilation> which maintains pre-compiled source > instructions. > > > https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/LocalMachine.java#L47-L62 > < > https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/LocalMachine.java#L47-L62 > > > > Now when bytecode (containing instructions for execution) is submitted to > LocalMachine, it will lookup the registered UUID and if it exists, use the > pre-compiled source code. As you can see, a pre-compilation has everything > staged and ready for use. > > > https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/bytecode/compiler/SourceCompilation.java > < > https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/bytecode/compiler/SourceCompilation.java > > > > For remote execution, we simply need RemoteMachine which would serialize > and deserialize Bytecode and Traversers to some RemoteMachineServer (or > some provider-specific server able to use the basic protocol we will > develop). For instance: > > Machine machine = RemoteMachine.open(Map.of(“ip”,”127.0.0.1”,”port”,”32”)) > TraversalSource g = > Gremlin.traversal(machine).withProcessor(…).withStructure(…).withStrategy(…) > > // prior to V(), the bytecode is registered and a new “registration” > bytecode is returned and appended with V and count instructions. > g.V().count() > > // no registration occurs as the TraversalSource hasn’t changed, the > bytecode is simply submitted. > g.V().out().count() > > // the remote registration is removed > g.close() > > // a new registration occurs > g = g.withStrategy(…) > g.V().drop() > > // the remote registration is removed > g.close() > > Tada! > > WDYT?, > Marko. > > http://rredux.com <http://rredux.com/> > > > > >
