Hi,
Here is how the TP4 bytecode submission infrastructure is looking.
In TP3, TraversalSource maintained the “pre-compilation” of strategies,
database connectivity, etc. This was not smart for the following reasons:
1. It assumed the traversal would execute on the same machine that it
was created on.
2. We had to make an explicit distinction between local and remote
execution via RemoteStrategy.
3. RemoteStrategy passes an excessive amount of data over the wire on
each traversal submission (the source instructions!).
4. RemoteStrategy is bug prone with traversal inspection and
RemoteStep, etc.
In TP4, we are now going to assume that Bytecode (a traversal) is always
submitted somewhere and this “somewhere" could be local or remote. This
“somewhere” must implement the Machine interface.
https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/Machine.java
Machine makes explicit the TP4 communication protocol. The only objects being
transmitted are either Bytecode or Traversers. Simple.
Here is an example using LocalMachine:
Machine machine = LocalMachine.open();
TraversalSource g =
Gremlin.traversal(machine).withProcessor(…).withStructure(…).withStrategy(…)
The first time a traversal is generated from g, the Bytecode source
instructions are registered with the machine.
https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/language/gremlin/src/main/java/org/apache/tinkerpop/language/gremlin/TraversalSource.java#L99-L104
<https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/language/gremlin/src/main/java/org/apache/tinkerpop/language/gremlin/TraversalSource.java#L99-L104>
The intention is that, on registration, the Machine will pre-compile the source
instructions (sort strategies, ensure processor and structure
setup/connectivity). Machine.register() returns a new Bytecode which contains
the registration information for future lookup. This registration information
is Machine-specific and can even be nothing!
https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/BasicMachine.java#L37-L39
<https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/BasicMachine.java#L37-L39>
However, more intelligently, LocalMachine maintains a
Map<UUID,SourceCompilation> which maintains pre-compiled source instructions.
https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/LocalMachine.java#L47-L62
<https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/LocalMachine.java#L47-L62>
Now when bytecode (containing instructions for execution) is submitted to
LocalMachine, it will lookup the registered UUID and if it exists, use the
pre-compiled source code. As you can see, a pre-compilation has everything
staged and ready for use.
https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/bytecode/compiler/SourceCompilation.java
<https://github.com/apache/tinkerpop/blob/596caf3ab82f3b15c2c343af87be6d03f26d6d6e/java/machine/machine-core/src/main/java/org/apache/tinkerpop/machine/bytecode/compiler/SourceCompilation.java>
For remote execution, we simply need RemoteMachine which would serialize and
deserialize Bytecode and Traversers to some RemoteMachineServer (or some
provider-specific server able to use the basic protocol we will develop). For
instance:
Machine machine = RemoteMachine.open(Map.of(“ip”,”127.0.0.1”,”port”,”32”))
TraversalSource g =
Gremlin.traversal(machine).withProcessor(…).withStructure(…).withStrategy(…)
// prior to V(), the bytecode is registered and a new “registration” bytecode
is returned and appended with V and count instructions.
g.V().count()
// no registration occurs as the TraversalSource hasn’t changed, the bytecode
is simply submitted.
g.V().out().count()
// the remote registration is removed
g.close()
// a new registration occurs
g = g.withStrategy(…)
g.V().drop()
// the remote registration is removed
g.close()
Tada!
WDYT?,
Marko.
http://rredux.com <http://rredux.com/>
