So much. withProcessor(Processor). No more Compute. Process.submit(Graph) as we are now staging it so that every Processor (GraphComputer/GraphAgents) can work over any Graph. This will all be via Partitioner. withComputer() deprecated. Lots of cool stuff with Process strategies -- ProcessorTraveralStrategy like VertexProgramStratgegy and ActorProgramStrategy work directly with TraversalStrategies.GlobalCache. So much other stuf... I forget. Check the CHANGELOG. This was a massive undertaking but, thank god, its all backwards compatible (though with deprecation).
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/67849635 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/67849635 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/67849635 Branch: refs/heads/TINKERPOP-1564 Commit: 678496354bbb828c14f8d504da730ff206a1bace Parents: 3bf924b Author: Marko A. Rodriguez <okramma...@gmail.com> Authored: Fri Dec 16 04:43:50 2016 -0700 Committer: Marko A. Rodriguez <okramma...@gmail.com> Committed: Thu Jan 19 13:01:41 2017 -0700 ---------------------------------------------------------------------- CHANGELOG.asciidoc | 11 ++ .../akka/process/actor/AkkaGraphActors.java | 49 +++++-- .../akka/process/AkkaActorsProvider.java | 7 +- .../gremlin/akka/process/AkkaPlayTest.java | 5 +- .../process/computer/GiraphGraphComputer.java | 19 +++ .../tinkerpop/gremlin/process/Processor.java | 28 ++-- .../tinkerpop/gremlin/process/actor/Actors.java | 80 ----------- .../gremlin/process/actor/GraphActors.java | 45 ++++-- .../step/map/TraversalActorProgramStep.java | 26 ++-- .../decoration/ActorProgramStrategy.java | 80 ++--------- .../gremlin/process/computer/Computer.java | 61 ++++---- .../gremlin/process/computer/GraphComputer.java | 51 ++++++- .../decoration/VertexProgramStrategy.java | 141 +++++++++---------- .../process/traversal/TraversalSource.java | 31 +++- .../process/traversal/TraversalStrategies.java | 48 ++++--- .../dsl/graph/GraphTraversalSource.java | 2 +- .../strategy/ProcessorTraversalStrategy.java | 44 ++++++ .../gremlin/structure/util/StringFactory.java | 9 +- .../process/TraversalStrategiesTest.java | 5 + .../gremlin/process/traversal/BytecodeTest.java | 19 ++- .../jython/gremlin_python/process/strategies.py | 12 +- .../process/computer/GraphComputerTest.java | 13 +- .../traversal/step/map/PeerPressureTest.java | 2 + .../groovy/plugin/HadoopRemoteAcceptor.java | 5 +- .../hadoop/jsr223/HadoopRemoteAcceptor.java | 5 +- .../computer/AbstractHadoopGraphComputer.java | 11 +- .../process/computer/SparkGraphComputer.java | 17 +++ .../process/computer/TinkerGraphComputer.java | 35 +++++ .../process/TinkerGraphComputerProvider.java | 26 ++-- ...erGraphGroovyTranslatorComputerProvider.java | 2 +- 30 files changed, 513 insertions(+), 376 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/CHANGELOG.asciidoc ---------------------------------------------------------------------- diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index f0d1c4f..510dc83 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -27,6 +27,17 @@ TinkerPop 3.3.0 (Release Date: NOT OFFICIALLY RELEASED YET) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * Added more specific typing to various `__` traversal steps. E.g. `<A,Vertex>out()` is `<Vertex,Vertex>out()`. +* Added `Partitioner` which provides information about how a `Graph` is partitioned. +* Added `Graph.partitioner()` to access the graph's `Partitioner`. +* Added `Partition` which is accessible from `Partitioner` and provides `Vertex` and `Edge` iterators for partitions of the `Graph`. +* Added `GraphActors` which is an asynchronous, partition-centric, distributed, message passing graph processing framework. +* Added `akka-gremlin/` module which is an Akka implementation of `GraphActors`. +* Added `Processor` interface and both `GraphActors` and `GraphComputer` extend it. +* Deprecated `TraversalSource.withComputer()` in favor of `TraversalSource.withProcessor()`. +* Added `ProcessTraversalStrategy` which is used to get cached strategies associated with a `Processor`. +* Deprecated `Computer` in favor of `GraphComputer.open()`. +* Deprecated `Graph.compute()` and `GraphComputer.submit()` in favor of `GraphComputer.submit(Graph)`. +>>>>>>> So much. withProcessor(Processor). No more Compute. Process.submit(Graph) as we are now staging it so that every Processor (GraphComputer/GraphAgents) can work over any Graph. This will all be via Partitioner. withComputer() deprecated. Lots of cool stuff with Process strategies -- ProcessorTraveralStrategy like VertexProgramStratgegy and ActorProgramStrategy work directly with TraversalStrategies.GlobalCache. So much other stuf... I forget. Check the CHANGELOG. This was a massive undertaking but, thank god, its all backwards compatible (though with deprecation). * Updated Docker build scripts to include Python dependencies (NOTE: users should remove any previously generated TinkerPop Docker images). * Added "attachment requisite" `VertexProperty.element()` and `Property.element()` data in GraphSON serialization. * Added `Vertex`, `Edge`, `VertexProperty`, and `Property` serializers to Gremlin-Python and exposed tests that use graph object arguments. http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/akka-gremlin/src/main/java/org/apache/tinkerpop/gremlin/akka/process/actor/AkkaGraphActors.java ---------------------------------------------------------------------- diff --git a/akka-gremlin/src/main/java/org/apache/tinkerpop/gremlin/akka/process/actor/AkkaGraphActors.java b/akka-gremlin/src/main/java/org/apache/tinkerpop/gremlin/akka/process/actor/AkkaGraphActors.java index 5739369..c602dae 100644 --- a/akka-gremlin/src/main/java/org/apache/tinkerpop/gremlin/akka/process/actor/AkkaGraphActors.java +++ b/akka-gremlin/src/main/java/org/apache/tinkerpop/gremlin/akka/process/actor/AkkaGraphActors.java @@ -24,16 +24,21 @@ import akka.actor.Props; import com.typesafe.config.Config; import com.typesafe.config.ConfigFactory; import com.typesafe.config.ConfigValueFactory; +import org.apache.commons.configuration.BaseConfiguration; +import org.apache.commons.configuration.Configuration; import org.apache.tinkerpop.gremlin.process.actor.ActorProgram; import org.apache.tinkerpop.gremlin.process.actor.ActorsResult; import org.apache.tinkerpop.gremlin.process.actor.Address; import org.apache.tinkerpop.gremlin.process.actor.GraphActors; import org.apache.tinkerpop.gremlin.process.actor.util.DefaultActorsResult; +import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.Partitioner; import org.apache.tinkerpop.gremlin.structure.util.StringFactory; +import org.apache.tinkerpop.gremlin.structure.util.partitioner.HashPartitioner; import java.net.InetAddress; import java.net.UnknownHostException; +import java.util.Collections; import java.util.UUID; import java.util.concurrent.CompletableFuture; import java.util.concurrent.Future; @@ -45,11 +50,13 @@ import java.util.stream.Collectors; public final class AkkaGraphActors<R> implements GraphActors<R> { private ActorProgram<R> actorProgram; - private Partitioner partitioner; + private int workers = 1; + private Configuration configuration; private boolean executed = false; - public AkkaGraphActors() { - + private AkkaGraphActors(final Configuration configuration) { + this.configuration = configuration; + this.configuration.setProperty(GRAPH_ACTORS, AkkaGraphActors.class.getCanonicalName()); } @Override @@ -64,25 +71,28 @@ public final class AkkaGraphActors<R> implements GraphActors<R> { } @Override - public GraphActors<R> partitioner(final Partitioner partitioner) { - this.partitioner = partitioner; + public GraphActors<R> workers(final int workers) { + this.workers = workers; + this.configuration.setProperty(GRAPH_ACTORS_WORKERS, workers); return this; } @Override - public Future<R> submit() { + public Future<R> submit(final Graph graph) { if (this.executed) throw new IllegalStateException("Can not execute twice"); this.executed = true; - final ActorSystem system; + final Config config = ConfigFactory.defaultApplication().withValue("message-priorities", + ConfigValueFactory.fromAnyRef(this.actorProgram.getMessagePriorities(). + orElse(Collections.singletonList(Object.class)). + stream(). + map(Class::getCanonicalName). + collect(Collectors.toList()).toString())); + final ActorSystem system = ActorSystem.create("traversal-" + UUID.randomUUID(), config); final ActorsResult<R> result = new DefaultActorsResult<>(); - - final Config config = ConfigFactory.defaultApplication(). - withValue("message-priorities", - ConfigValueFactory.fromAnyRef(actorProgram.getMessagePriorities().get().stream().map(Class::getCanonicalName).collect(Collectors.toList()).toString())); - system = ActorSystem.create("traversal-" + UUID.randomUUID(), config); + final Partitioner partitioner = this.workers == 1 ? graph.partitioner() : new HashPartitioner(graph.partitioner(), this.workers); try { - new Address.Master(system.actorOf(Props.create(MasterActor.class, actorProgram, partitioner, result), "master").path().toString(), InetAddress.getLocalHost()); + new Address.Master(system.actorOf(Props.create(MasterActor.class, this.actorProgram, partitioner, result), "master").path().toString(), InetAddress.getLocalHost()); } catch (final UnknownHostException e) { throw new IllegalStateException(e.getMessage(), e); } @@ -93,5 +103,18 @@ public final class AkkaGraphActors<R> implements GraphActors<R> { return result.getResult(); }); } + + @Override + public Configuration configuration() { + return this.configuration; + } + + public static AkkaGraphActors open(final Configuration configuration) { + return new AkkaGraphActors(configuration); + } + + public static AkkaGraphActors open() { + return new AkkaGraphActors(new BaseConfiguration()); + } } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/akka-gremlin/src/test/java/org/apache/tinkerpop/gremlin/akka/process/AkkaActorsProvider.java ---------------------------------------------------------------------- diff --git a/akka-gremlin/src/test/java/org/apache/tinkerpop/gremlin/akka/process/AkkaActorsProvider.java b/akka-gremlin/src/test/java/org/apache/tinkerpop/gremlin/akka/process/AkkaActorsProvider.java index 3e9f5df..968fb99 100644 --- a/akka-gremlin/src/test/java/org/apache/tinkerpop/gremlin/akka/process/AkkaActorsProvider.java +++ b/akka-gremlin/src/test/java/org/apache/tinkerpop/gremlin/akka/process/AkkaActorsProvider.java @@ -23,7 +23,7 @@ import org.apache.commons.configuration.Configuration; import org.apache.tinkerpop.gremlin.AbstractGraphProvider; import org.apache.tinkerpop.gremlin.LoadGraphWith; import org.apache.tinkerpop.gremlin.akka.process.actor.AkkaGraphActors; -import org.apache.tinkerpop.gremlin.process.actor.Actors; +import org.apache.tinkerpop.gremlin.process.actor.GraphActors; import org.apache.tinkerpop.gremlin.process.traversal.TraversalInterruptionTest; import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; import org.apache.tinkerpop.gremlin.process.traversal.step.ComplexTest; @@ -39,7 +39,6 @@ import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.Partit import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.SubgraphStrategyProcessTest; import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.VertexProperty; -import org.apache.tinkerpop.gremlin.structure.util.partitioner.HashPartitioner; import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerEdge; import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerElement; import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph; @@ -147,8 +146,8 @@ public class AkkaActorsProvider extends AbstractGraphProvider { else { final GraphTraversalSource g = graph.traversal(); return RANDOM.nextBoolean() ? - g.withProcessor(Actors.of(AkkaGraphActors.class).workers(new Random().nextInt(15) + 1)) : - g.withProcessor(Actors.of(AkkaGraphActors.class)); + g.withProcessor(AkkaGraphActors.open().workers(new Random().nextInt(15) + 1)) : + g.withProcessor(GraphActors.open(AkkaGraphActors.class)); } } } \ No newline at end of file http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/akka-gremlin/src/test/java/org/apache/tinkerpop/gremlin/akka/process/AkkaPlayTest.java ---------------------------------------------------------------------- diff --git a/akka-gremlin/src/test/java/org/apache/tinkerpop/gremlin/akka/process/AkkaPlayTest.java b/akka-gremlin/src/test/java/org/apache/tinkerpop/gremlin/akka/process/AkkaPlayTest.java index 7de8304..df40748 100644 --- a/akka-gremlin/src/test/java/org/apache/tinkerpop/gremlin/akka/process/AkkaPlayTest.java +++ b/akka-gremlin/src/test/java/org/apache/tinkerpop/gremlin/akka/process/AkkaPlayTest.java @@ -20,11 +20,10 @@ package org.apache.tinkerpop.gremlin.akka.process; import org.apache.tinkerpop.gremlin.akka.process.actor.AkkaGraphActors; -import org.apache.tinkerpop.gremlin.process.actor.Actors; +import org.apache.tinkerpop.gremlin.process.actor.GraphActors; import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoIo; -import org.apache.tinkerpop.gremlin.structure.util.partitioner.HashPartitioner; import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph; import org.junit.Ignore; import org.junit.Test; @@ -42,7 +41,7 @@ public class AkkaPlayTest { public void testPlay1() throws Exception { final Graph graph = TinkerGraph.open(); graph.io(GryoIo.build()).readGraph("../data/tinkerpop-modern.kryo"); - GraphTraversalSource g = graph.traversal().withProcessor(Actors.of(AkkaGraphActors.class).workers(3)); + GraphTraversalSource g = graph.traversal().withProcessor(GraphActors.open(AkkaGraphActors.class).workers(3)); // System.out.println(g.V().group().by("name").by(outE().values("weight").fold()).toList()); for (int i = 0; i < 1000; i++) { http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/giraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/giraph/process/computer/GiraphGraphComputer.java ---------------------------------------------------------------------- diff --git a/giraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/giraph/process/computer/GiraphGraphComputer.java b/giraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/giraph/process/computer/GiraphGraphComputer.java index 1be548a..3047ee4 100644 --- a/giraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/giraph/process/computer/GiraphGraphComputer.java +++ b/giraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/giraph/process/computer/GiraphGraphComputer.java @@ -20,6 +20,7 @@ package org.apache.tinkerpop.gremlin.giraph.process.computer; import org.apache.commons.configuration.BaseConfiguration; import org.apache.commons.configuration.Configuration; +import org.apache.commons.configuration.ConfigurationUtils; import org.apache.commons.configuration.FileConfiguration; import org.apache.commons.configuration.PropertiesConfiguration; import org.apache.giraph.conf.GiraphConfiguration; @@ -57,6 +58,7 @@ import org.apache.tinkerpop.gremlin.process.computer.MemoryComputeKey; import org.apache.tinkerpop.gremlin.process.computer.VertexProgram; import org.apache.tinkerpop.gremlin.process.computer.util.DefaultComputerResult; import org.apache.tinkerpop.gremlin.process.computer.util.MapMemory; +import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.io.Storage; import org.apache.tinkerpop.gremlin.structure.io.gryo.kryoshim.KryoShimServiceLoader; import org.apache.tinkerpop.gremlin.util.Gremlin; @@ -99,6 +101,18 @@ public final class GiraphGraphComputer extends AbstractHadoopGraphComputer imple this.useWorkerThreadsInConfiguration = this.giraphConfiguration.getInt(GiraphConstants.MAX_WORKERS, -666) != -666 || this.giraphConfiguration.getInt(GiraphConstants.NUM_COMPUTE_THREADS.getKey(), -666) != -666; } + public static GiraphGraphComputer open(final org.apache.commons.configuration.Configuration configuration) { + return HadoopGraph.open(configuration).compute(GiraphGraphComputer.class); + } + + @Override + public Future<ComputerResult> submit(final Graph graph) { + this.hadoopGraph = (HadoopGraph)graph; + final Configuration configuration = this.hadoopGraph.configuration(); + configuration.getKeys().forEachRemaining(key -> this.giraphConfiguration.set(key, configuration.getProperty(key).toString())); + return this.submit(); + } + @Override public GraphComputer workers(final int workers) { this.useWorkerThreadsInConfiguration = false; @@ -131,6 +145,11 @@ public final class GiraphGraphComputer extends AbstractHadoopGraphComputer imple return ComputerSubmissionHelper.runWithBackgroundThread(this::submitWithExecutor, "GiraphSubmitter"); } + @Override + public org.apache.commons.configuration.Configuration configuration() { + return ConfUtil.makeApacheConfiguration(this.giraphConfiguration); + } + private Future<ComputerResult> submitWithExecutor(final Executor exec) { final long startTime = System.currentTimeMillis(); final Configuration apacheConfiguration = ConfUtil.makeApacheConfiguration(this.giraphConfiguration); http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/Processor.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/Processor.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/Processor.java index bffda58..78c544c 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/Processor.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/Processor.java @@ -19,9 +19,11 @@ package org.apache.tinkerpop.gremlin.process; -import org.apache.tinkerpop.gremlin.process.traversal.TraversalSource; +import org.apache.commons.configuration.Configuration; +import org.apache.tinkerpop.gremlin.process.traversal.strategy.ProcessorTraversalStrategy; +import org.apache.tinkerpop.gremlin.structure.Graph; -import java.io.Serializable; +import java.util.concurrent.Future; /** * This is a marker interface that denotes that the respective implementation is able to evaluate/execute/process a @@ -31,22 +33,10 @@ import java.io.Serializable; */ public interface Processor { - /** - * A {@link Processor} description provides the necessary configuration to create a {@link Processor}. - * This also entails {@link org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy} creation - * for a {@link TraversalSource}. - * - * @param <P> The type of {@link Processor} this description is used for. - */ - public static interface Description<P extends Processor> extends Cloneable, Serializable { - - /** - * Add respective {@link org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies} to the - * provided {@link TraversalSource}. - * - * @param traversalSource the traversal source to add processor-specific strategies to - */ - public TraversalSource addTraversalStrategies(final TraversalSource traversalSource); - } + public Configuration configuration(); + + public ProcessorTraversalStrategy<? extends Processor> getProcessorTraversalStrategy(); + + public Future submit(final Graph graph); } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/Actors.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/Actors.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/Actors.java deleted file mode 100644 index bba7674..0000000 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/Actors.java +++ /dev/null @@ -1,80 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ - -package org.apache.tinkerpop.gremlin.process.actor; - -import org.apache.tinkerpop.gremlin.process.Processor; -import org.apache.tinkerpop.gremlin.process.actor.traversal.strategy.decoration.ActorProgramStrategy; -import org.apache.tinkerpop.gremlin.process.traversal.TraversalSource; - -/** - * @author Marko A. Rodriguez (http://markorodriguez.com) - */ -public final class Actors implements Processor.Description<GraphActors> { - - private Class<? extends GraphActors> graphActorsClass; - private int workers = 1; - - private Actors(final Class<? extends GraphActors> graphActorsClass) { - this.graphActorsClass = graphActorsClass; - } - - public static Actors of(final Class<? extends GraphActors> graphActorsClass) { - return new Actors(graphActorsClass); - } - - public Actors graphActors(final Class<? extends GraphActors> graphActorsClass) { - final Actors clone = this.clone(); - clone.graphActorsClass = graphActorsClass; - return clone; - } - - public Actors workers(final int workers) { - final Actors clone = this.clone(); - clone.workers = workers; - return clone; - } - - public Class<? extends GraphActors> getGraphActorsClass() { - return this.graphActorsClass; - } - - public int getWorkers() { - return this.workers; - } - - - @Override - public String toString() { - return this.graphActorsClass.getSimpleName().toLowerCase(); - } - - public Actors clone() { - try { - return (Actors) super.clone(); - } catch (final CloneNotSupportedException e) { - throw new IllegalStateException(e.getMessage(), e); - } - } - - @Override - public TraversalSource addTraversalStrategies(final TraversalSource traversalSource) { - return traversalSource.withStrategies(new ActorProgramStrategy(this)); - } -} http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/GraphActors.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/GraphActors.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/GraphActors.java index 0cc2790..63804ab 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/GraphActors.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/GraphActors.java @@ -19,8 +19,13 @@ package org.apache.tinkerpop.gremlin.process.actor; +import org.apache.commons.configuration.BaseConfiguration; +import org.apache.commons.configuration.Configuration; import org.apache.tinkerpop.gremlin.process.Processor; -import org.apache.tinkerpop.gremlin.structure.Partitioner; +import org.apache.tinkerpop.gremlin.process.actor.traversal.strategy.decoration.ActorProgramStrategy; +import org.apache.tinkerpop.gremlin.process.traversal.Traversal; +import org.apache.tinkerpop.gremlin.process.traversal.strategy.ProcessorTraversalStrategy; +import org.apache.tinkerpop.gremlin.structure.Graph; import java.util.concurrent.Future; @@ -32,6 +37,9 @@ import java.util.concurrent.Future; */ public interface GraphActors<R> extends Processor { + public static final String GRAPH_ACTORS = "gremlin.graphActors"; + public static final String GRAPH_ACTORS_WORKERS = "gremlin.graphActors.workers"; + /** * Provide the {@link ActorProgram} that the GraphActors will execute. * @@ -41,19 +49,40 @@ public interface GraphActors<R> extends Processor { public GraphActors<R> program(final ActorProgram<R> program); /** - * Provide the {@link Partitioner} that the GraphActors will execute over. - * Typically, there will be a single {@link org.apache.tinkerpop.gremlin.process.actor.Actor.Worker} - * for each {@link org.apache.tinkerpop.gremlin.structure.Partition} in the partitioner. + * Specify the number of workers per {@link Graph} {@link org.apache.tinkerpop.gremlin.structure.Partition}. * - * @param partitioner the partitioner defining the data partitions - * @return the updated GraphActors with newly defined partitioner + * @param workers the number of workers per partition + * @return the updated GraphActors with newly defined workers */ - public GraphActors<R> partitioner(final Partitioner partitioner); + public GraphActors<R> workers(final int workers); /** * Submit the {@link ActorProgram} for execution by the {@link GraphActors}. * * @return a {@link Future} denoting a reference to the asynchronous computation's result */ - public Future<R> submit(); + public Future<R> submit(final Graph graph); + + /** + * Returns an {@link ActorProgramStrategy} which enables a {@link Traversal} to execute on {@link GraphActors}. + * + * @return a traversal strategy capable of executing traversals on a GraphActors + */ + public default ProcessorTraversalStrategy<GraphActors> getProcessorTraversalStrategy() { + return new ActorProgramStrategy(this); + } + + public static <A extends GraphActors> A open(final Configuration configuration) { + try { + return (A) Class.forName(configuration.getString(GRAPH_ACTORS)).getMethod("open", Configuration.class).invoke(null, configuration); + } catch (final Exception e) { + throw new IllegalArgumentException(e.getMessage(), e); + } + } + + public static <A extends GraphActors> A open(final Class<A> graphActorsClass) { + final BaseConfiguration configuration = new BaseConfiguration(); + configuration.setProperty(GRAPH_ACTORS, graphActorsClass.getCanonicalName()); + return GraphActors.open(configuration); + } } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/traversal/step/map/TraversalActorProgramStep.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/traversal/step/map/TraversalActorProgramStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/traversal/step/map/TraversalActorProgramStep.java index 5d643af..e4520aa 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/traversal/step/map/TraversalActorProgramStep.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/traversal/step/map/TraversalActorProgramStep.java @@ -19,6 +19,7 @@ package org.apache.tinkerpop.gremlin.process.actor.traversal.step.map; +import org.apache.commons.configuration.Configuration; import org.apache.tinkerpop.gremlin.process.actor.ActorProgram; import org.apache.tinkerpop.gremlin.process.actor.GraphActors; import org.apache.tinkerpop.gremlin.process.actor.traversal.TraversalActorProgram; @@ -27,7 +28,6 @@ import org.apache.tinkerpop.gremlin.process.traversal.Traverser; import org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep; import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep; import org.apache.tinkerpop.gremlin.process.traversal.traverser.util.TraverserSet; -import org.apache.tinkerpop.gremlin.structure.Partitioner; import org.apache.tinkerpop.gremlin.structure.util.StringFactory; import java.util.NoSuchElementException; @@ -37,22 +37,16 @@ import java.util.NoSuchElementException; */ public final class TraversalActorProgramStep<S, E> extends AbstractStep<E, E> { - private final Class<? extends GraphActors> actorsClass; + private final Traversal.Admin<S, E> actorsTraversal; - private final Partitioner partitioner; + private final Configuration graphActorsConfiguration; private boolean first = true; - public TraversalActorProgramStep(final Traversal.Admin<?, ?> traversal, final Class<? extends GraphActors> actorsClass, final Partitioner partitioner) { + public TraversalActorProgramStep(final Traversal.Admin<?, ?> traversal, final Configuration graphActorsConfiguration) { super(traversal); - this.actorsClass = actorsClass; + this.graphActorsConfiguration = graphActorsConfiguration; this.actorsTraversal = (Traversal.Admin) traversal.clone(); this.actorsTraversal.setParent(EmptyStep.instance()); - this.partitioner = partitioner; - } - - @Override - public String toString() { - return StringFactory.stepString(this, this.actorsTraversal); } @Override @@ -60,14 +54,20 @@ public final class TraversalActorProgramStep<S, E> extends AbstractStep<E, E> { if (this.first) { this.first = false; try { - final GraphActors<TraverserSet<E>> graphActors = this.actorsClass.newInstance(); + final GraphActors<TraverserSet<E>> graphActors = GraphActors.open(this.graphActorsConfiguration); final ActorProgram<TraverserSet<E>> actorProgram = new TraversalActorProgram<>(this.actorsTraversal); - graphActors.partitioner(this.partitioner).program(actorProgram).submit().get().forEach(this.starts::add); + graphActors.program(actorProgram).submit(this.getTraversal().getGraph().get()).get().forEach(this.starts::add); } catch (final Exception e) { throw new IllegalStateException(e.getMessage(), e); } } return this.starts.next(); } + + @Override + public String toString() { + return StringFactory.stepString(this, this.actorsTraversal); + } + } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/traversal/strategy/decoration/ActorProgramStrategy.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/traversal/strategy/decoration/ActorProgramStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/traversal/strategy/decoration/ActorProgramStrategy.java index 81bcda6..7e713de 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/traversal/strategy/decoration/ActorProgramStrategy.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/actor/traversal/strategy/decoration/ActorProgramStrategy.java @@ -20,8 +20,6 @@ package org.apache.tinkerpop.gremlin.process.actor.traversal.strategy.decoration; import org.apache.commons.configuration.Configuration; -import org.apache.commons.configuration.MapConfiguration; -import org.apache.tinkerpop.gremlin.process.actor.Actors; import org.apache.tinkerpop.gremlin.process.actor.GraphActors; import org.apache.tinkerpop.gremlin.process.actor.traversal.step.map.TraversalActorProgramStep; import org.apache.tinkerpop.gremlin.process.remote.traversal.strategy.decoration.RemoteStrategy; @@ -29,34 +27,25 @@ import org.apache.tinkerpop.gremlin.process.traversal.Traversal; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep; import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy; +import org.apache.tinkerpop.gremlin.process.traversal.strategy.ProcessorTraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.strategy.verification.ReadOnlyStrategy; import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper; -import org.apache.tinkerpop.gremlin.structure.util.empty.EmptyGraph; -import org.apache.tinkerpop.gremlin.structure.util.partitioner.HashPartitioner; -import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils; import java.util.Collections; -import java.util.HashMap; -import java.util.List; -import java.util.Map; import java.util.Set; /** * @author Marko A. Rodriguez (http://markorodriguez.com) */ public final class ActorProgramStrategy extends AbstractTraversalStrategy<TraversalStrategy.DecorationStrategy> - implements TraversalStrategy.DecorationStrategy { + implements TraversalStrategy.DecorationStrategy, ProcessorTraversalStrategy<GraphActors> { private static final Set<Class<? extends DecorationStrategy>> PRIORS = Collections.singleton(RemoteStrategy.class); - private final Actors actors; + private final Configuration graphActorsConfiguration; - private ActorProgramStrategy() { - this(null); - } - - public ActorProgramStrategy(final Actors actors) { - this.actors = actors; + public ActorProgramStrategy(final GraphActors graphActors) { + this.graphActorsConfiguration = graphActors.configuration(); } @Override @@ -66,10 +55,7 @@ public final class ActorProgramStrategy extends AbstractTraversalStrategy<Traver if (!(traversal.getParent() instanceof EmptyStep)) return; - final TraversalActorProgramStep<?, ?> actorStep = new TraversalActorProgramStep<>(traversal, this.actors.getGraphActorsClass(), - 1 == this.actors.getWorkers() ? - traversal.getGraph().orElse(EmptyGraph.instance()).partitioner() : - new HashPartitioner(traversal.getGraph().orElse(EmptyGraph.instance()).partitioner(), this.actors.getWorkers())); + final TraversalActorProgramStep<?, ?> actorStep = new TraversalActorProgramStep<>(traversal, this.graphActorsConfiguration); TraversalHelper.removeAllSteps(traversal); traversal.addStep(actorStep); @@ -86,64 +72,22 @@ public final class ActorProgramStrategy extends AbstractTraversalStrategy<Traver //////////////////////////////////////////////////////////// - public static final String GRAPH_ACTORS = "graphActors"; - public static final String WORKERS = "workers"; - @Override public Configuration getConfiguration() { - final Map<String, Object> map = new HashMap<>(); - map.put(GRAPH_ACTORS, this.actors.getGraphActorsClass().getCanonicalName()); - map.put(WORKERS, this.actors.getWorkers()); - return new MapConfiguration(map); + return this.graphActorsConfiguration; } public static ActorProgramStrategy create(final Configuration configuration) { try { - final ActorProgramStrategy.Builder builder = ActorProgramStrategy.build(); - for (final String key : (List<String>) IteratorUtils.asList(configuration.getKeys())) { - if (key.equals(GRAPH_ACTORS)) - builder.graphComputer((Class) Class.forName(configuration.getString(key))); - else if (key.equals(WORKERS)) - builder.workers(configuration.getInt(key)); - else - throw new IllegalArgumentException("The provided key is unknown: " + key); - } - return builder.create(); - } catch (final ClassNotFoundException e) { + return new ActorProgramStrategy(GraphActors.open(configuration)); + } catch (final Exception e) { throw new IllegalArgumentException(e.getMessage(), e); } } - public static ActorProgramStrategy.Builder build() { - return new ActorProgramStrategy.Builder(); - } - - public final static class Builder { - - private Actors actors = Actors.of(GraphActors.class); - - private Builder() { - } - - public Builder computer(final Actors actors) { - this.actors = actors; - return this; - } - - public Builder graphComputer(final Class<? extends GraphActors> graphActorsClass) { - this.actors = this.actors.graphActors(graphActorsClass); - return this; - } - - - public Builder workers(final int workers) { - this.actors = this.actors.workers(workers); - return this; - } - - public ActorProgramStrategy create() { - return new ActorProgramStrategy(this.actors); - } + @Override + public GraphActors getProcessor() { + return GraphActors.open(this.graphActorsConfiguration); } } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/Computer.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/Computer.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/Computer.java index 96d5e7c..0e15ef7 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/Computer.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/Computer.java @@ -19,25 +19,31 @@ package org.apache.tinkerpop.gremlin.process.computer; -import org.apache.tinkerpop.gremlin.process.Processor; -import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration.VertexProgramStrategy; +import org.apache.commons.configuration.Configuration; +import org.apache.commons.configuration.MapConfiguration; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; -import org.apache.tinkerpop.gremlin.process.traversal.TraversalSource; -import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies; -import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; import org.apache.tinkerpop.gremlin.structure.Edge; import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.Vertex; +import java.io.Serializable; import java.util.HashMap; -import java.util.List; import java.util.Map; import java.util.function.Function; +import static org.apache.tinkerpop.gremlin.process.computer.GraphComputer.EDGES; +import static org.apache.tinkerpop.gremlin.process.computer.GraphComputer.GRAPH_COMPUTER; +import static org.apache.tinkerpop.gremlin.process.computer.GraphComputer.PERSIST; +import static org.apache.tinkerpop.gremlin.process.computer.GraphComputer.RESULT; +import static org.apache.tinkerpop.gremlin.process.computer.GraphComputer.VERTICES; +import static org.apache.tinkerpop.gremlin.process.computer.GraphComputer.WORKERS; + /** * @author Marko A. Rodriguez (http://markorodriguez.com) + * @deprecated As of release 3.3.0, replaced by use of {@link GraphComputer#open(Configuration)}. */ -public final class Computer implements Processor.Description<GraphComputer>, Function<Graph, GraphComputer> { +@Deprecated +public final class Computer implements Function<Graph, GraphComputer>, Serializable, Cloneable { private Class<? extends GraphComputer> graphComputerClass = GraphComputer.class; private Map<String, Object> configuration = new HashMap<>(); @@ -47,35 +53,18 @@ public final class Computer implements Processor.Description<GraphComputer>, Fun private Traversal<Vertex, Vertex> vertices = null; private Traversal<Vertex, Edge> edges = null; - private Computer(final Class<? extends GraphComputer> graphComputerClass) { - this.graphComputerClass = graphComputerClass; - } - private Computer() { } - public static Computer of() { - return new Computer(GraphComputer.class); - } - - public static Computer of(final Class<? extends GraphComputer> graphComputerClass) { - return new Computer(graphComputerClass); + private Computer(final Class<? extends GraphComputer> graphComputerClass) { + this.graphComputerClass = graphComputerClass; } - - /** - * @deprecated As of release 3.3.0, replaced by using {@link Computer#of()}. - */ - @Deprecated public static Computer compute() { return new Computer(GraphComputer.class); } - /** - * @deprecated As of release 3.3.0, replaced by using {@link Computer#of(Class)}. - */ - @Deprecated public static Computer compute(final Class<? extends GraphComputer> graphComputerClass) { return new Computer(graphComputerClass); } @@ -143,7 +132,7 @@ public final class Computer implements Processor.Description<GraphComputer>, Fun if (null != this.vertices) computer = computer.vertices(this.vertices); if (null != this.edges) - computer.edges(this.edges); + computer = computer.edges(this.edges); return computer; } @@ -167,9 +156,21 @@ public final class Computer implements Processor.Description<GraphComputer>, Fun } } - @Override - public TraversalSource addTraversalStrategies(final TraversalSource traversalSource) { - return traversalSource.withStrategies(new VertexProgramStrategy(this)); + public Configuration configuration() { + final Map<String, Object> map = new HashMap<>(); + map.put(GRAPH_COMPUTER, this.graphComputerClass.getCanonicalName()); + if (-1 != this.workers) + map.put(WORKERS, this.workers); + if (null != this.persist) + map.put(PERSIST, this.persist.name()); + if (null != this.resultGraph) + map.put(RESULT, this.resultGraph.name()); + if (null != this.vertices) + map.put(VERTICES, this.vertices); + if (null != this.edges) + map.put(EDGES, this.edges); + map.putAll(this.getConfiguration()); + return new MapConfiguration(map); } ///////////////// http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/GraphComputer.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/GraphComputer.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/GraphComputer.java index d63b9e8..25074fd 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/GraphComputer.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/GraphComputer.java @@ -18,9 +18,13 @@ */ package org.apache.tinkerpop.gremlin.process.computer; +import org.apache.commons.configuration.Configuration; import org.apache.tinkerpop.gremlin.process.Processor; +import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration.VertexProgramStrategy; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; +import org.apache.tinkerpop.gremlin.process.traversal.strategy.ProcessorTraversalStrategy; import org.apache.tinkerpop.gremlin.structure.Edge; +import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.Vertex; import java.util.concurrent.Future; @@ -35,6 +39,13 @@ import java.util.concurrent.Future; */ public interface GraphComputer extends Processor { + public static final String GRAPH_COMPUTER = "gremlin.graphComputer"; + public static final String WORKERS = "gremlin.graphComputer.workers"; + public static final String PERSIST = "gremlin.graphComputer.persist"; + public static final String RESULT = "gremlin.graphComputer.result"; + public static final String VERTICES = "gremlin.graphComputer.vertices"; + public static final String EDGES = "gremlin.graphComputer.edges"; + public enum ResultGraph { /** * When the computation is complete, the {@link org.apache.tinkerpop.gremlin.structure.Graph} in {@link ComputerResult} is the original graph that spawned the graph computer. @@ -147,12 +158,50 @@ public interface GraphComputer extends Processor { } /** + * Get the configuration associated with the {@link GraphComputer} + * + * @return the GraphComputer's configuration + */ + public Configuration configuration(); + + /** + * Returns a {@link VertexProgramStrategy} which enables a {@link Traversal} to execute on {@link GraphComputer}. + * + * @return a traversal strategy capable of executing traversals on a GraphComputer + */ + public default ProcessorTraversalStrategy<GraphComputer> getProcessorTraversalStrategy() { + return new VertexProgramStrategy(this); + } + + public static <A extends GraphComputer> A open(final Configuration configuration) { + try { + return (A) Class.forName(configuration.getString(GRAPH_COMPUTER)).getMethod("open", Configuration.class).invoke(null, configuration); + } catch (final Exception e) { + throw new IllegalArgumentException(e.getMessage(), e); + } + } + + /** * Submit the {@link VertexProgram} and the set of {@link MapReduce} jobs for execution by the {@link GraphComputer}. * - * @return a {@link Future} denoting a reference to the asynchronous computation and where to get the {@link org.apache.tinkerpop.gremlin.process.computer.util.DefaultComputerResult} when its is complete. + * @return a {@link Future} denoting a reference to the computational result + * @deprecated As of release 3.3.0, replaced by use of {@link GraphComputer#submit(Graph)}. */ + @Deprecated public Future<ComputerResult> submit(); + /** + * Submit the configured {@link GraphComputer} to the provided {@link Graph}. + * That is, execute the {@link VertexProgram} over the {@link Graph}. + * + * @param graph the graph to execute the vertex program over + * @return a {@link Future} denoting a reference to the computational result + */ + @Override + public default Future<ComputerResult> submit(final Graph graph) { + return this.submit(); + } + public default Features features() { return new Features() { }; http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/decoration/VertexProgramStrategy.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/decoration/VertexProgramStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/decoration/VertexProgramStrategy.java index ac2e75a..bd6892f 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/decoration/VertexProgramStrategy.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/decoration/VertexProgramStrategy.java @@ -19,7 +19,9 @@ package org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration; +import org.apache.commons.configuration.BaseConfiguration; import org.apache.commons.configuration.Configuration; +import org.apache.commons.configuration.ConfigurationConverter; import org.apache.commons.configuration.MapConfiguration; import org.apache.tinkerpop.gremlin.process.computer.Computer; import org.apache.tinkerpop.gremlin.process.computer.GraphComputer; @@ -30,41 +32,50 @@ import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.Traversa import org.apache.tinkerpop.gremlin.process.remote.traversal.strategy.decoration.RemoteStrategy; import org.apache.tinkerpop.gremlin.process.traversal.Step; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; -import org.apache.tinkerpop.gremlin.process.traversal.TraversalSource; -import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__; import org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep; import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep; import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy; +import org.apache.tinkerpop.gremlin.process.traversal.strategy.ProcessorTraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal; import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper; import org.apache.tinkerpop.gremlin.structure.Edge; +import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.Vertex; -import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils; import java.util.HashMap; import java.util.HashSet; -import java.util.List; import java.util.Map; -import java.util.Optional; import java.util.Set; +import static org.apache.tinkerpop.gremlin.process.computer.GraphComputer.GRAPH_COMPUTER; + /** * @author Marko A. Rodriguez (http://markorodriguez.com) */ -public final class VertexProgramStrategy extends AbstractTraversalStrategy<TraversalStrategy.DecorationStrategy> implements TraversalStrategy.DecorationStrategy { +public final class VertexProgramStrategy extends AbstractTraversalStrategy<TraversalStrategy.DecorationStrategy> implements TraversalStrategy.DecorationStrategy, ProcessorTraversalStrategy<GraphComputer> { - private static final VertexProgramStrategy INSTANCE = new VertexProgramStrategy(Computer.of()); + private static final VertexProgramStrategy INSTANCE = new VertexProgramStrategy(Computer.compute()); - private final Computer computer; + private final Map<String, Object> graphComputerConfiguration; private VertexProgramStrategy() { - this(null); + this(new BaseConfiguration()); + } + + private VertexProgramStrategy(final Configuration configuration) { + this.graphComputerConfiguration = new HashMap<>((Map) ConfigurationConverter.getMap(configuration)); + if (!this.graphComputerConfiguration.containsKey(GRAPH_COMPUTER)) + this.graphComputerConfiguration.put(GRAPH_COMPUTER, GraphComputer.class.getCanonicalName()); } public VertexProgramStrategy(final Computer computer) { - this.computer = computer; + this(computer.configuration()); + } + + public VertexProgramStrategy(final GraphComputer graphComputer) { + this(graphComputer.configuration()); } @Override @@ -133,7 +144,7 @@ public final class VertexProgramStrategy extends AbstractTraversalStrategy<Trave traversal.addStep(new ComputerResultStep<>(traversal)); } // all vertex computing steps needs the graph computer function - traversal.getSteps().stream().filter(step -> step instanceof VertexComputing).forEach(step -> ((VertexComputing) step).setComputer(this.computer)); + traversal.getSteps().stream().filter(step -> step instanceof VertexComputing).forEach(step -> ((VertexComputing) step).setComputer(this.createComputer())); } private static Step<?, ?> getFirstLegalOLAPStep(Step<?, ?> currentStep) { @@ -156,23 +167,29 @@ public final class VertexProgramStrategy extends AbstractTraversalStrategy<Trave return EmptyStep.instance(); } - public static Optional<Computer> getComputer(final TraversalStrategies strategies) { - final Optional<TraversalStrategy<?>> optional = strategies.toList().stream().filter(strategy -> strategy instanceof VertexProgramStrategy).findAny(); - return optional.isPresent() ? Optional.of(((VertexProgramStrategy) optional.get()).computer) : Optional.empty(); - } - - public void addGraphComputerStrategies(final TraversalSource traversalSource) { - Class<? extends GraphComputer> graphComputerClass; - if (this.computer.getGraphComputerClass().equals(GraphComputer.class)) { - try { - graphComputerClass = this.computer.apply(traversalSource.getGraph()).getClass(); - } catch (final Exception e) { - graphComputerClass = GraphComputer.class; + private final Computer createComputer() { + try { + Computer computer = Computer.compute(); + for (final String key : this.graphComputerConfiguration.keySet()) { + if (key.equals(GRAPH_COMPUTER)) + computer = computer.graphComputer((Class) Class.forName((String) this.graphComputerConfiguration.get(key))); + else if (key.equals(GraphComputer.WORKERS)) + computer = computer.workers((int) this.graphComputerConfiguration.get(key)); + else if (key.equals(GraphComputer.PERSIST)) + computer = computer.persist(GraphComputer.Persist.valueOf((String) this.graphComputerConfiguration.get(key))); + else if (key.equals(GraphComputer.RESULT)) + computer = computer.result(GraphComputer.ResultGraph.valueOf((String) this.graphComputerConfiguration.get(key))); + else if (key.equals(GraphComputer.VERTICES)) + computer = computer.vertices((Traversal) this.graphComputerConfiguration.get(key)); + else if (key.equals(GraphComputer.EDGES)) + computer = computer.edges((Traversal) this.graphComputerConfiguration.get(key)); + else + computer = computer.configure(key, this.graphComputerConfiguration.get(key)); } - } else - graphComputerClass = this.computer.getGraphComputerClass(); - final List<TraversalStrategy<?>> graphComputerStrategies = TraversalStrategies.GlobalCache.getStrategies(graphComputerClass).toList(); - traversalSource.getStrategies().addStrategies(graphComputerStrategies.toArray(new TraversalStrategy[graphComputerStrategies.size()])); + return computer; + } catch (final ClassNotFoundException e) { + throw new IllegalArgumentException(e.getMessage(), e); + } } public static VertexProgramStrategy instance() { @@ -181,60 +198,40 @@ public final class VertexProgramStrategy extends AbstractTraversalStrategy<Trave //////////////////////////////////////////////////////////// - public static final String GRAPH_COMPUTER = "graphComputer"; - public static final String WORKERS = "workers"; - public static final String PERSIST = "persist"; - public static final String RESULT = "result"; - public static final String VERTICES = "vertices"; - public static final String EDGES = "edges"; - @Override - public Configuration getConfiguration() { - final Map<String, Object> map = new HashMap<>(); - map.put(GRAPH_COMPUTER, this.computer.getGraphComputerClass().getCanonicalName()); - if (-1 != this.computer.getWorkers()) - map.put(WORKERS, this.computer.getWorkers()); - if (null != this.computer.getPersist()) - map.put(PERSIST, this.computer.getPersist().name()); - if (null != this.computer.getResultGraph()) - map.put(RESULT, this.computer.getResultGraph().name()); - if (null != this.computer.getVertices()) - map.put(VERTICES, this.computer.getVertices()); - if (null != this.computer.getEdges()) - map.put(EDGES, this.computer.getEdges()); - map.putAll(this.computer.getConfiguration()); - return new MapConfiguration(map); + public static VertexProgramStrategy create(final Configuration configuration) { + return new VertexProgramStrategy(configuration); } - public static VertexProgramStrategy create(final Configuration configuration) { - try { - final VertexProgramStrategy.Builder builder = VertexProgramStrategy.build(); - for (final String key : (List<String>) IteratorUtils.asList(configuration.getKeys())) { - if (key.equals(GRAPH_COMPUTER)) - builder.graphComputer((Class) Class.forName(configuration.getString(key))); - else if (key.equals(WORKERS)) - builder.workers(configuration.getInt(key)); - else if (key.equals(PERSIST)) - builder.persist(GraphComputer.Persist.valueOf(configuration.getString(key))); - else if (key.equals(RESULT)) - builder.result(GraphComputer.ResultGraph.valueOf(configuration.getString(key))); - else if (key.equals(VERTICES)) - builder.vertices((Traversal) configuration.getProperty(key)); - else if (key.equals(EDGES)) - builder.edges((Traversal) configuration.getProperty(key)); - else - builder.configure(key, configuration.getProperty(key)); - } - return builder.create(); - } catch (final ClassNotFoundException e) { - throw new IllegalArgumentException(e.getMessage(), e); - } + @Override + public Configuration getConfiguration() { + return new MapConfiguration(this.graphComputerConfiguration); } + @Deprecated public static Builder build() { return new Builder(); } + @Override + public GraphComputer getProcessor() { + if (GraphComputer.class.getCanonicalName().equals(this.graphComputerConfiguration.get(GRAPH_COMPUTER))) + throw new IllegalStateException("This is an artifact of using the older Computer builder model."); + return GraphComputer.open(new MapConfiguration(this.graphComputerConfiguration)); + } + + @Deprecated + public Class<? extends GraphComputer> getGraphComputerClassHistoric(final Graph graph) { + try { + final GraphComputer graphComputer = graph.compute(); + this.graphComputerConfiguration.put(GRAPH_COMPUTER, graphComputer.getClass().getCanonicalName()); + return graphComputer.getClass(); + } catch (final Exception e) { + return GraphComputer.class; + } + } + + @Deprecated public final static class Builder { private Computer computer = Computer.compute(); http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalSource.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalSource.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalSource.java index 20926fc..02894a3 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalSource.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalSource.java @@ -25,6 +25,7 @@ import org.apache.tinkerpop.gremlin.process.computer.Computer; import org.apache.tinkerpop.gremlin.process.computer.GraphComputer; import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration.VertexProgramStrategy; import org.apache.tinkerpop.gremlin.process.remote.RemoteConnection; +import org.apache.tinkerpop.gremlin.process.traversal.strategy.ProcessorTraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.SackStrategy; import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.SideEffectStrategy; import org.apache.tinkerpop.gremlin.structure.Graph; @@ -32,6 +33,7 @@ import org.apache.tinkerpop.gremlin.util.function.ConstantSupplier; import java.io.Serializable; import java.lang.reflect.Constructor; +import java.util.List; import java.util.Optional; import java.util.function.BinaryOperator; import java.util.function.Supplier; @@ -109,8 +111,17 @@ public interface TraversalSource extends Cloneable, AutoCloseable { clone.getStrategies().addStrategies(traversalStrategies); clone.getBytecode().addSource(TraversalSource.Symbols.withStrategies, traversalStrategies); for (final TraversalStrategy traversalStrategy : traversalStrategies) { - if (traversalStrategy instanceof VertexProgramStrategy) { - ((VertexProgramStrategy) traversalStrategy).addGraphComputerStrategies(clone); // TODO: this is not generalized + if (traversalStrategy instanceof ProcessorTraversalStrategy) { + List<TraversalStrategy<?>> processorStrategies; + try { + final Class<? extends Processor> processorClass = ((ProcessorTraversalStrategy<?>) traversalStrategy).getProcessor().getClass(); + processorStrategies = TraversalStrategies.GlobalCache.getStrategies(processorClass).toList(); + } catch (final Exception e) { + // this is an issue due to the historic Computer way in which VertexProgramStrategies were created + // when (deprecated) Computer goes away, this try/catch block can be removed + processorStrategies = TraversalStrategies.GlobalCache.getStrategies(((VertexProgramStrategy) traversalStrategy).getGraphComputerClassHistoric(clone.getGraph())).toList(); + } + clone.getStrategies().addStrategies(processorStrategies.toArray(new TraversalStrategy[processorStrategies.size()])); } } return clone; @@ -131,13 +142,13 @@ public interface TraversalSource extends Cloneable, AutoCloseable { } /** - * Define the type of {@link Processor} that will evaluate all subsequent {@link Traversal}s spawned from this source. + * Provide the {@link Processor} that will be used to evaluate all subsequent {@link Traversal}s spawned from this source. * * @param processor the description of the processor to use * @return a new traversal source with updated strategies */ - public default TraversalSource withProcessor(final Processor.Description processor) { - return processor.addTraversalStrategies(this.clone()); + public default TraversalSource withProcessor(final Processor processor) { + return this.withStrategies((TraversalStrategy) processor.getProcessorTraversalStrategy()); } /** @@ -159,7 +170,9 @@ public interface TraversalSource extends Cloneable, AutoCloseable { * * @param computer a builder to generate a graph computer from the graph * @return a new traversal source with updated strategies + * @deprecated As of release 3.3.0, replaced by {@link TraversalSource#withProcessor(Processor)}. */ + @Deprecated public default TraversalSource withComputer(final Computer computer) { return this.withStrategies(new VertexProgramStrategy(computer)); } @@ -170,9 +183,11 @@ public interface TraversalSource extends Cloneable, AutoCloseable { * * @param graphComputerClass the graph computer class * @return a new traversal source with updated strategies + * @deprecated As of release 3.3.0, replaced by {@link TraversalSource#withProcessor(Processor)}. */ + @Deprecated public default TraversalSource withComputer(final Class<? extends GraphComputer> graphComputerClass) { - return this.withStrategies(new VertexProgramStrategy(Computer.of(graphComputerClass))); + return this.withStrategies(new VertexProgramStrategy(Computer.compute(graphComputerClass))); } /** @@ -180,9 +195,11 @@ public interface TraversalSource extends Cloneable, AutoCloseable { * This adds a {@link VertexProgramStrategy} to the strategies. * * @return a new traversal source with updated strategies + * @deprecated As of release 3.3.0, replaced by {@link TraversalSource#withProcessor(Processor)}. */ + @Deprecated public default TraversalSource withComputer() { - return this.withStrategies(new VertexProgramStrategy(Computer.of())); + return this.withStrategies(new VertexProgramStrategy(Computer.compute())); } /** http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java index 015df70..96dae61 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/TraversalStrategies.java @@ -18,6 +18,7 @@ */ package org.apache.tinkerpop.gremlin.process.traversal; +import org.apache.tinkerpop.gremlin.process.actor.GraphActors; import org.apache.tinkerpop.gremlin.process.computer.GraphComputer; import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.GraphFilterStrategy; import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.ConnectiveStrategy; @@ -197,7 +198,8 @@ public interface TraversalStrategies extends Serializable, Cloneable { public static final class GlobalCache { private static final Map<Class<? extends Graph>, TraversalStrategies> GRAPH_CACHE = new HashMap<>(); - private static final Map<Class<? extends GraphComputer>, TraversalStrategies> GRAPH_COMPUTER_CACHE = new HashMap<>(); + private static final Map<Class<? extends GraphComputer>, TraversalStrategies> COMPUTER_CACHE = new HashMap<>(); + private static final Map<Class<? extends GraphActors>, TraversalStrategies> ACTORS_CACHE = new HashMap<>(); static { final TraversalStrategies graphStrategies = new DefaultTraversalStrategies(); @@ -225,37 +227,47 @@ public interface TraversalStrategies extends Serializable, Cloneable { OrderLimitStrategy.instance(), PathProcessorStrategy.instance(), ComputerVerificationStrategy.instance()); - GRAPH_COMPUTER_CACHE.put(GraphComputer.class, graphComputerStrategies); + COMPUTER_CACHE.put(GraphComputer.class, graphComputerStrategies); + + ///////////////////// + + final TraversalStrategies graphActorsStrategies = new DefaultTraversalStrategies(); + ACTORS_CACHE.put(GraphActors.class, graphActorsStrategies); } - public static void registerStrategies(final Class graphOrGraphComputerClass, final TraversalStrategies traversalStrategies) { - if (Graph.class.isAssignableFrom(graphOrGraphComputerClass)) - GRAPH_CACHE.put(graphOrGraphComputerClass, traversalStrategies); - else if (GraphComputer.class.isAssignableFrom(graphOrGraphComputerClass)) - GRAPH_COMPUTER_CACHE.put(graphOrGraphComputerClass, traversalStrategies); + public static void registerStrategies(final Class graphOrProcessorClass, final TraversalStrategies traversalStrategies) { + if (Graph.class.isAssignableFrom(graphOrProcessorClass)) + GRAPH_CACHE.put(graphOrProcessorClass, traversalStrategies); + else if (GraphComputer.class.isAssignableFrom(graphOrProcessorClass)) + COMPUTER_CACHE.put(graphOrProcessorClass, traversalStrategies); + else if (GraphActors.class.isAssignableFrom(graphOrProcessorClass)) + ACTORS_CACHE.put(graphOrProcessorClass, traversalStrategies); else - throw new IllegalArgumentException("The TraversalStrategies.GlobalCache only supports Graph and GraphComputer strategy caching: " + graphOrGraphComputerClass.getCanonicalName()); + throw new IllegalArgumentException("The TraversalStrategies.GlobalCache only supports Graph, GraphComputer, and GraphActors strategy caching: " + graphOrProcessorClass.getCanonicalName()); } - public static TraversalStrategies getStrategies(final Class graphOrGraphComputerClass) { + public static TraversalStrategies getStrategies(final Class graphOrProcessorClass) { try { // be sure to load the class so that its static{} traversal strategy registration component is loaded. // this is more important for GraphComputer classes as they are typically not instantiated prior to strategy usage like Graph classes. - final String graphComputerClassName = null != graphOrGraphComputerClass.getDeclaringClass() ? - graphOrGraphComputerClass.getCanonicalName().replace("." + graphOrGraphComputerClass.getSimpleName(), "$" + graphOrGraphComputerClass.getSimpleName()) : - graphOrGraphComputerClass.getCanonicalName(); + final String graphComputerClassName = null != graphOrProcessorClass.getDeclaringClass() ? + graphOrProcessorClass.getCanonicalName().replace("." + graphOrProcessorClass.getSimpleName(), "$" + graphOrProcessorClass.getSimpleName()) : + graphOrProcessorClass.getCanonicalName(); Class.forName(graphComputerClassName); } catch (final ClassNotFoundException e) { throw new IllegalStateException(e.getMessage(), e); } - if (Graph.class.isAssignableFrom(graphOrGraphComputerClass)) { - final TraversalStrategies traversalStrategies = GRAPH_CACHE.get(graphOrGraphComputerClass); + if (Graph.class.isAssignableFrom(graphOrProcessorClass)) { + final TraversalStrategies traversalStrategies = GRAPH_CACHE.get(graphOrProcessorClass); return null == traversalStrategies ? GRAPH_CACHE.get(Graph.class) : traversalStrategies; - } else if (GraphComputer.class.isAssignableFrom(graphOrGraphComputerClass)) { - final TraversalStrategies traversalStrategies = GRAPH_COMPUTER_CACHE.get(graphOrGraphComputerClass); - return null == traversalStrategies ? GRAPH_COMPUTER_CACHE.get(GraphComputer.class) : traversalStrategies; + } else if (GraphComputer.class.isAssignableFrom(graphOrProcessorClass)) { + final TraversalStrategies traversalStrategies = COMPUTER_CACHE.get(graphOrProcessorClass); + return null == traversalStrategies ? COMPUTER_CACHE.get(GraphComputer.class) : traversalStrategies; + } else if (GraphActors.class.isAssignableFrom(graphOrProcessorClass)) { + final TraversalStrategies traversalStrategies = ACTORS_CACHE.get(graphOrProcessorClass); + return null == traversalStrategies ? ACTORS_CACHE.get(GraphActors.class) : traversalStrategies; } else { - throw new IllegalArgumentException("The TraversalStrategies.GlobalCache only supports Graph and GraphComputer strategy caching: " + graphOrGraphComputerClass.getCanonicalName()); + throw new IllegalArgumentException("The TraversalStrategies.GlobalCache only supports Graph, GraphComputer, and GraphActors strategy caching: " + graphOrProcessorClass.getCanonicalName()); } } } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalSource.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalSource.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalSource.java index d6facb7..bd23c65 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalSource.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalSource.java @@ -134,7 +134,7 @@ public class GraphTraversalSource implements TraversalSource { } @Override - public GraphTraversalSource withProcessor(final Processor.Description processor) { + public GraphTraversalSource withProcessor(final Processor processor) { return (GraphTraversalSource) TraversalSource.super.withProcessor(processor); } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/ProcessorTraversalStrategy.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/ProcessorTraversalStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/ProcessorTraversalStrategy.java new file mode 100644 index 0000000..30e6caf --- /dev/null +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/ProcessorTraversalStrategy.java @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.tinkerpop.gremlin.process.traversal.strategy; + +import org.apache.tinkerpop.gremlin.process.Processor; +import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies; +import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; + +import java.util.Optional; + +/** + * @author Marko A. Rodriguez (http://markorodriguez.com) + */ +public interface ProcessorTraversalStrategy<P extends Processor> { + + public P getProcessor(); + + public static <P extends Processor> Optional<P> getProcessor(final TraversalStrategies strategies) { + for (final TraversalStrategy strategy : strategies.toList()) { + if (strategy instanceof ProcessorTraversalStrategy) + return Optional.of(((ProcessorTraversalStrategy<P>) strategy).getProcessor()); + } + return Optional.empty(); + } + + +} http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/StringFactory.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/StringFactory.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/StringFactory.java index 553ebe3..61d9551 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/StringFactory.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/StringFactory.java @@ -18,14 +18,13 @@ */ package org.apache.tinkerpop.gremlin.structure.util; +import org.apache.tinkerpop.gremlin.process.Processor; import org.apache.tinkerpop.gremlin.process.actor.GraphActors; -import org.apache.tinkerpop.gremlin.process.computer.Computer; import org.apache.tinkerpop.gremlin.process.computer.ComputerResult; import org.apache.tinkerpop.gremlin.process.computer.GraphComputer; import org.apache.tinkerpop.gremlin.process.computer.MapReduce; import org.apache.tinkerpop.gremlin.process.computer.Memory; import org.apache.tinkerpop.gremlin.process.computer.VertexProgram; -import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration.VertexProgramStrategy; import org.apache.tinkerpop.gremlin.process.traversal.Step; import org.apache.tinkerpop.gremlin.process.traversal.Translator; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; @@ -33,6 +32,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.TraversalSideEffects; import org.apache.tinkerpop.gremlin.process.traversal.TraversalSource; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; +import org.apache.tinkerpop.gremlin.process.traversal.strategy.ProcessorTraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalRing; import org.apache.tinkerpop.gremlin.structure.Edge; import org.apache.tinkerpop.gremlin.structure.Graph; @@ -49,7 +49,6 @@ import java.lang.reflect.Modifier; import java.util.Collection; import java.util.List; import java.util.Map; -import java.util.Optional; import java.util.function.Function; import java.util.function.Predicate; import java.util.stream.Collectors; @@ -156,8 +155,8 @@ public final class StringFactory { public static String traversalSourceString(final TraversalSource traversalSource) { final String graphString = traversalSource.getGraph().toString(); - final Optional<Computer> optional = VertexProgramStrategy.getComputer(traversalSource.getStrategies()); - return traversalSource.getClass().getSimpleName().toLowerCase() + L_BRACKET + graphString + COMMA_SPACE + (optional.isPresent() ? optional.get().toString() : "standard") + R_BRACKET; + final String processorString = ProcessorTraversalStrategy.getProcessor(traversalSource.getStrategies()).map(Processor::toString).orElse("standard"); + return traversalSource.getClass().getSimpleName().toLowerCase() + L_BRACKET + graphString + COMMA_SPACE + processorString + R_BRACKET; } public static String featureString(final Graph.Features features) { http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java index bb58a06..c8fcc33 100644 --- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java +++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java @@ -166,6 +166,11 @@ public class TraversalStrategiesTest { } @Override + public Configuration configuration() { + return new BaseConfiguration(); + } + + @Override public Future<ComputerResult> submit() { return new CompletableFuture<>(); } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/BytecodeTest.java ---------------------------------------------------------------------- diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/BytecodeTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/BytecodeTest.java index 62ef89b..27cde58 100644 --- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/BytecodeTest.java +++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/BytecodeTest.java @@ -20,8 +20,8 @@ package org.apache.tinkerpop.gremlin.process.traversal; import org.apache.tinkerpop.gremlin.process.computer.Computer; +import org.apache.tinkerpop.gremlin.process.computer.GraphComputer; import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration.VertexProgramStrategy; -import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal; import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__; import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.SubgraphStrategy; @@ -29,10 +29,9 @@ import org.apache.tinkerpop.gremlin.process.traversal.strategy.verification.Read import org.apache.tinkerpop.gremlin.structure.Column; import org.apache.tinkerpop.gremlin.structure.util.empty.EmptyGraph; import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils; +import org.junit.Ignore; import org.junit.Test; -import java.util.List; - import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNotEquals; @@ -105,23 +104,23 @@ public class BytecodeTest { public void shouldIncludeBindingsInNestedTraversals() { final Bindings b = Bindings.instance(); final GraphTraversalSource g = EmptyGraph.instance().traversal(); - final Bytecode bytecode = g.V().in(b.of("a","created")).where(__.out(b.of("b","knows")).has("age",b.of("c",P.gt(32))).map(__.values(b.of("d","name")))).asAdmin().getBytecode(); + final Bytecode bytecode = g.V().in(b.of("a", "created")).where(__.out(b.of("b", "knows")).has("age", b.of("c", P.gt(32))).map(__.values(b.of("d", "name")))).asAdmin().getBytecode(); assertEquals(4, bytecode.getBindings().size()); assertEquals("created", bytecode.getBindings().get("a")); assertEquals("knows", bytecode.getBindings().get("b")); assertEquals(P.gt(32), bytecode.getBindings().get("c")); assertEquals("name", bytecode.getBindings().get("d")); // - Bytecode.Binding binding = (Bytecode.Binding)(bytecode.getStepInstructions()).get(1).getArguments()[0]; + Bytecode.Binding binding = (Bytecode.Binding) (bytecode.getStepInstructions()).get(1).getArguments()[0]; assertEquals("a", binding.variable()); assertEquals("created", binding.value()); - binding = (Bytecode.Binding) (((Bytecode)(bytecode.getStepInstructions()).get(2).getArguments()[0]).getStepInstructions()).get(0).getArguments()[0]; + binding = (Bytecode.Binding) (((Bytecode) (bytecode.getStepInstructions()).get(2).getArguments()[0]).getStepInstructions()).get(0).getArguments()[0]; assertEquals("b", binding.variable()); assertEquals("knows", binding.value()); - binding = (Bytecode.Binding) (((Bytecode)(bytecode.getStepInstructions()).get(2).getArguments()[0]).getStepInstructions()).get(1).getArguments()[1]; + binding = (Bytecode.Binding) (((Bytecode) (bytecode.getStepInstructions()).get(2).getArguments()[0]).getStepInstructions()).get(1).getArguments()[1]; assertEquals("c", binding.variable()); assertEquals(P.gt(32), binding.value()); - binding = (Bytecode.Binding) (((Bytecode)(((Bytecode)(bytecode.getStepInstructions()).get(2).getArguments()[0]).getStepInstructions()).get(2).getArguments()[0]).getStepInstructions()).get(0).getArguments()[0]; + binding = (Bytecode.Binding) (((Bytecode) (((Bytecode) (bytecode.getStepInstructions()).get(2).getArguments()[0]).getStepInstructions()).get(2).getArguments()[0]).getStepInstructions()).get(0).getArguments()[0]; assertEquals("d", binding.variable()); assertEquals("name", binding.value()); } @@ -141,7 +140,7 @@ public class BytecodeTest { final GraphTraversalSource g = EmptyGraph.instance().traversal(); Bytecode bytecode = g.withComputer(Computer.compute().workers(10)).getBytecode(); assertEquals(VertexProgramStrategy.build().create(), bytecode.getSourceInstructions().get(0).getArguments()[0]); - assertEquals(VertexProgramStrategy.build().workers(10).create().getConfiguration().getInt(VertexProgramStrategy.WORKERS), - ((VertexProgramStrategy) bytecode.getSourceInstructions().iterator().next().getArguments()[0]).getConfiguration().getInt(VertexProgramStrategy.WORKERS)); + assertEquals(VertexProgramStrategy.build().workers(10).create().getConfiguration().getInt(GraphComputer.WORKERS), + ((VertexProgramStrategy) bytecode.getSourceInstructions().iterator().next().getArguments()[0]).getConfiguration().getInt(GraphComputer.WORKERS)); } } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/67849635/gremlin-python/src/main/jython/gremlin_python/process/strategies.py ---------------------------------------------------------------------- diff --git a/gremlin-python/src/main/jython/gremlin_python/process/strategies.py b/gremlin-python/src/main/jython/gremlin_python/process/strategies.py index 8eb7fbd..ed8e7db 100644 --- a/gremlin-python/src/main/jython/gremlin_python/process/strategies.py +++ b/gremlin-python/src/main/jython/gremlin_python/process/strategies.py @@ -74,17 +74,17 @@ class VertexProgramStrategy(TraversalStrategy): configuration=None): TraversalStrategy.__init__(self) if graph_computer is not None: - self.configuration["graphComputer"] = graph_computer + self.configuration["gremlin.graphComputer"] = graph_computer if workers is not None: - self.configuration["workers"] = workers + self.configuration["gremlin.graphComputer.workers"] = workers if persist is not None: - self.configuration["persist"] = persist + self.configuration["gremlin.graphComputer.persist"] = persist if result is not None: - self.configuration["result"] = result + self.configuration["gremlin.graphComputer.result"] = result if vertices is not None: - self.configuration["vertices"] = vertices + self.configuration["gremlin.graphComputer.vertices"] = vertices if edges is not None: - self.configuration["edges"] = edges + self.configuration["gremlin.graphComputer.edges"] = edges if configuration is not None: self.configuration.update(configuration)
