svn commit: r1837268 - in /tinkerpop/site: img/logos/ibm-compose-janusgraph-logo.png index.html providers.html
Author: rdale Date: Thu Aug 2 02:38:24 2018 New Revision: 1837268 URL: http://svn.apache.org/viewvc?rev=1837268=rev Log: Deploy TinkerPop homepage Added: tinkerpop/site/img/logos/ibm-compose-janusgraph-logo.png (with props) Modified: tinkerpop/site/index.html tinkerpop/site/providers.html Added: tinkerpop/site/img/logos/ibm-compose-janusgraph-logo.png URL: http://svn.apache.org/viewvc/tinkerpop/site/img/logos/ibm-compose-janusgraph-logo.png?rev=1837268=auto == Binary file - no diff available. Propchange: tinkerpop/site/img/logos/ibm-compose-janusgraph-logo.png -- svn:mime-type = image/png Modified: tinkerpop/site/index.html URL: http://svn.apache.org/viewvc/tinkerpop/site/index.html?rev=1837268=1837267=1837268=diff == --- tinkerpop/site/index.html (original) +++ tinkerpop/site/index.html Thu Aug 2 02:38:24 2018 @@ -365,6 +365,7 @@ limitations under the License. https://grakn.ai/;>GRAKN.AI - Distributed OLTP/OLAP knowledge graph system. http://tinkerpop.apache.org/docs/current/reference/#sparkgraphcomputer;>Hadoop (Spark) - OLAP graph processor using Spark. https://github.com/rayokota/hgraphdb;>HGraphDB - OLTP graph database running on Apache HBase. +https://www.huaweicloud.com/en-us/product/ges.html;>Huawei Graph Engine Service - Fully-managed, distributed, at-scale graph query and analysis service that provides a visualized interactive analytics platform. https://console.ng.bluemix.net/catalog/services/ibm-graph/;>IBM Graph - OLTP graph database as a service. http://janusgraph.org/;>JanusGraph - Distributed OLTP and OLAP graph database with BerkeleyDB, Apache Cassandra and Apache HBase support. https://github.com/awslabs/dynamodb-janusgraph-storage-backend//;>JanusGraph (Amazon) - The Amazon DynamoDB Storage Backend for JanusGraph. Modified: tinkerpop/site/providers.html URL: http://svn.apache.org/viewvc/tinkerpop/site/providers.html?rev=1837268=1837267=1837268=diff == --- tinkerpop/site/providers.html (original) +++ tinkerpop/site/providers.html Thu Aug 2 02:38:24 2018 @@ -287,12 +287,12 @@ limitations under the License. -http://www.ibm.com/analytics/us/en/technology/cloud-data-services/graph/;> -http://www.ibm.com/analytics/us/en/technology/cloud-data-services/graph/;>IBM Graph is a scalable, easy-to-use, fully-managed graph database-as-a-service that is administered through IBM's Bluemix cloud platform. IBM Graph is available 24/7 and managed by a team of experts so developers can focus on building their application. Based on the Apache TinkerPop stack for building high-performance graph applications, IBM Graph provides users with a set of simplified HTTP API calls and the Gremlin graph traversal language. +https://compose.com/databases/janusgraph;> +IBM https://compose.com/databases/janusgraph;>Compose for JanusGraph provides a fully-managed, highly-available, and production-ready JanusGraph on AWS, GCP or IBM Cloud. Deployed in minutes, every JanusGraph deployment on Compose is built with highly available storage and graph engines. The JanusGraph Storage engine is a cluster of the Scylla database. As usage increases or application requirements change, users can vertically or horizontally scale the JanusGraph Engine and Storage to increase throughput or storage. http://janusgraph.org/;> - http://janusgraph.org/;>JanusGraph is an Apache2 licensed scalable, distributed graph database optimized for storing and querying graphs containing hundreds of billions of vertices and edges distributed across a multi-machine cluster. JanusGraph is a transactional database that can support thousands of concurrent users executing complex Gremlin traversals in real time. JanusGraph also provides an in-memory, compression-based OLAP processor as well as integrates with Apache TinkerPop's Spark OLAP processors. + http://janusgraph.org/;>JanusGraph is an Apache2 licensed scalable, distributed graph database optimized for storing and querying graphs containing hundreds of billions of vertices and edges distributed across a multi-machine cluster. JanusGraph is a transactional database that can support thousands of concurrent users executing complex Gremlin traversals in real time. JanusGraph also provides an in-memory, compression-based OLAP processor as well as integrates with Apache TinkerPop's Spark OLAP processors.
tinkerpop git commit: Added Huawei Graph Engine Service to listing - CTR
Repository: tinkerpop Updated Branches: refs/heads/master 69b6f96d7 -> 8f1fe6d8e Added Huawei Graph Engine Service to listing - CTR Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/8f1fe6d8 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/8f1fe6d8 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/8f1fe6d8 Branch: refs/heads/master Commit: 8f1fe6d8ebd753dc1fb708da53ba7a60354dbaf7 Parents: 69b6f96 Author: Robert Dale Authored: Wed Aug 1 22:33:57 2018 -0400 Committer: Robert Dale Committed: Wed Aug 1 22:33:57 2018 -0400 -- docs/site/home/index.html | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/8f1fe6d8/docs/site/home/index.html -- diff --git a/docs/site/home/index.html b/docs/site/home/index.html index 4ff692c..472e77a 100644 --- a/docs/site/home/index.html +++ b/docs/site/home/index.html @@ -224,6 +224,7 @@ limitations under the License. https://grakn.ai/;>GRAKN.AI - Distributed OLTP/OLAP knowledge graph system. http://tinkerpop.apache.org/docs/current/reference/#sparkgraphcomputer;>Hadoop (Spark) - OLAP graph processor using Spark. https://github.com/rayokota/hgraphdb;>HGraphDB - OLTP graph database running on Apache HBase. +https://www.huaweicloud.com/en-us/product/ges.html;>Huawei Graph Engine Service - Fully-managed, distributed, at-scale graph query and analysis service that provides a visualized interactive analytics platform. https://console.ng.bluemix.net/catalog/services/ibm-graph/;>IBM Graph - OLTP graph database as a service. http://janusgraph.org/;>JanusGraph - Distributed OLTP and OLAP graph database with BerkeleyDB, Apache Cassandra and Apache HBase support. https://github.com/awslabs/dynamodb-janusgraph-storage-backend//;>JanusGraph (Amazon) - The Amazon DynamoDB Storage Backend for JanusGraph.
tinkerpop git commit: TINKERPOP-2016 Bumped to Jackson 2.9.6
Repository: tinkerpop Updated Branches: refs/heads/TINKERPOP-2016 [created] 1829318f6 TINKERPOP-2016 Bumped to Jackson 2.9.6 Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/1829318f Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/1829318f Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/1829318f Branch: refs/heads/TINKERPOP-2016 Commit: 1829318f6bfce1bc360818c6a0c34492fa69ec9a Parents: 7fce137 Author: Robert Dale Authored: Wed Aug 1 20:02:14 2018 -0400 Committer: Robert Dale Committed: Wed Aug 1 20:02:14 2018 -0400 -- CHANGELOG.asciidoc | 1 + gremlin-shaded/pom.xml | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/1829318f/CHANGELOG.asciidoc -- diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index b6e85ce..8137b45 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -44,6 +44,7 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima * Added concrete configuration methods to `SparkGraphComputer` to make a more clear API for configuring it. * Fixed a bug in `TinkerGraphCountStrategy`, which didn't consider that certain map steps may not emit an element. * Fixed a bug in JavaScript GLV where DriverRemoteConnection close() method didn't returned a Promise instance. +* Bumped to Jackson 2.9.6. [[release-3-2-9]] === TinkerPop 3.2.9 (Release Date: May 8, 2018) http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/1829318f/gremlin-shaded/pom.xml -- diff --git a/gremlin-shaded/pom.xml b/gremlin-shaded/pom.xml index 192262d..3f41b30 100644 --- a/gremlin-shaded/pom.xml +++ b/gremlin-shaded/pom.xml @@ -49,8 +49,8 @@ limitations under the License. com.fasterxml.jackson.core jackson-databind - -2.9.4 + +2.9.6 true
[50/50] [abbrv] tinkerpop git commit: translated shortest path recipes into their respective OLAP version using the new shortestPath() step
translated shortest path recipes into their respective OLAP version using the new shortestPath() step Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/d8a3626f Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/d8a3626f Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/d8a3626f Branch: refs/heads/TINKERPOP-1990 Commit: d8a3626fd54d3e3bb2d883d090c9e289374591ea Parents: 362ed16 Author: Daniel Kuppitz Authored: Tue Jul 31 10:58:09 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 12:29:06 2018 -0700 -- docs/src/recipes/shortest-path.asciidoc | 63 .../Process/Traversal/GraphTraversal.cs | 2 +- 2 files changed, 64 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/d8a3626f/docs/src/recipes/shortest-path.asciidoc -- diff --git a/docs/src/recipes/shortest-path.asciidoc b/docs/src/recipes/shortest-path.asciidoc index 2e33055..e25d777 100644 --- a/docs/src/recipes/shortest-path.asciidoc +++ b/docs/src/recipes/shortest-path.asciidoc @@ -48,6 +48,17 @@ course, it is possible for there to be more than one path in the graph of the sa length three), but this example is not considering that. <2> It might be interesting to know the path lengths for all paths between vertex "1" and "5". <3> Alternatively, one might wish to do a path length distribution over all the paths. +<4> The preferred way to get a shortest path in OLAP is the `shortestPath()` step. + +The following code block demonstrates how the shortest path from `v[1]` to `v[5]` can be queried in OLAP, using the `shortestPath()` step. + +[gremlin-groovy,existing] + +g = g.withComputer() +g.V(1).shortestPath(). + with(ShortestPath.edges, Direction.OUT). + with(ShortestPath.target, hasId(5)) + The previous example defines the length of the path by the number of vertices in the path, but the "path" might also be measured by data within the graph itself. The following example use the same graph structure as the previous example, @@ -95,6 +106,17 @@ calculated cost. With some slight modifications given the use of `select` it bec the output. Note that the path with the lowest "cost" actually has a longer path length as determined by the graph structure. +The next code block demonstrates how the `shortestPath()` step can be used in OLAP to determine the shortest weighted path. + +[gremlin-groovy,existing] + +g = g.withComputer() +g.V(1).shortestPath(). + with(ShortestPath.edges, Direction.OUT). + with(ShortestPath.distance, 'weight'). + with(ShortestPath.target, hasId(5)) + + The following query illustrates how `select()` can be used to find all shortest weighted undirected paths in the modern toy graph. @@ -136,3 +158,44 @@ g.withSack(0.0).V().as("from"). <1> <7> Order the output by the start vertex id and then the end vertex id (for better readability). <8> Deduplicate vertex pairs (the shortest path from `v[1]` to `v[6]` is the same as the path from `v[6]` to `v[1]`). +Again, this can be translated into an OLAP query using the `shortestPath()` step. + +[gremlin-groovy,existing] + +result = g.withComputer().V(). + shortestPath(). +with(ShortestPath.distance, 'weight'). +with(ShortestPath.includeEdges, true). + filter(count(local).is(gt(1))). + group(). +by(project('from','to'). + by(limit(local, 1)). + by(tail(local, 1))). + unfold(). + order(). +by(select(keys).select('from').id()). +by(select(keys).select('to').id()).toList() + + +The obvious difference in the result is the absence of property values in the OLAP result. Since OLAP traversers are not +allowed to leave the local star graph, it's not possible to have the exact same result in an OLAP query. However, the determined +shortest paths can be passed back into the OLTP `GraphTraversalSource`, which can then be used to query the values. + +[gremlin-groovy,existing] + +g.withSideEffect('v', []).<1> + inject(result.toArray()).as('kv').select(values). + unfold(). + map(unfold().as('v_or_e'). + coalesce(V().where(eq('v_or_e')).store('v'), + select('v').tail(local, 1).bothE().where(eq('v_or_e'))). + values('name','weight'). + fold()). + group(). +by(select('kv').select(keys)).unfold(). + order(). +by(select(keys).select('from').id()). +by(select(keys).select('to').id()).toList() + + +<1> The side-effect `v` is used to keep track of the last processed vertex, hence it needs to be an order-preserving list. Without this explicit definition `v` would become a `BulkSet` which doesn't preserve the insert order.
[22/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Testing for GraphSON and IoRegistry configuration
TINKERPOP-1996 Testing for GraphSON and IoRegistry configuration Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/ae3f685a Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/ae3f685a Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/ae3f685a Branch: refs/heads/TINKERPOP-1990 Commit: ae3f685ad7af326c9f1282da296e9db49f94da03 Parents: 7f1bf17 Author: Stephen Mallette Authored: Fri Jul 20 11:55:09 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 11:55:09 2018 -0400 -- .../step/sideEffect/TinkerGraphIoStepTest.java | 23 1 file changed, 23 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ae3f685a/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/traversal/step/sideEffect/TinkerGraphIoStepTest.java -- diff --git a/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/traversal/step/sideEffect/TinkerGraphIoStepTest.java b/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/traversal/step/sideEffect/TinkerGraphIoStepTest.java index 06c4db8..8bab7da 100644 --- a/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/traversal/step/sideEffect/TinkerGraphIoStepTest.java +++ b/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/traversal/step/sideEffect/TinkerGraphIoStepTest.java @@ -72,4 +72,27 @@ public class TinkerGraphIoStepTest { assertEquals(1, emptyG.V().has("custom", new CustomId("a", uuid)).count().next().intValue()); } + +@Test +public void shouldWriteReadWithCustomIoRegistryGraphSON() throws Exception { +final UUID uuid = UUID.randomUUID(); +g.addV("person").property("name","stephen").property("custom", new CustomId("a", uuid)).iterate(); + +final File file = TestHelper.generateTempFile(TinkerGraphIoStepTest.class, "shouldWriteReadWithCustomIoRegistryGraphSON", ".json"); +g.io(file.getAbsolutePath()).with(IO.registry, CustomId.CustomIdIoRegistry.class.getName()).write().iterate(); + +final Graph emptyGraph = TinkerGraph.open(); +final GraphTraversalSource emptyG = emptyGraph.traversal(); + +try { +emptyG.io(file.getAbsolutePath()).read().iterate(); +fail("Can't read without a registry"); +} catch (Exception ignored) { +// do nothing +} + +emptyG.io(file.getAbsolutePath()).with(IO.registry, CustomId.CustomIdIoRegistry.instance()).read().iterate(); + +assertEquals(1, emptyG.V().has("custom", new CustomId("a", uuid)).count().next().intValue()); +} }
[43/50] [abbrv] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/69b6f96d Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/69b6f96d Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/69b6f96d Branch: refs/heads/TINKERPOP-1990 Commit: 69b6f96d7ac8aa706ce65dee436d7038b8b3eaac Parents: b183edd 8af7837 Author: Daniel Kuppitz Authored: Wed Aug 1 11:27:17 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 11:27:17 2018 -0700 -- docker/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --
[27/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Fixed up typos in docs
TINKERPOP-1996 Fixed up typos in docs Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/10478be0 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/10478be0 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/10478be0 Branch: refs/heads/TINKERPOP-1990 Commit: 10478be0c9cabb6b20723c3185464977767172bc Parents: 38dc70d Author: Stephen Mallette Authored: Fri Jul 27 07:10:41 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 27 07:10:41 2018 -0400 -- docs/src/reference/the-traversal.asciidoc | 6 +++--- docs/src/upgrade/release-3.4.x.asciidoc | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/10478be0/docs/src/reference/the-traversal.asciidoc -- diff --git a/docs/src/reference/the-traversal.asciidoc b/docs/src/reference/the-traversal.asciidoc index c861d87..d096933 100644 --- a/docs/src/reference/the-traversal.asciidoc +++ b/docs/src/reference/the-traversal.asciidoc @@ -1060,9 +1060,9 @@ job of the `io()`-step. By default, TinkerPop supports three formats for importi NOTE: Additional documentation for TinkerPop IO formats can be found in the link:http://tinkerpop.apache.org/docs/x.y.z/dev/io/[IO Reference]. -By itself the `io()` step merely configures the kind of importing and exporting that is going +By itself the `io()`-step merely configures the kind of importing and exporting that is going to occur and it is the follow-on call to the `read()` or `write()` step that determines which of those actions will -execute. Therefore, a typical usage of the `io()` step would look like this: +execute. Therefore, a typical usage of the `io()`-step would look like this: [source,java] @@ -1073,7 +1073,7 @@ g.io(someOutputFile).write().iterate() IMPORTANT: The commands above are still traversals and therefore require iteration to be executed, hence the use of `iterate()` as a termination step. -By default, the `io()` step will try to detect the right file format using the file name extension. To gain greater +By default, the `io()`-step will try to detect the right file format using the file name extension. To gain greater control of the format use the `with()` step modulator to provide further information to `io()`. For example: [source,java] http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/10478be0/docs/src/upgrade/release-3.4.x.asciidoc -- diff --git a/docs/src/upgrade/release-3.4.x.asciidoc b/docs/src/upgrade/release-3.4.x.asciidoc index 0bb3903..8d87397 100644 --- a/docs/src/upgrade/release-3.4.x.asciidoc +++ b/docs/src/upgrade/release-3.4.x.asciidoc @@ -81,7 +81,7 @@ g.io(someInputFile).read().iterate() g.io(someOutputFile).write().iterate() -While `io()` step is still single-threaded for OLTP style loading, it can be utilized in conjunction with OLAP which +While `io()`-step is still single-threaded for OLTP style loading, it can be utilized in conjunction with OLAP which internally uses `CloneVertexProgram` and therefore any graph `InputFormat` or `OutputFormat` can be configured in conjunction with this step for parallel loads of large datasets.
[44/50] [abbrv] tinkerpop git commit: hmmmm
h Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/101e5d8d Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/101e5d8d Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/101e5d8d Branch: refs/heads/TINKERPOP-1990 Commit: 101e5d8d6ebc2d818e73923d89ed1b3bebb53d3d Parents: eae0184 Author: Daniel Kuppitz Authored: Mon Jul 30 09:43:26 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 12:26:30 2018 -0700 -- .../org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/101e5d8d/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java index 200645e..4f41cb5 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java @@ -63,7 +63,7 @@ import java.util.Iterator; @Graph.OptOut( test = "org.apache.tinkerpop.gremlin.process.traversal.step.map.ShortestPathTest", method = "*", -reason = "h") +reason = "https://issues.apache.org/jira/browse/TINKERPOP-1976;) @Graph.OptOut( test = "org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.TranslationStrategyProcessTest", method = "*",
[28/50] [abbrv] tinkerpop git commit: Bumped default Xmx on Gremlin Server
Bumped default Xmx on Gremlin Server Seems like the default is too small for anything but the toy graphs and people getting started who aren't JVM experts keep running into problems CTR Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/2b045f30 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/2b045f30 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/2b045f30 Branch: refs/heads/TINKERPOP-1990 Commit: 2b045f30db0577464f1ff353464c7cf5d3c41746 Parents: 3b8c828 Author: Stephen Mallette Authored: Mon Jul 30 07:14:42 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 07:14:42 2018 -0400 -- gremlin-server/src/main/bin/gremlin-server.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/2b045f30/gremlin-server/src/main/bin/gremlin-server.sh -- diff --git a/gremlin-server/src/main/bin/gremlin-server.sh b/gremlin-server/src/main/bin/gremlin-server.sh index fcede2b..06b065a 100755 --- a/gremlin-server/src/main/bin/gremlin-server.sh +++ b/gremlin-server/src/main/bin/gremlin-server.sh @@ -50,7 +50,7 @@ fi # Set Java options if [ "$JAVA_OPTIONS" = "" ] ; then -JAVA_OPTIONS="-Xms32m -Xmx512m" +JAVA_OPTIONS="-Xms512m -Xmx4096m" fi # Execute the application and return its exit code
[23/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Fixed verification on io()
TINKERPOP-1996 Fixed verification on io() Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/23c71b60 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/23c71b60 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/23c71b60 Branch: refs/heads/TINKERPOP-1990 Commit: 23c71b60cc34c14e9defc6a9a7ab6c57c090460b Parents: ae3f685 Author: Stephen Mallette Authored: Fri Jul 20 13:22:47 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 13:22:47 2018 -0400 -- .../verification/StandardVerificationStrategy.java | 6 -- .../StandardVerificationStrategyTest.java | 16 +++- 2 files changed, 15 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/23c71b60/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategy.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategy.java index 258d345..7d88ed9 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategy.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategy.java @@ -84,8 +84,10 @@ public final class StandardVerificationStrategy extends AbstractTraversalStrateg throw new VerificationException("The profile()-Step cannot be specified multiple times.", traversal); } -if (traversal.getStartStep() instanceof ReadWriting && traversal.getSteps().size() != 1) { -if (TraversalHelper.getStepsOfClass(NoneStep.class, traversal).size() != traversal.getSteps().size() - 1) +if (traversal.getStartStep() instanceof ReadWriting && !endStep.equals(traversal.getStartStep())) { +final int total = TraversalHelper.getStepsOfClass(NoneStep.class, traversal).size() + +TraversalHelper.getStepsOfClass(RequirementsStep.class, traversal).size() + 1; +if (total != traversal.getSteps().size()) throw new VerificationException("The io() step must be the first and only step in the traversal - it cannot be used with other steps", traversal); } } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/23c71b60/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategyTest.java -- diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategyTest.java index deec002..96a68b2 100644 --- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategyTest.java +++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategyTest.java @@ -18,6 +18,7 @@ */ package org.apache.tinkerpop.gremlin.process.traversal.strategy.verification; +import org.apache.tinkerpop.gremlin.TestHelper; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies; import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__; @@ -29,6 +30,7 @@ import org.junit.Test; import org.junit.runner.RunWith; import org.junit.runners.Parameterized; +import java.io.File; import java.util.Arrays; import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.out; @@ -44,18 +46,22 @@ import static org.junit.Assert.fail; @RunWith(Parameterized.class) public class StandardVerificationStrategyTest { + @Parameterized.Parameters(name = "{0}") -public static Iterable data() { +public static Iterable data() throws Exception { + +final String file = TestHelper.generateTempFile(StandardVerificationStrategyTest.class, "shouldBeVerified", ".kryo").getAbsolutePath(); + return Arrays.asList(new Object[][]{ // traversals that should fail verification {"__.repeat(out().fold().unfold()).times(2)", repeat(out().fold().unfold()).times(2), false}, {"__.repeat(sum()).times(2)", repeat(sum()).times(2), false}, {"__.repeat(out().count())",
[21/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 No need to assert io() against VertexProgramStrategy
TINKERPOP-1996 No need to assert io() against VertexProgramStrategy Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/7f1bf178 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/7f1bf178 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/7f1bf178 Branch: refs/heads/TINKERPOP-1990 Commit: 7f1bf1783efb8a7eca17d0367af66c6289455fd8 Parents: e6e4413 Author: Stephen Mallette Authored: Fri Jul 20 11:52:13 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 11:52:13 2018 -0400 -- .../strategy/decoration/VertexProgramStrategyTest.java | 6 -- 1 file changed, 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7f1bf178/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/VertexProgramStrategyTest.java -- diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/VertexProgramStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/VertexProgramStrategyTest.java index d3bb6ef..8ceef48 100644 --- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/VertexProgramStrategyTest.java +++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/VertexProgramStrategyTest.java @@ -43,7 +43,6 @@ import static org.junit.Assert.assertEquals; /** * @author Marko A. Rodriguez (http://markorodriguez.com) - * @author Stephen Mallette (http://stephen.genoprime.com) */ @RunWith(Parameterized.class) public class VertexProgramStrategyTest { @@ -68,12 +67,7 @@ public class VertexProgramStrategyTest { public static Iterable generateTestParameters() { final ComputerResultStep computerResultStep = new ComputerResultStep(EmptyTraversal.instance()); - -// The tests for io() need to verify that there is no change i.e. we don't want the step getting wrapped up in -// traversalvertexprogramstep stuff or else it won't execute properly in OLAP return Arrays.asList(new Traversal[][]{ -{ EmptyGraph.instance().traversal().io("blah.json"), EmptyGraph.instance().traversal().io("blah.json")}, -{ EmptyGraph.instance().traversal().io("blah.json"), EmptyGraph.instance().traversal().io("blah.json")}, {__.V().out().count(), start().addStep(traversal(__.V().out().count())).addStep(computerResultStep)}, {__.V().pageRank().out().count(), start().pageRank().asAdmin().addStep(traversal(__.V().out().count())).addStep(computerResultStep)}, {__.V().out().pageRank(), start().addStep(traversal(__.V().out())).pageRank().asAdmin().addStep(traversal(__.identity())).addStep(computerResultStep)},
[48/50] [abbrv] tinkerpop git commit: removed wildcard imports
removed wildcard imports Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/62d0dbcc Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/62d0dbcc Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/62d0dbcc Branch: refs/heads/TINKERPOP-1990 Commit: 62d0dbcc373dd4f4fc411b7ab129f8b71275498b Parents: e1ba81b Author: Daniel Kuppitz Authored: Mon Jul 30 08:47:52 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 12:26:30 2018 -0700 -- .../step/map/ShortestPathVertexProgramStep.java | 16 1 file changed, 12 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/62d0dbcc/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ShortestPathVertexProgramStep.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ShortestPathVertexProgramStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ShortestPathVertexProgramStep.java index 692cb6a..5a0f338 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ShortestPathVertexProgramStep.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/ShortestPathVertexProgramStep.java @@ -19,11 +19,15 @@ package org.apache.tinkerpop.gremlin.process.computer.traversal.step.map; -import org.apache.tinkerpop.gremlin.process.computer.*; +import org.apache.tinkerpop.gremlin.process.computer.ComputerResult; +import org.apache.tinkerpop.gremlin.process.computer.GraphFilter; +import org.apache.tinkerpop.gremlin.process.computer.Memory; import org.apache.tinkerpop.gremlin.process.computer.search.path.ShortestPathVertexProgram; import org.apache.tinkerpop.gremlin.process.computer.traversal.TraversalVertexProgram; -import org.apache.tinkerpop.gremlin.process.traversal.*; -import org.apache.tinkerpop.gremlin.process.traversal.step.*; +import org.apache.tinkerpop.gremlin.process.traversal.Traversal; +import org.apache.tinkerpop.gremlin.process.traversal.Traverser; +import org.apache.tinkerpop.gremlin.process.traversal.step.Configuring; +import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent; import org.apache.tinkerpop.gremlin.process.traversal.step.util.Parameters; import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement; import org.apache.tinkerpop.gremlin.process.traversal.traverser.util.TraverserSet; @@ -35,7 +39,11 @@ import org.apache.tinkerpop.gremlin.structure.util.StringFactory; import org.apache.tinkerpop.gremlin.util.Serializer; import java.io.IOException; -import java.util.*; +import java.util.Arrays; +import java.util.Base64; +import java.util.List; +import java.util.NoSuchElementException; +import java.util.Set; /** * @author Daniel Kuppitz (http://gremlin.guru)
[26/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Fixed up typos in docs
TINKERPOP-1996 Fixed up typos in docs Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/38dc70df Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/38dc70df Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/38dc70df Branch: refs/heads/TINKERPOP-1990 Commit: 38dc70df6db105595784a93ecf5f5726201a962d Parents: fdb35c6 Author: Stephen Mallette Authored: Fri Jul 27 07:07:17 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 27 07:07:17 2018 -0400 -- docs/src/reference/the-traversal.asciidoc | 8 1 file changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/38dc70df/docs/src/reference/the-traversal.asciidoc -- diff --git a/docs/src/reference/the-traversal.asciidoc b/docs/src/reference/the-traversal.asciidoc index c3b9300..c861d87 100644 --- a/docs/src/reference/the-traversal.asciidoc +++ b/docs/src/reference/the-traversal.asciidoc @@ -1086,9 +1086,9 @@ g.io(someOutputFile). write().iterate() -The `IO` class is a helper for the `io()` step that provides expressions that can be used to help configure it +The `IO` class is a helper for the `io()`-step that provides expressions that can be used to help configure it and in this case it allows direct specification of the "reader" or "writer" to use. The "reader" actually refers to -a `GraphReader` implementation and the `writer" refers to a `GraphWriter` implementation. The implementations of +a `GraphReader` implementation and the "writer" refers to a `GraphWriter` implementation. The implementations of those interfaces provided by default are the standard TinkerPop implementations. That default is an important point to consider for users. The default TinkerPop implementations are not designed with @@ -1100,7 +1100,7 @@ writing perspective is not that different in there are no parallel operations in to disk requires a single pass of the data without high memory requirements for larger datasets. In general, TinkerPop recommends that users examine the native bulk import/export tools of the graph implementation -that they choose. Those tools will often outperform the `io()` step and perhaps be easier to use with a greater +that they choose. Those tools will often outperform the `io()`-step and perhaps be easier to use with a greater feature set. That said, graph providers do have the option to optimize `io()` to back it with their own import/export utilities and therefore the default behavior provided by TinkerPop described above might be overridden by the graph. @@ -,7 +,7 @@ functionality internally using <>. With t can be imported/exported assuming that there is a Hadoop `InputFormat` or `OutputFormat` to support it. IMPORTANT: Remote Gremlin Console users or Gremlin Language Variant (GLV) users (e.g. gremlin-python) who utilize -the `io()` step should recall that their `read()` or `write()` operation will occur on the server and not locally +the `io()`-step should recall that their `read()` or `write()` operation will occur on the server and not locally and therefore the file specified for import/export must be something accessible by the server. GraphSON and Gryo formats are extensible allowing users and graph providers to extend supported serialization options.
[40/50] [abbrv] tinkerpop git commit: This closes #892
This closes #892 Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/b183edd1 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/b183edd1 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/b183edd1 Branch: refs/heads/TINKERPOP-1990 Commit: b183edd10a52dc65022c0270951b5bd9f2b5be7e Parents: f50aeb6 Author: Stephen Mallette Authored: Tue Jul 31 15:54:01 2018 -0400 Committer: Stephen Mallette Committed: Tue Jul 31 15:54:01 2018 -0400 -- --
[31/50] [abbrv] tinkerpop git commit: Added a better reason for some OptOut on RemoteGraph CTR
Added a better reason for some OptOut on RemoteGraph CTR Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/00a085aa Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/00a085aa Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/00a085aa Branch: refs/heads/TINKERPOP-1990 Commit: 00a085aa15cfc877222712f8572c89f1c096597c Parents: 2b045f3 Author: Stephen Mallette Authored: Mon Jul 30 11:40:52 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 11:40:52 2018 -0400 -- .../org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/00a085aa/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java index 8a1ab02..4b23475 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java @@ -55,11 +55,11 @@ import java.util.Iterator; @Graph.OptOut( test = "org.apache.tinkerpop.gremlin.process.traversal.step.map.PeerPressureTest", method = "*", -reason = "h") +reason = "https://issues.apache.org/jira/browse/TINKERPOP-1976;) @Graph.OptOut( test = "org.apache.tinkerpop.gremlin.process.traversal.step.map.PageRankTest", method = "*", -reason = "h") +reason = "https://issues.apache.org/jira/browse/TINKERPOP-1976;) @Graph.OptOut( test = "org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.TranslationStrategyProcessTest", method = "*",
[18/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Undeprecated some Io related classes
TINKERPOP-1996 Undeprecated some Io related classes These classes still have use as part of IoRegistry which is still in use and I don't see a clear way to get rid of that easily. We'd have to change the whole system for serialization configuration to accomplish that so I guess this stuff stays for now. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/8fd3bf21 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/8fd3bf21 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/8fd3bf21 Branch: refs/heads/TINKERPOP-1990 Commit: 8fd3bf21efac092f619254245ba83614327c4dcb Parents: c97d747 Author: Stephen Mallette Authored: Fri Jul 20 10:47:20 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 10:47:20 2018 -0400 -- .../main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java | 3 --- .../apache/tinkerpop/gremlin/structure/io/graphml/GraphMLIo.java | 3 --- .../tinkerpop/gremlin/structure/io/graphson/GraphSONIo.java | 3 --- .../org/apache/tinkerpop/gremlin/structure/io/gryo/GryoIo.java| 3 --- 4 files changed, 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/8fd3bf21/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java index 0971e31..bae56c5 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java @@ -18,7 +18,6 @@ */ package org.apache.tinkerpop.gremlin.structure.io; -import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; import org.apache.tinkerpop.gremlin.structure.Graph; import java.io.IOException; @@ -32,9 +31,7 @@ import java.util.function.Consumer; * internal {@link Mapper} (if the format has such capability). * * @author Stephen Mallette (http://stephen.genoprime.com) - * @deprecated As of release 3.4.0, replaced by {@link GraphTraversalSource#io(String)}. */ -@Deprecated public interface Io { /** http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/8fd3bf21/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphml/GraphMLIo.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphml/GraphMLIo.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphml/GraphMLIo.java index b6af646..88431bf 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphml/GraphMLIo.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphml/GraphMLIo.java @@ -18,7 +18,6 @@ */ package org.apache.tinkerpop.gremlin.structure.io.graphml; -import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.io.Io; import org.apache.tinkerpop.gremlin.structure.io.IoRegistry; @@ -38,9 +37,7 @@ import java.util.function.Consumer; * such things. * * @author Stephen Mallette (http://stephen.genoprime.com) - * @deprecated As of release 3.4.0, replaced by {@link GraphTraversalSource#io(String)}. */ -@Deprecated public final class GraphMLIo implements Io { private final Graph graph; private Optional> onMapper; http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/8fd3bf21/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONIo.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONIo.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONIo.java index 7f8b835..a3923a1 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONIo.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONIo.java @@ -18,7 +18,6 @@ */ package org.apache.tinkerpop.gremlin.structure.io.graphson; -import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.io.Io; import org.apache.tinkerpop.gremlin.structure.io.IoRegistry; @@ -37,9 +36,7 @@ import java.util.function.Consumer; * interfaces should see the {@link GraphSONMapper} for information on the expectations for the {@link IoRegistry}.
[17/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Fixed up general strategy application around io()
TINKERPOP-1996 Fixed up general strategy application around io() The GraphComputer was not being set properly in the HadoopIoStep and therefore executions of OLAP runs would not work even if withComputer(SparkGraphComputer) was set. It only worked if the gremlin.hadoop.defaultGraphComputer property was set which was weird. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c97d747f Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c97d747f Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c97d747f Branch: refs/heads/TINKERPOP-1990 Commit: c97d747f4b05cef42eb0e53adae50fcdee083a2e Parents: 4d979cf Author: Stephen Mallette Authored: Fri Jul 20 10:30:00 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 10:30:00 2018 -0400 -- .../step/map/TraversalVertexProgramStep.java| 4 --- .../decoration/VertexProgramStrategy.java | 30 +--- .../traversal/step/sideEffect/IoStep.java | 2 -- .../traversal/strategy/HadoopIoStrategy.java| 28 -- 4 files changed, 33 insertions(+), 31 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c97d747f/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/TraversalVertexProgramStep.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/TraversalVertexProgramStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/TraversalVertexProgramStep.java index 4eb950f..30cfee5 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/TraversalVertexProgramStep.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/step/map/TraversalVertexProgramStep.java @@ -19,19 +19,15 @@ package org.apache.tinkerpop.gremlin.process.computer.traversal.step.map; -import org.apache.tinkerpop.gremlin.process.computer.GraphComputer; import org.apache.tinkerpop.gremlin.process.computer.GraphFilter; import org.apache.tinkerpop.gremlin.process.computer.Memory; import org.apache.tinkerpop.gremlin.process.computer.traversal.MemoryTraversalSideEffects; import org.apache.tinkerpop.gremlin.process.computer.traversal.TraversalVertexProgram; -import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration.VertexProgramStrategy; -import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.finalization.ComputerFinalizationStrategy; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.step.TraversalParent; import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement; -import org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversalStrategies; import org.apache.tinkerpop.gremlin.process.traversal.util.PureTraversal; import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.util.StringFactory; http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c97d747f/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/decoration/VertexProgramStrategy.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/decoration/VertexProgramStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/decoration/VertexProgramStrategy.java index cb99652..fa6e23f 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/decoration/VertexProgramStrategy.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/strategy/decoration/VertexProgramStrategy.java @@ -101,23 +101,21 @@ public final class VertexProgramStrategy extends AbstractTraversalStrategy computerTraversal = new DefaultTraversal<>(); +final Step firstLegalOLAPStep = getFirstLegalOLAPStep(currentStep); +final Step lastLegalOLAPStep = getLastLegalOLAPStep(currentStep); +if (!(firstLegalOLAPStep instanceof EmptyStep)) { +final int index = TraversalHelper.stepIndex(firstLegalOLAPStep, traversal); +TraversalHelper.removeToTraversal(firstLegalOLAPStep, lastLegalOLAPStep.getNextStep(), (Traversal.Admin) computerTraversal); +final TraversalVertexProgramStep
[38/50] [abbrv] tinkerpop git commit: dedup build-helper-maven-plugin - CTR
dedup build-helper-maven-plugin - CTR Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/dfe79c24 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/dfe79c24 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/dfe79c24 Branch: refs/heads/TINKERPOP-1990 Commit: dfe79c24f4decb667ad20c729928a50f8cb1ec0b Parents: c49c0cc Author: Robert Dale Authored: Tue Jul 31 13:12:31 2018 -0400 Committer: Robert Dale Committed: Tue Jul 31 13:12:31 2018 -0400 -- pom.xml | 16 +--- 1 file changed, 1 insertion(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/dfe79c24/pom.xml -- diff --git a/pom.xml b/pom.xml index b669872..f2629ad 100644 --- a/pom.xml +++ b/pom.xml @@ -305,6 +305,7 @@ limitations under the License. false + parse-version @@ -407,21 +408,6 @@ limitations under the License. - - -org.codehaus.mojo -build-helper-maven-plugin - 3.0.0 - - -parse-version - -parse-version - -validate - - - org.revapi revapi-maven-plugin
[39/50] [abbrv] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/f50aeb6d Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/f50aeb6d Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/f50aeb6d Branch: refs/heads/TINKERPOP-1990 Commit: f50aeb6d62009845f42d9558b7cc8e576c91cf4e Parents: a2db39c dfe79c2 Author: Robert Dale Authored: Tue Jul 31 13:12:40 2018 -0400 Committer: Robert Dale Committed: Tue Jul 31 13:12:40 2018 -0400 -- pom.xml | 16 +--- 1 file changed, 1 insertion(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f50aeb6d/pom.xml --
[33/50] [abbrv] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/7d21ee0b Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/7d21ee0b Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/7d21ee0b Branch: refs/heads/TINKERPOP-1990 Commit: 7d21ee0b8b8770a68e6b5fb6f84500170f6c2a82 Parents: 4dccf14 ddc6694 Author: Stephen Mallette Authored: Mon Jul 30 11:41:28 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 11:41:28 2018 -0400 -- .../org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --
[13/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Added support for setting IoRegistries using with()
TINKERPOP-1996 Added support for setting IoRegistries using with() IORegistry instances are important because they feed serializer information to the Reader/Writer instances. Of all the configuration options that one seemed like the most important to make possible using with(). Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/ff71c6ab Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/ff71c6ab Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/ff71c6ab Branch: refs/heads/TINKERPOP-1990 Commit: ff71c6abee0b39d7ee95128c3d64906daad96a76 Parents: ae3b149 Author: Stephen Mallette Authored: Thu Jul 19 16:13:57 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 16:13:57 2018 -0400 -- .../tinkerpop/gremlin/process/traversal/IO.java | 7 ++ .../traversal/step/sideEffect/IoStep.java | 75 .../process/traversal/step/util/Parameters.java | 4 +- .../Process/Traversal/GraphTraversal.cs | 21 ++ .../src/Gremlin.Net/Process/Traversal/IO.cs | 2 + .../gremlin-javascript/lib/process/traversal.js | 4 ++ .../jython/gremlin_python/process/traversal.py | 2 + .../gremlin/structure/io/util/CustomId.java | 39 ++ .../step/sideEffect/TinkerGraphIoStepTest.java | 75 9 files changed, 181 insertions(+), 48 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ff71c6ab/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/IO.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/IO.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/IO.java index 6668cf1..67b4670 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/IO.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/IO.java @@ -23,6 +23,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSo import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.io.GraphReader; import org.apache.tinkerpop.gremlin.structure.io.GraphWriter; +import org.apache.tinkerpop.gremlin.structure.io.IoRegistry; import org.apache.tinkerpop.gremlin.structure.io.graphml.GraphMLReader; import org.apache.tinkerpop.gremlin.structure.io.graphml.GraphMLWriter; import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONReader; @@ -73,4 +74,10 @@ public class IO { * the file extension provided to it. */ public static final String writer = Graph.Hidden.hide("tinkerpop.io.writer"); + +/** + * A key that identifies the fully qualified class names of {@link IoRegistry} instances to use. May be specified + * multiple times (i.e. once for each registry) using the {@link GraphTraversal#with(String, Object)} modulator. + */ +public static final String registry = Graph.Hidden.hide("tinkerpop.io.registry"); } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ff71c6ab/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/IoStep.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/IoStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/IoStep.java index 9804333..1d4f40b 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/IoStep.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/IoStep.java @@ -31,10 +31,13 @@ import org.apache.tinkerpop.gremlin.process.traversal.util.FastNoSuchElementExce import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.io.GraphReader; import org.apache.tinkerpop.gremlin.structure.io.GraphWriter; +import org.apache.tinkerpop.gremlin.structure.io.IoRegistry; import org.apache.tinkerpop.gremlin.structure.io.graphml.GraphMLReader; import org.apache.tinkerpop.gremlin.structure.io.graphml.GraphMLWriter; +import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONMapper; import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONReader; import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONWriter; +import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoMapper; import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader; import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoWriter; import org.apache.tinkerpop.gremlin.structure.util.StringFactory; @@ -46,6 +49,9 @@ import java.io.IOException; import
[19/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Added upgrade docs
TINKERPOP-1996 Added upgrade docs Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/ded7c187 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/ded7c187 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/ded7c187 Branch: refs/heads/TINKERPOP-1990 Commit: ded7c187480a68b6f7be2d0cb777461f57a2871c Parents: 8fd3bf2 Author: Stephen Mallette Authored: Fri Jul 20 11:20:03 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 11:20:03 2018 -0400 -- docs/src/upgrade/release-3.4.x.asciidoc | 49 ++-- 1 file changed, 47 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ded7c187/docs/src/upgrade/release-3.4.x.asciidoc -- diff --git a/docs/src/upgrade/release-3.4.x.asciidoc b/docs/src/upgrade/release-3.4.x.asciidoc index 97e14e3..0bb3903 100644 --- a/docs/src/upgrade/release-3.4.x.asciidoc +++ b/docs/src/upgrade/release-3.4.x.asciidoc @@ -36,7 +36,7 @@ modulator. This modulator allows the step it is modifying to accept configuratio behavior of the step itself. A good example of its usage is shown with the revised syntax of the `pageRank()` step which now uses `with()` to replace the old `by()` options: -[groovy] +[source,groovy] g.V().hasLabel('person'). pageRank(). @@ -49,7 +49,7 @@ g.V().hasLabel('person'). A similar change was made for `peerPressure()` step: -[groovy] +[source,groovy] g.V().hasLabel('person'). peerPressure(). @@ -65,6 +65,33 @@ release where breaking changes are allowed. See: link:https://issues.apache.org/jira/browse/TINKERPOP-1975[TINKERPOP-1975], link:http://tinkerpop.apache.org/docs/3.4.0/reference/#with-step[Reference Documentation] + io() Step + +There have been some important changes to IO operations for reading and writing graph data. The use of `Graph.io()` +has been deprecated to further remove dependence on the Graph (Structure) API for users and to extend these basic +operations to GLV users by making these features available as part of the Gremlin language. + +It is now possible to simply use Gremlin: + +[source,groovy] + +graph = ... +g = graph.traversal() +g.io(someInputFile).read().iterate() +g.io(someOutputFile).write().iterate() + + +While `io()` step is still single-threaded for OLTP style loading, it can be utilized in conjunction with OLAP which +internally uses `CloneVertexProgram` and therefore any graph `InputFormat` or `OutputFormat` can be configured in +conjunction with this step for parallel loads of large datasets. + +It is also worth noting that the `io()`-step may be overriden by graph providers to utilize their native bulk-loading +features, so consult the documentation of the implementation being used to determine if there are any improved +efficiencies there. + +See: link:https://issues.apache.org/jira/browse/TINKERPOP-1996[TINKERPOP-1996], +link:http://tinkerpop.apache.org/docs/3.4.0/reference/#io-step[Reference Documentation] + Removal of Giraph Support Support for Giraph has been removed as of this version. There were a number of reasons for this decision which were @@ -284,6 +311,24 @@ See: link:https://issues.apache.org/jira/browse/TINKERPOP-1522[TINKERPOP-1522] Graph Database Providers += io() Step + +The new `io()`-step that was introduced provides some new changes to consider. Note that `Graph.io()` has been +deprecated and users are no longer instructed to utilize that method. It is not yet decided when that method will be +removed completely, but given the public nature of it and the high chance of common usage, it should be hanging around +for some time. + +As with any step in Gremlin, it is possible to replace it with a more provider specific implementation that could be +more efficient. Developing a `TraversalStrategy` to do this is encouraged, especially for those graph providers who +might have special bulk loaders that could be abstracted by this step. Examples of this are already shown with +`HadoopGraph` which replaces the simple single-threaded loader with `CloneVertexProgram`. Graph providers are +encouraged to use the `with()` step to capture any necessary configurations required for their underlying loader to +work. Graph providers should not feel restricted to `graphson`, `gryo` and `graphml` formats either. If a graph +supports CSV or some custom graph specific format, it shouldn't be difficult to gather the configurations necessary to +make that available to users. + +See: link:https://issues.apache.org/jira/browse/TINKERPOP-1996[TINKERPOP-1996] + = Caching Graph Features For graph implementations that have expensive creation times, it can be time consuming to run the
[47/50] [abbrv] tinkerpop git commit: split ShortestPath example code block in the docs
split ShortestPath example code block in the docs Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/362ed16e Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/362ed16e Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/362ed16e Branch: refs/heads/TINKERPOP-1990 Commit: 362ed16e4cb6e7d762f5b21863a840b45a8e7491 Parents: 101e5d8 Author: Daniel Kuppitz Authored: Mon Jul 30 09:49:26 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 12:26:30 2018 -0700 -- docs/src/reference/the-traversal.asciidoc | 33 +++--- 1 file changed, 19 insertions(+), 14 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/362ed16e/docs/src/reference/the-traversal.asciidoc -- diff --git a/docs/src/reference/the-traversal.asciidoc b/docs/src/reference/the-traversal.asciidoc index f4a60fb..edc0645 100644 --- a/docs/src/reference/the-traversal.asciidoc +++ b/docs/src/reference/the-traversal.asciidoc @@ -2681,29 +2681,24 @@ using the `with()`-modulator with the options given below. [gremlin-groovy,modern] -a = g.withComputer() -a.V().shortestPath() <1> -a.V().has('person','name','marko').shortestPath() <2> -a.V().shortestPath().with(ShortestPath.target, __.has('name','peter')) <3> -a.V().shortestPath(). +g = g.withComputer() +g.V().shortestPath() <1> +g.V().has('person','name','marko').shortestPath() <2> +g.V().shortestPath().with(ShortestPath.target, __.has('name','peter')) <3> +g.V().shortestPath(). with(ShortestPath.edges, Direction.IN). with(ShortestPath.target, __.has('name','josh')) <4> -a.V().has('person','name','marko'). +g.V().has('person','name','marko'). shortestPath(). with(ShortestPath.target, __.has('name','josh')) <5> -a.V().has('person','name','marko'). +g.V().has('person','name','marko'). shortestPath(). with(ShortestPath.target, __.has('name','josh')). with(ShortestPath.distance, 'weight') <6> -a.V().has('person','name','marko'). +g.V().has('person','name','marko'). shortestPath(). with(ShortestPath.target, __.has('name','josh')). with(ShortestPath.includeEdges, true) <7> -g.inject(a.V().shortestPath(). - with(ShortestPath.distance, 'weight'). - with(ShortestPath.includeEdges, true). - with(ShortestPath.maxDistance, 1).toList().toArray()). - map(unfold().values('name','weight').fold()) <8> <1> Find all shortest paths. @@ -2713,7 +2708,17 @@ g.inject(a.V().shortestPath(). <5> Find all shortest paths from `marko` to `josh`. <6> Find all shortest paths from `marko` to `josh` using a custom distance property. <7> Find all shortest paths from `marko` to `josh` and include edges in the result. -<8> Find all shortest paths using a custom distance property and limit the distance to 1. Inject the result into a OLTP GraphTraversal in order to be able to select properties from all elements in all paths. + +[gremlin-groovy,modern] + +g.inject(g.withComputer().V().shortestPath(). + with(ShortestPath.distance, 'weight'). + with(ShortestPath.includeEdges, true). + with(ShortestPath.maxDistance, 1).toList().toArray()). + map(unfold().values('name','weight').fold()) <1> + + +<1> Find all shortest paths using a custom distance property and limit the distance to 1. Inject the result into a OLTP `GraphTraversal` in order to be able to select properties from all elements in all paths. *Additional References*
[34/50] [abbrv] tinkerpop git commit: Merge branch 'TINKERPOP-1996'
Merge branch 'TINKERPOP-1996' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/edd82346 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/edd82346 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/edd82346 Branch: refs/heads/TINKERPOP-1990 Commit: edd8234684bedb7c073479724b2ee645a801cbc5 Parents: 7d21ee0 10478be Author: Stephen Mallette Authored: Tue Jul 31 07:35:52 2018 -0400 Committer: Stephen Mallette Committed: Tue Jul 31 07:35:52 2018 -0400 -- CHANGELOG.asciidoc | 4 + docs/src/recipes/centrality.asciidoc| 2 +- .../reference/implementations-neo4j.asciidoc| 5 +- .../implementations-tinkergraph.asciidoc| 8 +- docs/src/reference/the-graph.asciidoc | 370 --- docs/src/reference/the-traversal.asciidoc | 186 +- docs/src/upgrade/release-3.4.x.asciidoc | 49 ++- .../tinkerpop/gremlin/jsr223/CoreImports.java | 2 + .../step/map/TraversalVertexProgramStep.java| 4 - .../decoration/VertexProgramStrategy.java | 11 +- .../tinkerpop/gremlin/process/traversal/IO.java | 83 + .../traversal/dsl/graph/GraphTraversal.java | 37 ++ .../dsl/graph/GraphTraversalSource.java | 22 ++ .../process/traversal/step/ReadWriting.java | 48 +++ .../traversal/step/sideEffect/IoStep.java | 254 + .../process/traversal/step/util/Parameters.java | 4 +- .../ComputerVerificationStrategy.java | 3 +- .../StandardVerificationStrategy.java | 9 + .../tinkerpop/gremlin/structure/Graph.java | 30 ++ .../tinkerpop/gremlin/structure/io/IoCore.java | 3 + .../traversal/dsl/graph/GraphTraversalTest.java | 5 +- .../decoration/VertexProgramStrategyTest.java | 2 +- .../StandardVerificationStrategyTest.java | 13 +- .../glv/GraphTraversalSource.template | 4 +- gremlin-dotnet/glv/IO.template | 46 +++ gremlin-dotnet/glv/generate.groovy | 13 +- .../Process/Traversal/GraphTraversal.cs | 15 +- .../Process/Traversal/GraphTraversalSource.cs | 11 + .../src/Gremlin.Net/Process/Traversal/IO.cs | 56 +++ .../Gherkin/TraversalEvaluation/IOParameter.cs | 82 .../TraversalEvaluation/TraversalParser.cs | 7 + .../groovy/jsr223/GroovyTranslatorProvider.java | 6 + gremlin-javascript/glv/TraversalSource.template | 9 + gremlin-javascript/glv/generate.groovy | 4 + .../lib/process/graph-traversal.js | 30 ++ .../gremlin-javascript/lib/process/traversal.js | 29 ++ .../test/cucumber/feature-steps.js | 2 + gremlin-python/glv/TraversalSource.template | 11 + gremlin-python/glv/generate.groovy | 4 + .../gremlin_python/process/graph_traversal.py | 13 + .../jython/gremlin_python/process/traversal.py | 21 ++ .../src/main/jython/radish/feature_steps.py | 5 +- gremlin-test/features/sideEffect/Read.feature | 84 + gremlin-test/features/sideEffect/Write.feature | 60 +++ .../gremlin/AbstractGraphProvider.java | 22 +- .../apache/tinkerpop/gremlin/TestHelper.java| 24 +- .../gremlin/process/ProcessComputerSuite.java | 4 + .../gremlin/process/ProcessStandardSuite.java | 4 + .../process/traversal/step/map/ReadTest.java| 188 ++ .../process/traversal/step/map/WriteTest.java | 183 + .../ElementIdStrategyProcessTest.java | 1 - .../gremlin/structure/io/util/CustomId.java | 39 ++ .../gremlin/process/FeatureCoverageTest.java| 6 +- .../traversal/step/sideEffect/HadoopIoStep.java | 188 ++ .../traversal/strategy/HadoopIoStrategy.java| 83 + .../gremlin/hadoop/structure/HadoopGraph.java | 24 ++ .../step/sideEffect/TinkerGraphIoStepTest.java | 98 + .../structure/TinkerGraphPlayTest.java | 6 +- ...ctTinkerGraphGraphSONTranslatorProvider.java | 5 + .../gryo/TinkerGraphGryoTranslatorProvider.java | 7 +- 60 files changed, 2121 insertions(+), 427 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/edd82346/CHANGELOG.asciidoc -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/edd82346/docs/src/reference/the-traversal.asciidoc -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/edd82346/docs/src/upgrade/release-3.4.x.asciidoc -- diff --cc docs/src/upgrade/release-3.4.x.asciidoc index 2d699dc,8d87397..9951024 --- a/docs/src/upgrade/release-3.4.x.asciidoc +++ b/docs/src/upgrade/release-3.4.x.asciidoc @@@ -31,12 -31,12 +31,12
[41/50] [abbrv] tinkerpop git commit: CTR: Fixed build by adding python3-dev dependency to the base image Dockerfile
CTR: Fixed build by adding python3-dev dependency to the base image Dockerfile Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/7fce1377 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/7fce1377 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/7fce1377 Branch: refs/heads/TINKERPOP-1990 Commit: 7fce1377f70190954f858901c7a3698dbd17cf82 Parents: e3018fb Author: Daniel Kuppitz Authored: Wed Aug 1 11:26:51 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 11:26:51 2018 -0700 -- docker/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7fce1377/docker/Dockerfile -- diff --git a/docker/Dockerfile b/docker/Dockerfile index dbc06ef..9ae74aa 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -30,7 +30,7 @@ RUN apt-get update \ && apt-get update \ && apt-get install -y oracle-java8-installer curl gawk git maven openssh-server subversion zip \ && sh -c 'curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > /etc/apt/trusted.gpg.d/microsoft.gpg' \ -&& apt-get install -y --force-yes dotnet-sdk-2.1.101 python python-dev python-pip build-essential mono-devel \ +&& apt-get install -y --force-yes dotnet-sdk-2.1.101 python python-dev python3-dev python-pip build-essential mono-devel \ && pip install virtualenv virtualenvwrapper \ && pip install --upgrade pip \ && rm -rf /var/lib/apt/lists/* /var/cache/oracle-jdk8-installer
[25/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Prevent OLTP style execution in Hadoop of io()
TINKERPOP-1996 Prevent OLTP style execution in Hadoop of io() Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/fdb35c69 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/fdb35c69 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/fdb35c69 Branch: refs/heads/TINKERPOP-1990 Commit: fdb35c69422f95199563e938ec30a2408d419a58 Parents: e9ebacf Author: Stephen Mallette Authored: Fri Jul 20 16:13:47 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 16:13:47 2018 -0400 -- .../process/computer/traversal/strategy/HadoopIoStrategy.java | 6 ++ 1 file changed, 6 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/fdb35c69/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/strategy/HadoopIoStrategy.java -- diff --git a/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/strategy/HadoopIoStrategy.java b/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/strategy/HadoopIoStrategy.java index af3db6e..3614745 100644 --- a/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/strategy/HadoopIoStrategy.java +++ b/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/strategy/HadoopIoStrategy.java @@ -29,6 +29,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.step.ReadWriting; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IoStep; import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy; +import org.apache.tinkerpop.gremlin.process.traversal.strategy.verification.VerificationException; import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper; import java.util.Arrays; @@ -54,6 +55,11 @@ public final class HadoopIoStrategy extends AbstractTraversalStrategy traversal) { +// since hadoopgraph can't be modified we can't try to use the existing IoStep for standard processing +// without graphcomputer +if (traversal.getStartStep() instanceof IoStep) +throw new VerificationException("HadoopGraph requires a GraphComputer for io() step", traversal); + // VertexProgramStrategy should wrap up the IoStep in a TraversalVertexProgramStep. use that to grab the // GraphComputer that was injected in there and push that in to the HadoopIoStep. this step pattern match // is fairly specific and since you really can't chain together steps after io() this approach should work
[36/50] [abbrv] tinkerpop git commit: Merge branch 'tp32' into tp33
Merge branch 'tp32' into tp33 Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c49c0ccb Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c49c0ccb Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c49c0ccb Branch: refs/heads/TINKERPOP-1990 Commit: c49c0ccb3bce76caabb4b0a9c7efe9df43801b39 Parents: ddc6694 e3018fb Author: Stephen Mallette Authored: Tue Jul 31 08:35:24 2018 -0400 Committer: Stephen Mallette Committed: Tue Jul 31 08:35:24 2018 -0400 -- docs/src/dev/developer/development-environment.asciidoc | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c49c0ccb/docs/src/dev/developer/development-environment.asciidoc --
[03/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Removed use of graph.io() in docs
TINKERPOP-1996 Removed use of graph.io() in docs Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/5bf19e2d Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/5bf19e2d Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/5bf19e2d Branch: refs/heads/TINKERPOP-1990 Commit: 5bf19e2d35aa2f050ecca0a7b2190553b0422720 Parents: 6d05805 Author: Stephen Mallette Authored: Mon Jul 16 09:29:01 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 13:40:10 2018 -0400 -- docs/src/recipes/centrality.asciidoc | 2 +- docs/src/reference/implementations-neo4j.asciidoc | 5 +++-- .../src/reference/implementations-tinkergraph.asciidoc | 8 docs/src/reference/the-traversal.asciidoc | 13 +++-- 4 files changed, 15 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/5bf19e2d/docs/src/recipes/centrality.asciidoc -- diff --git a/docs/src/recipes/centrality.asciidoc b/docs/src/recipes/centrality.asciidoc index 8504589..0c1d4bc 100644 --- a/docs/src/recipes/centrality.asciidoc +++ b/docs/src/recipes/centrality.asciidoc @@ -156,7 +156,7 @@ give it the highest rank. Consider the following example using the Grateful Dead [gremlin-groovy] -graph.io(graphml()).readGraph('data/grateful-dead.xml') +g.io('data/grateful-dead.xml').read() g.V().repeat(groupCount('m').by('name').out()).times(5).cap('m'). <1> order(local).by(values, desc).limit(local, 10).next() <2> g.V().repeat(groupCount('m').by('name').out().timeLimit(100)).times(5).cap('m'). <3> http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/5bf19e2d/docs/src/reference/implementations-neo4j.asciidoc -- diff --git a/docs/src/reference/implementations-neo4j.asciidoc b/docs/src/reference/implementations-neo4j.asciidoc index 7e05221..1760bd6 100644 --- a/docs/src/reference/implementations-neo4j.asciidoc +++ b/docs/src/reference/implementations-neo4j.asciidoc @@ -93,8 +93,8 @@ labels), a linear scan of the vertex-label partition is still faster than a line [gremlin-groovy] graph = Neo4jGraph.open('/tmp/neo4j') -graph.io(graphml()).readGraph('data/grateful-dead.xml') g = graph.traversal() +g.io('data/grateful-dead.xml').read() g.tx().commit() clock(1000) {g.V().hasLabel('artist').has('name','Garcia').iterate()} <1> graph.cypher("CREATE INDEX ON :artist(name)") <2> @@ -161,7 +161,8 @@ It is possible to leverage Cypher from within Gremlin by using the `Neo4jGraph.c [gremlin-groovy] graph = Neo4jGraph.open('/tmp/neo4j') -graph.io(gryo()).readGraph('data/tinkerpop-modern.kryo') +g = graph.traversal() +g.io('data/tinkerpop-modern.kryo').read() graph.cypher('MATCH (a {name:"marko"}) RETURN a') graph.cypher('MATCH (a {name:"marko"}) RETURN a').select('a').out('knows').values('name') graph.close() http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/5bf19e2d/docs/src/reference/implementations-tinkergraph.asciidoc -- diff --git a/docs/src/reference/implementations-tinkergraph.asciidoc b/docs/src/reference/implementations-tinkergraph.asciidoc index 5b04126..acd37dd 100644 --- a/docs/src/reference/implementations-tinkergraph.asciidoc +++ b/docs/src/reference/implementations-tinkergraph.asciidoc @@ -90,12 +90,12 @@ TinkerGraph over the Grateful Dead graph. graph = TinkerGraph.open() g = graph.traversal() -graph.io(graphml()).readGraph('data/grateful-dead.xml') +g.io('data/grateful-dead.xml').read() clock(1000) {g.V().has('name','Garcia').iterate()} <1> graph = TinkerGraph.open() g = graph.traversal() graph.createIndex('name',Vertex.class) -graph.io(graphml()).readGraph('data/grateful-dead.xml') +g.io('data/grateful-dead.xml').read() clock(1000){g.V().has('name','Garcia').iterate()} <2> @@ -158,13 +158,13 @@ cardinality to `list` or else the data will import as `single`. Consider the fo [gremlin-groovy] graph = TinkerGraph.open() -graph.io(gryo()).readGraph("data/tinkerpop-crew.kryo") g = graph.traversal() +g.io("data/tinkerpop-crew.kryo").read() g.V().properties() conf = new BaseConfiguration() conf.setProperty("gremlin.tinkergraph.defaultVertexPropertyCardinality","list") graph = TinkerGraph.open(conf) -graph.io(gryo()).readGraph("data/tinkerpop-crew.kryo") g = graph.traversal() +g.io("data/tinkerpop-crew.kryo").read() g.V().properties() http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/5bf19e2d/docs/src/reference/the-traversal.asciidoc -- diff --git
[20/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Verification strategy to prevent io() from misuse
TINKERPOP-1996 Verification strategy to prevent io() from misuse Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/e6e4413e Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/e6e4413e Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/e6e4413e Branch: refs/heads/TINKERPOP-1990 Commit: e6e4413e46594ae8f6d408de4e3d3bd85c228f53 Parents: ded7c18 Author: Stephen Mallette Authored: Fri Jul 20 11:47:24 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 11:47:24 2018 -0400 -- .../strategy/verification/StandardVerificationStrategy.java | 7 +++ .../verification/StandardVerificationStrategyTest.java| 5 + 2 files changed, 12 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e6e4413e/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategy.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategy.java index 50faa25..258d345 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategy.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategy.java @@ -23,6 +23,7 @@ import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.finaliza import org.apache.tinkerpop.gremlin.process.traversal.Step; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; +import org.apache.tinkerpop.gremlin.process.traversal.step.ReadWriting; import org.apache.tinkerpop.gremlin.process.traversal.step.branch.RepeatStep; import org.apache.tinkerpop.gremlin.process.traversal.step.filter.NoneStep; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.ProfileSideEffectStep; @@ -39,6 +40,7 @@ import java.util.Set; /** * @author Marko A. Rodriguez (http://markorodriguez.com) + * @author Stephen Mallette (http://stephen.genoprime.com) */ public final class StandardVerificationStrategy extends AbstractTraversalStrategy implements TraversalStrategy.VerificationStrategy { @@ -81,6 +83,11 @@ public final class StandardVerificationStrategy extends AbstractTraversalStrateg if (TraversalHelper.getStepsOfClass(ProfileSideEffectStep.class, traversal).size() > 1) { throw new VerificationException("The profile()-Step cannot be specified multiple times.", traversal); } + +if (traversal.getStartStep() instanceof ReadWriting && traversal.getSteps().size() != 1) { +if (TraversalHelper.getStepsOfClass(NoneStep.class, traversal).size() != traversal.getSteps().size() - 1) +throw new VerificationException("The io() step must be the first and only step in the traversal - it cannot be used with other steps", traversal); +} } public static StandardVerificationStrategy instance() { http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e6e4413e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategyTest.java -- diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategyTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategyTest.java index ee96266..deec002 100644 --- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategyTest.java +++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/StandardVerificationStrategyTest.java @@ -24,6 +24,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__; import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.RequirementsStrategy; import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement; import org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversalStrategies; +import org.apache.tinkerpop.gremlin.structure.util.empty.EmptyGraph; import org.junit.Test; import org.junit.runner.RunWith; import org.junit.runners.Parameterized; @@ -50,7 +51,11 @@ public class StandardVerificationStrategyTest {
[06/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Updated changelog
TINKERPOP-1996 Updated changelog Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/576649fd Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/576649fd Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/576649fd Branch: refs/heads/TINKERPOP-1990 Commit: 576649fd5456f6390bf9481d01438a7e78db083e Parents: f148e93 Author: Stephen Mallette Authored: Fri Jul 13 15:22:43 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 13:40:10 2018 -0400 -- CHANGELOG.asciidoc | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/576649fd/CHANGELOG.asciidoc -- diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index bc78bbe..1add03c 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -28,6 +28,7 @@ This release also includes changes from <>. * Bumped to Netty 4.1.25. * Bumped to Spark 2.3.1. * Added the `io()` start step and `read()` and `write()` termination steps to the Gremlin language. +* Added `GraphFeatures.supportsIoRead()` and `GraphFeatures.supportsIoWrite()`. * Deprecated `Graph.io()` and related infrastructure. * Moved `Parameterizing` interface to the `org.apache.tinkerpop.gremlin.process.traversal.step` package with other marker interfaces of its type. * Replaced `Parameterizing.addPropertyMutations()` with `Configuring.configure()`.
[49/50] [abbrv] tinkerpop git commit: added comments in SPVP
added comments in SPVP Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/eae0184a Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/eae0184a Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/eae0184a Branch: refs/heads/TINKERPOP-1990 Commit: eae0184a636ed296a2b67bea12a64c93e683df5d Parents: 62d0dbc Author: Daniel Kuppitz Authored: Mon Jul 30 09:40:43 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 12:26:30 2018 -0700 -- .../search/path/ShortestPathVertexProgram.java | 32 +++- .../step/map/ShortestPathVertexProgramStep.java | 3 ++ 2 files changed, 34 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/eae0184a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/search/path/ShortestPathVertexProgram.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/search/path/ShortestPathVertexProgram.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/search/path/ShortestPathVertexProgram.java index 549dff9..1949c53 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/search/path/ShortestPathVertexProgram.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/search/path/ShortestPathVertexProgram.java @@ -152,6 +152,7 @@ public class ShortestPathVertexProgram implements VertexProgram(v -> v); for (final Traverser.Admin traverser : this.haltedTraversers) { @@ -254,10 +255,15 @@ public class ShortestPathVertexProgram implements VertexProgram>> paths = new HashMap<>(); final Path path; final Set pathSet = new HashSet<>(); @@ -267,12 +273,14 @@ public class ShortestPathVertexProgram implements VertexProgram>> paths = vertex.>>>property(PATHS).orElseGet(HashMap::new); final Iterator> iterator = messenger.receiveMessages(); @@ -286,6 +294,7 @@ public class ShortestPathVertexProgram implements VertexProgram pathSet = new HashSet<>(); pathSet.add(newPath); paths.put(sourceVertex, Pair.with(distance, pathSet)); } else { +// if the path length is equal to the current shortest path's length, add the new path +// to the set of shortest paths paths.get(sourceVertex).getValue1().add(newPath); } } } else if (!exceedsMaxDistance(distance)) { +// store the new path as the shortest path from the source vertex to the current vertex final Set pathSet = new HashSet<>(); pathSet.add(newPath = extendPath(sourcePath, triplet.getValue1(), vertex)); paths.put(sourceVertex, Pair.with(distance, pathSet)); } +// if a new path was found, send messages to adjacent vertices, otherwise do nothing as there's no +// chance to find any new paths going forward if (newPath != null) { vertex.property(VertexProperty.Cardinality.single, PATHS, paths); processEdges(vertex, newPath, distance, messenger); @@ -315,6 +331,7 @@ public class ShortestPathVertexProgram implements VertexProgram filterTraversal = this.sourceVertexFilterTraversal.getPure(); filterTraversal.addStart(filterTraversal.getTraverserGenerator().generate(vertex, filterTraversal.getStartStep(), 1)); return filterTraversal.hasNext(); } +// ...otherwise use halted traversers to determine whether this is a start vertex return vertex.property(TraversalVertexProgram.HALTED_TRAVERSERS).isPresent(); } @@ -429,6 +451,7 @@ public class ShortestPathVertexProgram implements VertexProgram paths = memory.get(SHORTEST_PATHS); if (vertex.property(TraversalVertexProgram.HALTED_TRAVERSERS).isPresent()) { +// replace the current set of halted traversers with new new traversers that hold the shortest paths +// found for this vertex final TraverserSet haltedTraversers = vertex.value(TraversalVertexProgram.HALTED_TRAVERSERS); final TraverserSet newHaltedTraversers = new TraverserSet<>(); for (final Traverser.Admin traverser : haltedTraversers) { @@ -471,6 +496,11 @@ public class ShortestPathVertexProgram implements VertexProgram 0; } +/** + * Move any valid path into the VP's memory. + * @param vertex The current
[04/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Moved IoStep implementations to sideEffect package
TINKERPOP-1996 Moved IoStep implementations to sideEffect package These steps really aren't quite sideEffects and not quite map steps either but they seem to fit better as sideEffect. meh Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/6d05805a Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/6d05805a Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/6d05805a Branch: refs/heads/TINKERPOP-1990 Commit: 6d05805ada657bcb3f50a60aa0c313c29d4611bb Parents: 62175c2 Author: Stephen Mallette Authored: Sat Jul 14 06:23:54 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 13:40:10 2018 -0400 -- .../dsl/graph/GraphTraversalSource.java | 2 +- .../process/traversal/step/map/IoStep.java | 241 --- .../traversal/step/sideEffect/IoStep.java | 241 +++ .../traversal/step/map/HadoopIoStep.java| 170 - .../traversal/step/sideEffect/HadoopIoStep.java | 163 + .../traversal/strategy/HadoopIoStrategy.java| 5 +- 6 files changed, 407 insertions(+), 415 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/6d05805a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalSource.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalSource.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalSource.java index 7357418..df1b108 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalSource.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalSource.java @@ -31,7 +31,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.step.map.AddEdgeStartStep; import org.apache.tinkerpop.gremlin.process.traversal.step.map.AddVertexStartStep; import org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep; -import org.apache.tinkerpop.gremlin.process.traversal.step.map.IoStep; +import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IoStep; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.InjectStep; import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.RequirementsStrategy; import org.apache.tinkerpop.gremlin.process.traversal.traverser.TraverserRequirement; http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/6d05805a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/IoStep.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/IoStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/IoStep.java deleted file mode 100644 index 668b3dc..000 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/IoStep.java +++ /dev/null @@ -1,241 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ -package org.apache.tinkerpop.gremlin.process.traversal.step.map; - -import org.apache.tinkerpop.gremlin.process.traversal.IO; -import org.apache.tinkerpop.gremlin.process.traversal.Traversal; -import org.apache.tinkerpop.gremlin.process.traversal.Traverser; -import org.apache.tinkerpop.gremlin.process.traversal.step.ReadWriting; -import org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep; -import org.apache.tinkerpop.gremlin.process.traversal.step.util.Parameters; -import org.apache.tinkerpop.gremlin.process.traversal.traverser.util.EmptyTraverser; -import org.apache.tinkerpop.gremlin.process.traversal.util.FastNoSuchElementException; -import org.apache.tinkerpop.gremlin.structure.Graph; -import
[37/50] [abbrv] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/a2db39cc Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/a2db39cc Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/a2db39cc Branch: refs/heads/TINKERPOP-1990 Commit: a2db39cc710dd2809bab3fb6b2db44257458dcc8 Parents: edd8234 c49c0cc Author: Stephen Mallette Authored: Tue Jul 31 08:35:34 2018 -0400 Committer: Stephen Mallette Committed: Tue Jul 31 08:35:34 2018 -0400 -- docs/src/dev/developer/development-environment.asciidoc | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) --
[24/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Added IoStep to list of unsupported steps
TINKERPOP-1996 Added IoStep to list of unsupported steps If this isn't there then GraphReader/Writer will blow up as it tries to mutate the graph. IoStep is an OLTP only step. For OLAP each graph implementation will need to add its own GraphComputer-ready step. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/e9ebacfc Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/e9ebacfc Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/e9ebacfc Branch: refs/heads/TINKERPOP-1990 Commit: e9ebacfc84aca0e0ac30720bba68939f77c4c5d0 Parents: 23c71b6 Author: Stephen Mallette Authored: Fri Jul 20 15:08:45 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 15:08:45 2018 -0400 -- .../strategy/verification/ComputerVerificationStrategy.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e9ebacfc/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/ComputerVerificationStrategy.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/ComputerVerificationStrategy.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/ComputerVerificationStrategy.java index 2d076b6..5c2d299 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/ComputerVerificationStrategy.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/verification/ComputerVerificationStrategy.java @@ -28,6 +28,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.step.Mutating; import org.apache.tinkerpop.gremlin.process.traversal.step.PathProcessor; import org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.InjectStep; +import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IoStep; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SubgraphStep; import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep; import org.apache.tinkerpop.gremlin.process.traversal.step.util.ProfileStep; @@ -46,7 +47,7 @@ public final class ComputerVerificationStrategy extends AbstractTraversalStrateg private static final ComputerVerificationStrategy INSTANCE = new ComputerVerificationStrategy(); private static final Set> UNSUPPORTED_STEPS = new HashSet<>(Arrays.asList( -InjectStep.class, Mutating.class, SubgraphStep.class, ComputerResultStep.class +InjectStep.class, Mutating.class, SubgraphStep.class, ComputerResultStep.class, IoStep.class )); private ComputerVerificationStrategy() {
[16/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Pass configurations from with() through to Hadoop
TINKERPOP-1996 Pass configurations from with() through to Hadoop This will allow users to override or add to the Hadoop/Spark/OLAP configuration as needed Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/4d979cf8 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/4d979cf8 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/4d979cf8 Branch: refs/heads/TINKERPOP-1990 Commit: 4d979cf8dc4482d574191a76a0c60e281f887563 Parents: 51dc821 Author: Stephen Mallette Authored: Fri Jul 20 07:35:48 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 07:35:48 2018 -0400 -- .../traversal/step/sideEffect/HadoopIoStep.java | 29 ++-- 1 file changed, 27 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/4d979cf8/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/step/sideEffect/HadoopIoStep.java -- diff --git a/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/step/sideEffect/HadoopIoStep.java b/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/step/sideEffect/HadoopIoStep.java index ca369b6..2e96276 100644 --- a/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/step/sideEffect/HadoopIoStep.java +++ b/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/step/sideEffect/HadoopIoStep.java @@ -29,6 +29,7 @@ import org.apache.tinkerpop.gremlin.process.computer.clone.CloneVertexProgram; import org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.VertexProgramStep; import org.apache.tinkerpop.gremlin.process.traversal.IO; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; +import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal; import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; import org.apache.tinkerpop.gremlin.process.traversal.step.ReadWriting; import org.apache.tinkerpop.gremlin.process.traversal.step.util.Parameters; @@ -37,7 +38,9 @@ import org.apache.tinkerpop.gremlin.structure.util.StringFactory; /** * An OLAP oriented step for doing IO operations with {@link GraphTraversalSource#io(String)} which uses the - * {@link CloneVertexProgram} for its implementation. + * {@link CloneVertexProgram} for its implementation. Standard Hadoop OLAP configurations can be passed using the + * {@link GraphTraversal#with(String, Object)} step modulator as all options aside from those in {@link IO} will be + * transferred. * * @author Stephen Mallette (http://stephen.genoprime.com) */ @@ -69,7 +72,6 @@ public class HadoopIoStep extends VertexProgramStep implements ReadWriting { @Override public void configure(final Object... keyValues) { -// TODO: probably should write to the Configuration selectively - no need for actual Parameters? this.parameters.set(null, keyValues); } @@ -121,6 +123,8 @@ public class HadoopIoStep extends VertexProgramStep implements ReadWriting { graph.configuration().setProperty(Constants.GREMLIN_HADOOP_GRAPH_READER, inputFormatClassName); graph.configuration().setProperty(Constants.GREMLIN_HADOOP_INPUT_LOCATION, file); + +addParametersToConfiguration(graph); } private void configureForWrite(final Graph graph) { @@ -137,6 +141,27 @@ public class HadoopIoStep extends VertexProgramStep implements ReadWriting { graph.configuration().setProperty(Constants.GREMLIN_HADOOP_GRAPH_WRITER, outputFormatClassName); graph.configuration().setProperty(Constants.GREMLIN_HADOOP_OUTPUT_LOCATION, file); + +addParametersToConfiguration(graph); +} + +/** + * Overwrites all configurations from values passed using {@link GraphTraversal#with(String, Object)}. + */ +private void addParametersToConfiguration(final Graph graph) { +parameters.getRaw(IO.writer, IO.writer, IO.registry).entrySet().forEach(kv -> { +if (kv.getValue().size() == 1) +graph.configuration().setProperty(kv.getKey().toString(), kv.getValue().get(0)); +else { +// reset the default configuration with the first option then add to that for List options +for (int ix = 0; ix < kv.getValue().size(); ix++) { +if (ix == 0) + graph.configuration().setProperty(kv.getKey().toString(), kv.getValue().get(ix)); +else + graph.configuration().addProperty(kv.getKey().toString(),
[42/50] [abbrv] tinkerpop git commit: Merge branch 'tp32' into tp33
Merge branch 'tp32' into tp33 Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/8af78372 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/8af78372 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/8af78372 Branch: refs/heads/TINKERPOP-1990 Commit: 8af7837286c7eba82c831576a3291c77bfb727e9 Parents: dfe79c2 7fce137 Author: Daniel Kuppitz Authored: Wed Aug 1 11:27:04 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 11:27:04 2018 -0700 -- docker/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --
[08/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Added support/testing for io() in GLVs
TINKERPOP-1996 Added support/testing for io() in GLVs Had to revert to using iterate() and stop read/write() from terminating the traversal. Kinda stinks, but we rely on iterate() quite heavily and for remoting allowing read()/write() to terminate means that the traversal will execute during traversal construction in the translator (which is early and potentially bad). Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/048ea21c Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/048ea21c Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/048ea21c Branch: refs/heads/TINKERPOP-1990 Commit: 048ea21c0adf9297233d06222e75c64dbe1fa1ca Parents: f8e3b8a Author: Stephen Mallette Authored: Thu Jul 19 13:30:02 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 13:41:01 2018 -0400 -- .../traversal/dsl/graph/GraphTraversal.java | 11 +-- gremlin-dotnet/glv/IO.template | 46 +++ gremlin-dotnet/glv/generate.groovy | 8 ++ .../src/Gremlin.Net/Process/Traversal/IO.cs | 54 + .../Gherkin/TraversalEvaluation/IOParameter.cs | 82 +++ .../TraversalEvaluation/TraversalParser.cs | 7 ++ gremlin-javascript/glv/TraversalSource.template | 9 +++ gremlin-javascript/glv/generate.groovy | 4 + .../gremlin-javascript/lib/process/traversal.js | 25 ++ .../test/cucumber/feature-steps.js | 2 + gremlin-python/glv/TraversalSource.template | 11 +++ gremlin-python/glv/generate.groovy | 4 + .../jython/gremlin_python/process/traversal.py | 19 + .../src/main/jython/radish/feature_steps.py | 5 +- gremlin-test/features/sideEffect/Read.feature | 84 gremlin-test/features/sideEffect/Write.feature | 60 ++ .../gremlin/AbstractGraphProvider.java | 2 +- .../process/traversal/step/map/ReadTest.java| 57 ++--- 18 files changed, 471 insertions(+), 19 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/048ea21c/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java index 20f8996..a675ad1 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java @@ -124,6 +124,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.GroupCount import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.GroupSideEffectStep; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IdentityStep; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.InjectStep; +import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IoStep; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.LambdaSideEffectStep; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.ProfileSideEffectStep; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SackValueStep; @@ -2734,13 +2735,13 @@ public interface GraphTraversal extends Traversal { -/ IO TERMINATOR STEPS / +/ IO STEPS / /** * This step is technically a step modulator for the the {@link GraphTraversalSource#io(String)} step which * instructs the step to perform a read with its given configuration. * - * @return the traversal that has been iterated with the read IO action executed + * @return the traversal with the {@link IoStep} modulated to read * @see http://tinkerpop.apache.org/docs/${project.version}/reference/#io-step; target="_blank">Reference Documentation - IO Step * @see http://tinkerpop.apache.org/docs/${project.version}/reference/#read-step; target="_blank">Reference Documentation - Read Step * @since 3.4.0 @@ -2748,14 +2749,14 @@ public interface GraphTraversal extends Traversal { public default GraphTraversal read() { this.asAdmin().getBytecode().addStep(Symbols.read); ((ReadWriting) this.asAdmin().getEndStep()).setMode(ReadWriting.Mode.READING); -return this.iterate(); +return this; } /** * This step is technically a step modulator for the the {@link GraphTraversalSource#io(String)} step which * instructs the
[12/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Added iterate() to read()/write() steps in docs
TINKERPOP-1996 Added iterate() to read()/write() steps in docs Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/ae3b1497 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/ae3b1497 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/ae3b1497 Branch: refs/heads/TINKERPOP-1990 Commit: ae3b149789b8a9779d0549cee8a6ca2e2febbd2b Parents: a580b6f Author: Stephen Mallette Authored: Thu Jul 19 14:24:15 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 14:24:15 2018 -0400 -- docs/src/recipes/centrality.asciidoc| 2 +- .../reference/implementations-neo4j.asciidoc| 4 +-- .../implementations-tinkergraph.asciidoc| 8 ++--- docs/src/reference/the-traversal.asciidoc | 35 +++- 4 files changed, 26 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ae3b1497/docs/src/recipes/centrality.asciidoc -- diff --git a/docs/src/recipes/centrality.asciidoc b/docs/src/recipes/centrality.asciidoc index 0c1d4bc..59e6a4d 100644 --- a/docs/src/recipes/centrality.asciidoc +++ b/docs/src/recipes/centrality.asciidoc @@ -156,7 +156,7 @@ give it the highest rank. Consider the following example using the Grateful Dead [gremlin-groovy] -g.io('data/grateful-dead.xml').read() +g.io('data/grateful-dead.xml').read().iterate() g.V().repeat(groupCount('m').by('name').out()).times(5).cap('m'). <1> order(local).by(values, desc).limit(local, 10).next() <2> g.V().repeat(groupCount('m').by('name').out().timeLimit(100)).times(5).cap('m'). <3> http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ae3b1497/docs/src/reference/implementations-neo4j.asciidoc -- diff --git a/docs/src/reference/implementations-neo4j.asciidoc b/docs/src/reference/implementations-neo4j.asciidoc index 1760bd6..cfbf612 100644 --- a/docs/src/reference/implementations-neo4j.asciidoc +++ b/docs/src/reference/implementations-neo4j.asciidoc @@ -94,7 +94,7 @@ labels), a linear scan of the vertex-label partition is still faster than a line graph = Neo4jGraph.open('/tmp/neo4j') g = graph.traversal() -g.io('data/grateful-dead.xml').read() +g.io('data/grateful-dead.xml').read().iterate() g.tx().commit() clock(1000) {g.V().hasLabel('artist').has('name','Garcia').iterate()} <1> graph.cypher("CREATE INDEX ON :artist(name)") <2> @@ -162,7 +162,7 @@ It is possible to leverage Cypher from within Gremlin by using the `Neo4jGraph.c graph = Neo4jGraph.open('/tmp/neo4j') g = graph.traversal() -g.io('data/tinkerpop-modern.kryo').read() +g.io('data/tinkerpop-modern.kryo').read().iterate() graph.cypher('MATCH (a {name:"marko"}) RETURN a') graph.cypher('MATCH (a {name:"marko"}) RETURN a').select('a').out('knows').values('name') graph.close() http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ae3b1497/docs/src/reference/implementations-tinkergraph.asciidoc -- diff --git a/docs/src/reference/implementations-tinkergraph.asciidoc b/docs/src/reference/implementations-tinkergraph.asciidoc index acd37dd..fe14d0f 100644 --- a/docs/src/reference/implementations-tinkergraph.asciidoc +++ b/docs/src/reference/implementations-tinkergraph.asciidoc @@ -90,12 +90,12 @@ TinkerGraph over the Grateful Dead graph. graph = TinkerGraph.open() g = graph.traversal() -g.io('data/grateful-dead.xml').read() +g.io('data/grateful-dead.xml').read().iterate() clock(1000) {g.V().has('name','Garcia').iterate()} <1> graph = TinkerGraph.open() g = graph.traversal() graph.createIndex('name',Vertex.class) -g.io('data/grateful-dead.xml').read() +g.io('data/grateful-dead.xml').read().iterate() clock(1000){g.V().has('name','Garcia').iterate()} <2> @@ -159,12 +159,12 @@ cardinality to `list` or else the data will import as `single`. Consider the fo graph = TinkerGraph.open() g = graph.traversal() -g.io("data/tinkerpop-crew.kryo").read() +g.io("data/tinkerpop-crew.kryo").read().iterate() g.V().properties() conf = new BaseConfiguration() conf.setProperty("gremlin.tinkergraph.defaultVertexPropertyCardinality","list") graph = TinkerGraph.open(conf) g = graph.traversal() -g.io("data/tinkerpop-crew.kryo").read() +g.io("data/tinkerpop-crew.kryo").read().iterate() g.V().properties() http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ae3b1497/docs/src/reference/the-traversal.asciidoc -- diff --git a/docs/src/reference/the-traversal.asciidoc b/docs/src/reference/the-traversal.asciidoc index cca71f6..cd2f5f2 100644 ---
[45/50] [abbrv] tinkerpop git commit: TINKERPOP-1990 Implemented `ShortestPathVertexProgram` and `ShortestPathVertexProgramStep`.
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e1ba81b4/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessComputerSuite.java -- diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessComputerSuite.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessComputerSuite.java index eab562d..857b3f0 100644 --- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessComputerSuite.java +++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/ProcessComputerSuite.java @@ -26,6 +26,7 @@ import org.apache.tinkerpop.gremlin.process.computer.bulkloading.BulkLoaderVerte import org.apache.tinkerpop.gremlin.process.computer.clone.CloneVertexProgramTest; import org.apache.tinkerpop.gremlin.process.computer.clustering.peerpressure.PeerPressureVertexProgramTest; import org.apache.tinkerpop.gremlin.process.computer.ranking.pagerank.PageRankVertexProgramTest; +import org.apache.tinkerpop.gremlin.process.computer.search.path.ShortestPathVertexProgramTest; import org.apache.tinkerpop.gremlin.process.traversal.TraversalEngine; import org.apache.tinkerpop.gremlin.process.traversal.TraversalInterruptionComputerTest; import org.apache.tinkerpop.gremlin.process.traversal.step.ComplexTest; @@ -71,6 +72,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.step.map.ProgramTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.ProjectTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.PropertiesTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.ReadTest; +import org.apache.tinkerpop.gremlin.process.traversal.step.map.ShortestPathTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.SumTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.UnfoldTest; @@ -166,6 +168,7 @@ public class ProcessComputerSuite extends AbstractGremlinSuite { ProgramTest.Traversals.class, PropertiesTest.Traversals.class, ReadTest.Traversals.class, +ShortestPathTest.Traversals.class, SelectTest.Traversals.class, UnfoldTest.Traversals.class, ValueMapTest.Traversals.class, @@ -194,6 +197,7 @@ public class ProcessComputerSuite extends AbstractGremlinSuite { // algorithms PageRankVertexProgramTest.class, PeerPressureVertexProgramTest.class, +ShortestPathVertexProgramTest.class, BulkLoaderVertexProgramTest.class, BulkDumperVertexProgramTest.class, CloneVertexProgramTest.class, @@ -258,6 +262,7 @@ public class ProcessComputerSuite extends AbstractGremlinSuite { ProjectTest.class, ProgramTest.class, PropertiesTest.class, +ShortestPathTest.class, SelectTest.class, UnfoldTest.class, ValueMapTest.class, http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e1ba81b4/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/computer/search/path/ShortestPathTestHelper.java -- diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/computer/search/path/ShortestPathTestHelper.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/computer/search/path/ShortestPathTestHelper.java new file mode 100644 index 000..7f3aa63 --- /dev/null +++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/computer/search/path/ShortestPathTestHelper.java @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.tinkerpop.gremlin.process.computer.search.path; + +import org.apache.tinkerpop.gremlin.process.AbstractGremlinProcessTest; +import org.apache.tinkerpop.gremlin.process.traversal.P; +import org.apache.tinkerpop.gremlin.process.traversal.Path; +import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; +import
[09/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Removed unecessary enum
TINKERPOP-1996 Removed unecessary enum Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/f8e3b8a1 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/f8e3b8a1 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/f8e3b8a1 Branch: refs/heads/TINKERPOP-1990 Commit: f8e3b8a1df1639405a29f261a9d7da147e88b356 Parents: 8187016 Author: Stephen Mallette Authored: Mon Jul 16 14:47:47 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 13:41:01 2018 -0400 -- .../process/traversal/step/sideEffect/IoStep.java | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f8e3b8a1/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/IoStep.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/IoStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/IoStep.java index 74b295b..9804333 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/IoStep.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/IoStep.java @@ -18,6 +18,8 @@ */ package org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect; +import org.apache.commons.configuration.BaseConfiguration; +import org.apache.commons.configuration.Configuration; import org.apache.tinkerpop.gremlin.process.traversal.IO; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; import org.apache.tinkerpop.gremlin.process.traversal.Traverser; @@ -52,12 +54,6 @@ import java.lang.reflect.Method; */ public class IoStep extends AbstractStep implements ReadWriting { -private enum Format { -GRYO, -GRAPHSON, -GRAPHML -} - private Parameters parameters = new Parameters(); private boolean first = true; private String file; @@ -219,6 +215,12 @@ public class IoStep extends AbstractStep implements ReadWriting { throw new IllegalStateException("Could not detect the file format - specify the writer explicitly or rename file with a standard extension"); } +private Configuration getConfFromParameters() { +final Configuration conf = new BaseConfiguration(); +parameters.getRaw().forEach((key, value) -> conf.setProperty(key.toString(), value.get(0))); +return conf; +} + @Override public int hashCode() { final int hash = super.hashCode() ^ this.parameters.hashCode();
[32/50] [abbrv] tinkerpop git commit: Merge branch 'tp32' into tp33
Merge branch 'tp32' into tp33 Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/ddc66941 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/ddc66941 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/ddc66941 Branch: refs/heads/TINKERPOP-1990 Commit: ddc66941ed6340edd9009f0aa277aa8326255c68 Parents: 88b6e14 00a085a Author: Stephen Mallette Authored: Mon Jul 30 11:41:21 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 11:41:21 2018 -0400 -- .../org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ddc66941/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java --
[10/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Used g.io() in tests by default
TINKERPOP-1996 Used g.io() in tests by default Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/81870168 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/81870168 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/81870168 Branch: refs/heads/TINKERPOP-1990 Commit: 8187016886b8b699ff107c9b6a7dfe95deb7e4a1 Parents: 5bf19e2 Author: Stephen Mallette Authored: Mon Jul 16 12:10:02 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 13:41:01 2018 -0400 -- CHANGELOG.asciidoc | 5 ++-- .../gremlin/AbstractGraphProvider.java | 8 +++ .../apache/tinkerpop/gremlin/TestHelper.java| 24 ++-- 3 files changed, 24 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/81870168/CHANGELOG.asciidoc -- diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 1add03c..1848421 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -25,11 +25,12 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima This release also includes changes from <>. -* Bumped to Netty 4.1.25. -* Bumped to Spark 2.3.1. +* `AbstractGraphProvider` uses `g.io()` for loading test data. * Added the `io()` start step and `read()` and `write()` termination steps to the Gremlin language. * Added `GraphFeatures.supportsIoRead()` and `GraphFeatures.supportsIoWrite()`. * Deprecated `Graph.io()` and related infrastructure. +* Bumped to Netty 4.1.25. +* Bumped to Spark 2.3.1. * Moved `Parameterizing` interface to the `org.apache.tinkerpop.gremlin.process.traversal.step` package with other marker interfaces of its type. * Replaced `Parameterizing.addPropertyMutations()` with `Configuring.configure()`. * Changed interface hierarchy for `Parameterizing` and `Mutating` interfaces as they are tightly related. http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/81870168/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGraphProvider.java -- diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGraphProvider.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGraphProvider.java index 75d033b..b6fc43c 100644 --- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGraphProvider.java +++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/AbstractGraphProvider.java @@ -25,6 +25,7 @@ import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoIo; import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader; import org.apache.commons.configuration.BaseConfiguration; import org.apache.commons.configuration.Configuration; +import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoResourceAccess; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -142,9 +143,8 @@ public abstract class AbstractGraphProvider implements GraphProvider { * @param path the path to the file to load into the graph */ protected void readIntoGraph(final Graph graph, final String path) throws IOException { -final GraphReader reader = GryoReader.build().create(); -try (final InputStream stream = AbstractGremlinTest.class.getResourceAsStream(path)) { -reader.readGraph(stream, graph); -} +final String dataFile = TestHelper.generateTempFileFromResource(graph.getClass(), +GryoResourceAccess.class, path.substring(path.lastIndexOf(File.separator) + 1), "", false).getAbsolutePath(); +graph.traversal().io(dataFile).read(); } } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/81870168/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/TestHelper.java -- diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/TestHelper.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/TestHelper.java index 38d9a25..cda32e2 100644 --- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/TestHelper.java +++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/TestHelper.java @@ -152,17 +152,27 @@ public final class TestHelper { * {@link TestHelper#makeTestDataPath} in a subdirectory called {@code temp/resources}. */ public static File generateTempFileFromResource(final Class graphClass, final Class resourceClass, final String resourceName, final String extension) throws IOException { +return generateTempFileFromResource(graphClass, resourceClass, resourceName, extension, true); +} + +/** + * Copies a file stored as part of a resource to the file system in the
[35/50] [abbrv] tinkerpop git commit: Added dev notes for python environment CTR
Added dev notes for python environment CTR Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/e3018fb2 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/e3018fb2 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/e3018fb2 Branch: refs/heads/TINKERPOP-1990 Commit: e3018fb2e98df62a21272446f63cea5ee550408e Parents: 00a085a Author: Stephen Mallette Authored: Tue Jul 31 08:35:06 2018 -0400 Committer: Stephen Mallette Committed: Tue Jul 31 08:35:06 2018 -0400 -- docs/src/dev/developer/development-environment.asciidoc | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e3018fb2/docs/src/dev/developer/development-environment.asciidoc -- diff --git a/docs/src/dev/developer/development-environment.asciidoc b/docs/src/dev/developer/development-environment.asciidoc index cde81c1..779f642 100644 --- a/docs/src/dev/developer/development-environment.asciidoc +++ b/docs/src/dev/developer/development-environment.asciidoc @@ -74,13 +74,14 @@ The site will be generated to the `target/site/home` directory. [[python-environment]] === Python Environment -As of TinkerPop 3.2.2, the build optionally requires link:https://www.python.org/[Python 2.x] to work with the -`gremlin-python` module. If Python is not installed, TinkerPop will still build with Maven, but native Python tests and +As of TinkerPop 3.2.2, the build optionally requires link:https://www.python.org/[Python] to build the `gremlin-python` +module. If Python is not installed, TinkerPop will still build with Maven, but native Python tests and Java tests that require Python code will be skipped. Developers should also install link:https://pypi.python.org/pypi/pip[pip] and link:https://virtualenv.pypa.io/en/stable/[virtualenv] (version 15.0.2 - older versions may cause build failures). -Once the Python environment is established, the full building and testing of `gremlin-python` may commence. It can be -done manually from the command line with: +The build expects two Python executables `python` and `python3` where `python` maps to 2.7.6 and `python3` is 3.4.3 or +higher. Once the Python environment is established, the full building and testing of `gremlin-python` may commence. It +can be done manually from the command line with: [source,text] mvn clean install -Pglv-python
[07/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Got read/write() tests running for OLAP
TINKERPOP-1996 Got read/write() tests running for OLAP Introduced new Graph.Features to provider better separation between graph mutation features and graph loading features - they are two different things as demonstrated by io(). Fixed HadoopIoStep/Strategy so that they properly handle the different input/output format types expected. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/f148e933 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/f148e933 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/f148e933 Branch: refs/heads/TINKERPOP-1990 Commit: f148e9331a945e0f4f707ea937b610e5902701c7 Parents: bd275a7 Author: Stephen Mallette Authored: Fri Jul 13 15:17:17 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 13:40:10 2018 -0400 -- .../tinkerpop/gremlin/structure/Graph.java | 27 + .../gremlin/AbstractGraphProvider.java | 18 -- .../gremlin/process/ProcessComputerSuite.java | 4 ++ .../process/traversal/step/map/ReadTest.java| 21 --- .../process/traversal/step/map/WriteTest.java | 14 - .../traversal/step/map/HadoopIoStep.java| 60 +++- .../traversal/strategy/HadoopIoStrategy.java| 2 +- .../gremlin/hadoop/structure/HadoopGraph.java | 16 ++ 8 files changed, 142 insertions(+), 20 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f148e933/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java index f1fc54a..f62b897 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java @@ -23,6 +23,8 @@ import org.apache.tinkerpop.gremlin.process.computer.GraphComputer; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; import org.apache.tinkerpop.gremlin.process.traversal.TraversalSource; import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; +import org.apache.tinkerpop.gremlin.structure.io.GraphReader; +import org.apache.tinkerpop.gremlin.structure.io.GraphWriter; import org.apache.tinkerpop.gremlin.structure.io.Io; import org.apache.tinkerpop.gremlin.structure.io.IoRegistry; import org.apache.tinkerpop.gremlin.structure.util.FeatureDescriptor; @@ -439,6 +441,8 @@ public interface Graph extends AutoCloseable, Host { public static final String FEATURE_PERSISTENCE = "Persistence"; public static final String FEATURE_THREADED_TRANSACTIONS = "ThreadedTransactions"; public static final String FEATURE_CONCURRENT_ACCESS = "ConcurrentAccess"; +public static final String FEATURE_IO_READ = "IoRead"; +public static final String FEATURE_IO_WRITE = "IoWrite"; /** * Determines if the {@code Graph} implementation supports {@link GraphComputer} based processing. @@ -489,6 +493,29 @@ public interface Graph extends AutoCloseable, Host { } /** + * Determines if the {@code Graph} implementations supports read operations as executed with the + * {@link GraphTraversalSource#io(String)} step. Graph implementations will generally support this by + * default as any graph that can support direct mutation through the Structure API will by default + * accept data from the standard TinkerPop {@link GraphReader} implementations. However, some graphs like + * {@code HadoopGraph} don't accept direct mutations but can still do reads from that {@code io()} step. + */ +@FeatureDescriptor(name = FEATURE_IO_READ) +public default boolean supportsIoRead() { +return true; +} + +/** + * Determines if the {@code Graph} implementations supports write operations as executed with the + * {@link GraphTraversalSource#io(String)} step. Graph implementations will generally support this by + * default given the standard TinkerPop {@link GraphWriter} implementations. However, some graphs like + * {@code HadoopGraph} will use a different approach to handle writes. + */ +@FeatureDescriptor(name = FEATURE_IO_WRITE) +public default boolean supportsIoWrite() { +return true; +} + +/** * Gets the features related to "graph sideEffects" operation. */ public default
[29/50] [abbrv] tinkerpop git commit: Merge branch 'tp32' into tp33
Merge branch 'tp32' into tp33 Conflicts: gremlin-server/src/main/bin/gremlin-server.sh Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/88b6e143 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/88b6e143 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/88b6e143 Branch: refs/heads/TINKERPOP-1990 Commit: 88b6e143dd81134074c17e5fa098f03972514273 Parents: 7985106 2b045f3 Author: Stephen Mallette Authored: Mon Jul 30 07:16:27 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 07:16:27 2018 -0400 -- gremlin-server/src/main/bin/gremlin-server.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/88b6e143/gremlin-server/src/main/bin/gremlin-server.sh -- diff --cc gremlin-server/src/main/bin/gremlin-server.sh index fec465c,06b065a..a09e10c --- a/gremlin-server/src/main/bin/gremlin-server.sh +++ b/gremlin-server/src/main/bin/gremlin-server.sh @@@ -90,191 -49,14 +90,191 @@@ els fi # Set Java options -if [ "$JAVA_OPTIONS" = "" ] ; then +if [[ "$JAVA_OPTIONS" = "" ]] ; then - JAVA_OPTIONS="-Xms32m -Xmx512m" + JAVA_OPTIONS="-Xms512m -Xmx4096m" fi -# Execute the application and return its exit code -if [ "$1" = "-i" ]; then - shift - exec $JAVA -Dlog4j.configuration=conf/log4j-server.properties $JAVA_OPTIONS -cp $CP:$CLASSPATH org.apache.tinkerpop.gremlin.server.util.GremlinServerInstall "$@" -else - exec $JAVA -Dlog4j.configuration=conf/log4j-server.properties $JAVA_OPTIONS -cp $CP:$CLASSPATH org.apache.tinkerpop.gremlin.server.GremlinServer "$@" -fi +# Build Java CLASSPATH +CP="$GREMLIN_HOME/conf/" +CP="$CP":$( echo $GREMLIN_HOME/lib/*.jar . | sed 's/ /:/g') +CP="$CP":$( find -L "$GREMLIN_HOME"/ext -mindepth 1 -maxdepth 1 -type d | \ +sort | sed 's/$/\/plugin\/*/' | tr '\n' ':' ) + +CLASSPATH="${CLASSPATH:-}:$CP" + +GREMLIN_SERVER_CMD=org.apache.tinkerpop.gremlin.server.GremlinServer +GREMLIN_INSTALL_CMD=org.apache.tinkerpop.gremlin.server.util.GremlinServerInstall + + +isRunning() { + if [[ -r "$PID_FILE" ]] ; then +PID=$(cat "$PID_FILE") +ps -p "$PID" &> /dev/null +return $? + else +return 1 + fi +} + +status() { + isRunning + RUNNING=$? +if [[ $RUNNING -gt 0 ]]; then + echo Server not running +else + echo Server running with PID $(cat "$PID_FILE") +fi +} + +stop() { + isRunning + RUNNING=$? + if [[ $RUNNING -gt 0 ]]; then +echo Server not running +rm -f "$PID_FILE" + else +kill "$PID" &> /dev/null || { echo "Unable to kill server [$PID]"; exit 1; } +for i in $(seq 1 60); do + ps -p "$PID" &> /dev/null || { echo "Server stopped [$PID]"; rm -f "$PID_FILE"; return 0; } + [[ $i -eq 30 ]] && kill "$PID" &> /dev/null + sleep 1 +done +echo "Unable to kill server [$PID]"; +exit 1; + fi +} + +start() { + isRunning + RUNNING=$? + if [[ $RUNNING -eq 0 ]]; then +echo Server already running with PID $(cat "$PID_FILE"). +exit 1 + fi + + if [[ -z "$RUNAS" ]]; then + +mkdir -p "$LOG_DIR" &>/dev/null +if [[ ! -d "$LOG_DIR" ]]; then + echo ERROR: LOG_DIR $LOG_DIR does not exist and could not be created. + exit 1 +fi + +mkdir -p "$PID_DIR" &>/dev/null +if [[ ! -d "$PID_DIR" ]]; then + echo ERROR: PID_DIR $PID_DIR does not exist and could not be created. + exit 1 +fi + +$JAVA -Dlog4j.configuration=$LOG4J_CONF $JAVA_OPTIONS -cp $CP:$CLASSPATH $GREMLIN_SERVER_CMD "$GREMLIN_YAML" >> "$LOG_FILE" 2>&1 & +PID=$! +disown $PID +echo $PID > "$PID_FILE" + else + +su -c "mkdir -p $LOG_DIR &>/dev/null" "$RUNAS" +if [[ ! -d "$LOG_DIR" ]]; then + echo ERROR: LOG_DIR $LOG_DIR does not exist and could not be created. + exit 1 +fi + +su -c "mkdir -p $PID_DIR &>/dev/null" "$RUNAS" +if [[ ! -d "$PID_DIR" ]]; then + echo ERROR: PID_DIR $PID_DIR does not exist and could not be created. + exit 1 +fi + +su -c "$JAVA -Dlog4j.configuration=$LOG4J_CONF $JAVA_OPTIONS -cp $CP:$CLASSPATH $GREMLIN_SERVER_CMD \"$GREMLIN_YAML\" >> \"$LOG_FILE\" 2>&1 & echo \$! " "$RUNAS" > "$PID_FILE" +chown "$RUNAS" "$PID_FILE" + fi + + isRunning + RUNNING=$? + if [[ $RUNNING -eq 0 ]]; then +echo Server started $(cat "$PID_FILE"). +exit 0 + else +echo Server failed +exit 1 + fi + +} + +startForeground() { + isRunning + RUNNING=$? + if [[ $RUNNING -eq 0 ]]; then +echo Server already running with PID $(cat "$PID_FILE"). +exit 1 + fi + + if [[ -z "$RUNAS" ]]; then +$JAVA
[14/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Fixed bad test assertions after last body of changes.
TINKERPOP-1996 Fixed bad test assertions after last body of changes. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/94233970 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/94233970 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/94233970 Branch: refs/heads/TINKERPOP-1990 Commit: 94233970f8dc7c5d7b8fdc4b9e7bdbf0ff905c25 Parents: ff71c6a Author: Stephen Mallette Authored: Fri Jul 20 06:31:42 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 06:31:42 2018 -0400 -- .../gremlin/process/traversal/step/map/WriteTest.java | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/94233970/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/WriteTest.java -- diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/WriteTest.java b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/WriteTest.java index 55e6f9c..9798095 100644 --- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/WriteTest.java +++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/WriteTest.java @@ -65,7 +65,7 @@ public abstract class WriteTest extends AbstractGremlinProcessTest { final Traversal traversal = get_g_io_writeXkryoX(fileToWrite); printTraversalForm(traversal); -assertFalse(traversal.hasNext()); +traversal.iterate(); assertThat(f.length() > 0, is(true)); } @@ -81,7 +81,7 @@ public abstract class WriteTest extends AbstractGremlinProcessTest { final Traversal traversal = get_g_io_write_withXwriter_gryoX(fileToWrite); printTraversalForm(traversal); -assertFalse(traversal.hasNext()); +traversal.iterate(); assertThat(f.length() > 0, is(true)); } @@ -97,7 +97,7 @@ public abstract class WriteTest extends AbstractGremlinProcessTest { final Traversal traversal = get_g_io_writeXjsonX(fileToWrite); printTraversalForm(traversal); -assertFalse(traversal.hasNext()); +traversal.iterate(); assertThat(f.length() > 0, is(true)); } @@ -113,7 +113,7 @@ public abstract class WriteTest extends AbstractGremlinProcessTest { final Traversal traversal = get_g_io_write_withXwriter_graphsonX(fileToWrite); printTraversalForm(traversal); -assertFalse(traversal.hasNext()); +traversal.iterate(); assertThat(f.length() > 0, is(true)); } @@ -129,7 +129,7 @@ public abstract class WriteTest extends AbstractGremlinProcessTest { final Traversal traversal = get_g_io_writeXxmlX(fileToWrite); printTraversalForm(traversal); -assertFalse(traversal.hasNext()); +traversal.iterate(); assertThat(f.length() > 0, is(true)); }
[15/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Added some docs around IO.registry
TINKERPOP-1996 Added some docs around IO.registry Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/51dc8212 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/51dc8212 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/51dc8212 Branch: refs/heads/TINKERPOP-1990 Commit: 51dc82122af6b8d8c783de58a0451b8b2071c051 Parents: 9423397 Author: Stephen Mallette Authored: Fri Jul 20 07:15:07 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 20 07:15:07 2018 -0400 -- docs/src/reference/the-traversal.asciidoc | 30 ++ 1 file changed, 30 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/51dc8212/docs/src/reference/the-traversal.asciidoc -- diff --git a/docs/src/reference/the-traversal.asciidoc b/docs/src/reference/the-traversal.asciidoc index cd2f5f2..c3b9300 100644 --- a/docs/src/reference/the-traversal.asciidoc +++ b/docs/src/reference/the-traversal.asciidoc @@ -1114,6 +1114,36 @@ IMPORTANT: Remote Gremlin Console users or Gremlin Language Variant (GLV) users the `io()` step should recall that their `read()` or `write()` operation will occur on the server and not locally and therefore the file specified for import/export must be something accessible by the server. +GraphSON and Gryo formats are extensible allowing users and graph providers to extend supported serialization options. +These extensions are exposed through `IoRegistry` implementations. To apply an `IoRegistry` use the `with()` option +and the `IO.registry` key, where the value is either an actual `IoRegistry` instance or the fully qualified class +name of one. + +[source,java] + +g.io(someInputFile). +with(IO.reader, IO.gryo). +with(IO.registry, TinkerIoRegistryV3d0.instance()) + read().iterate() +g.io(someOutputFile). +with(IO.writer,IO.graphson). +with(IO.registry, "org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV3d0") + write().iterate() + + +GLVs will obviously always be forced to use the latter form as they can't explicitly create an instance of an +`IoRegistry` to pass to the server (nor are `IoRegistry` instances necessarily serializable). + +The version of the formats (e.g. GraphSON 2.0 or 3.0) utilized by `io()` is determined entirely by the `IO.reader` and +`IO.writer` configurations or their defaults. The defaults will always be the latest version for the current release +of TinkerPop. It is also possible for graph providers to override these defaults, so consult the documentation of the +underlying graph database in use for any details on that. + +For more advanced configuration of `GraphReader` and `GraphWriter` operations (e.g. normalized output for GraphSON, +disabling class registrations for Gryo, etc.) then construct the appropriate `GraphReader` and `GraphWriter` using +the `build()` method on their implementations and use it directly. It can be passed directly to the `IO.reader` or +`IO.writer` options. Obviously, these are JVM based operations and thus not available to GLVs as portable features. + [[_graphml_reader_writer]] [[graphml]] GraphML
[01/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Fixed a bad method call for Configuring steps [Forced Update!]
Repository: tinkerpop Updated Branches: refs/heads/TINKERPOP-1990 7f5e71eda -> d8a3626fd (forced update) TINKERPOP-1996 Fixed a bad method call for Configuring steps Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/9e4da014 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/9e4da014 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/9e4da014 Branch: refs/heads/TINKERPOP-1990 Commit: 9e4da0149247a50277e2a468b0becf892426ce2e Parents: ae79637 Author: Stephen Mallette Authored: Fri Jul 13 11:37:02 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 13:40:09 2018 -0400 -- .../process/computer/traversal/strategy/HadoopIoStrategy.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/9e4da014/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/strategy/HadoopIoStrategy.java -- diff --git a/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/strategy/HadoopIoStrategy.java b/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/strategy/HadoopIoStrategy.java index 89ee04b..6d3899e 100644 --- a/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/strategy/HadoopIoStrategy.java +++ b/hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/process/computer/traversal/strategy/HadoopIoStrategy.java @@ -55,7 +55,7 @@ public final class HadoopIoStrategy extends AbstractTraversalStrategy -hadoopIoStep.configure(null, kv.getKey(), kv.getValue()) +hadoopIoStep.configure(kv.getKey(), kv.getValue()) ); TraversalHelper.replaceStep((Step) readWriting, hadoopIoStep, traversal);
[30/50] [abbrv] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/4dccf144 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/4dccf144 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/4dccf144 Branch: refs/heads/TINKERPOP-1990 Commit: 4dccf144c71603612ce976827fd24c3eead4fb24 Parents: 6358715 88b6e14 Author: Stephen Mallette Authored: Mon Jul 30 07:16:33 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 07:16:33 2018 -0400 -- gremlin-server/src/main/bin/gremlin-server.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --
[05/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Added docs for io()
TINKERPOP-1996 Added docs for io() Killed all the old IO documentation that utilized the GraphReader/Writer classes directly as well as the Graph.io() method that is now deprecated. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/62175c22 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/62175c22 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/62175c22 Branch: refs/heads/TINKERPOP-1990 Commit: 62175c228b77bdbda96c11015f2974828df8f3aa Parents: 576649f Author: Stephen Mallette Authored: Fri Jul 13 17:31:46 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 13:40:10 2018 -0400 -- docs/src/reference/the-graph.asciidoc | 370 - docs/src/reference/the-traversal.asciidoc | 140 ++ 2 files changed, 140 insertions(+), 370 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/62175c22/docs/src/reference/the-graph.asciidoc -- diff --git a/docs/src/reference/the-graph.asciidoc b/docs/src/reference/the-graph.asciidoc index 1bcc96f..e9305b2 100644 --- a/docs/src/reference/the-graph.asciidoc +++ b/docs/src/reference/the-graph.asciidoc @@ -345,376 +345,6 @@ In the above case, the call to `graph.tx().createThreadedTx()` creates a new `Gr `ThreadLocal` transaction, thus allowing each thread to operate on it in the same context. In this case, there would be three separate vertices persisted to the `Graph`. -== Gremlin I/O - -image:gremlin-io.png[width=250,float=right] The task of getting data in and out of `Graph` instances is the job of -the Gremlin I/O packages. Gremlin I/O provides two interfaces for reading and writing `Graph` instances: `GraphReader` -and `GraphWriter`. These interfaces expose methods that support: - -* Reading and writing an entire `Graph` -* Reading and writing a `Traversal` as adjacency list format -* Reading and writing a single `Vertex` (with and without associated `Edge` objects) -* Reading and writing a single `Edge` -* Reading and writing a single `VertexProperty` -* Reading and writing a single `Property` -* Reading and writing an arbitrary `Object` - -In all cases, these methods operate in the currency of `InputStream` and `OutputStream` objects, allowing graphs and -their related elements to be written to and read from files, byte arrays, etc. The `Graph` interface offers the `io` -method, which provides access to "reader/writer builder" objects that are pre-configured with serializers provided by -the `Graph`, as well as helper methods for the various I/O capabilities. Unless there are very advanced requirements -for the serialization process, it is always best to utilize the methods on the `Io` interface to construct -`GraphReader` and `GraphWriter` instances, as the implementation may provide some custom settings that would otherwise -have to be configured manually by the user to do the serialization. - -It is up to the implementations of the `GraphReader` and `GraphWriter` interfaces to choose the methods they -implement and the manner in which they work together. The only characteristic enforced and expected is that the write -methods should produce output that is compatible with the corresponding read method. For example, the output of -`writeVertices` should be readable as input to `readVertices` and the output of `writeProperty` should be readable as -input to `readProperty`. - -NOTE: Additional documentation for TinkerPop IO formats can be found in the link:http://tinkerpop.apache.org/docs/x.y.z/dev/io/[IO Reference]. - -=== GraphML Reader/Writer - -image:gremlin-graphml.png[width=350,float=left] The link:http://graphml.graphdrawing.org/[GraphML] file format is a -common XML-based representation of a graph. It is widely supported by graph-related tools and libraries making it a -solid interchange format for TinkerPop. In other words, if the intent is to work with graph data in conjunction with -applications outside of TinkerPop, GraphML may be the best choice to do that. Common use cases might be: - -* Generate a graph using link:https://networkx.github.io/[NetworkX], export it with GraphML and import it to TinkerPop. -* Produce a subgraph and export it to GraphML to be consumed by and visualized in link:https://gephi.org/[Gephi]. -* Migrate the data of an entire graph to a different graph database not supported by TinkerPop. - -As GraphML is a specification for the serialization of an entire graph and not the individual elements of a graph, -methods that support input and output of single vertices, edges, etc. are not supported. - -WARNING: GraphML is a "lossy" format in that it only supports primitive values for properties and does not have -support for `Graph` variables.
[02/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Deprecated Graph.io() and related infrastructure.
TINKERPOP-1996 Deprecated Graph.io() and related infrastructure. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/ae796378 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/ae796378 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/ae796378 Branch: refs/heads/TINKERPOP-1990 Commit: ae796378e07925f9385f3ec65c10022b59aab8b5 Parents: 328737a Author: Stephen Mallette Authored: Thu Jul 12 14:33:03 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 13:40:09 2018 -0400 -- CHANGELOG.asciidoc| 2 ++ .../main/java/org/apache/tinkerpop/gremlin/structure/Graph.java | 3 +++ .../main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java | 3 +++ .../java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java| 3 +++ .../apache/tinkerpop/gremlin/structure/io/graphml/GraphMLIo.java | 3 +++ .../tinkerpop/gremlin/structure/io/graphson/GraphSONIo.java | 3 +++ .../org/apache/tinkerpop/gremlin/structure/io/gryo/GryoIo.java| 3 +++ 7 files changed, 20 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ae796378/CHANGELOG.asciidoc -- diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index c0cd27d..bc78bbe 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -27,6 +27,8 @@ This release also includes changes from <>. * Bumped to Netty 4.1.25. * Bumped to Spark 2.3.1. +* Added the `io()` start step and `read()` and `write()` termination steps to the Gremlin language. +* Deprecated `Graph.io()` and related infrastructure. * Moved `Parameterizing` interface to the `org.apache.tinkerpop.gremlin.process.traversal.step` package with other marker interfaces of its type. * Replaced `Parameterizing.addPropertyMutations()` with `Configuring.configure()`. * Changed interface hierarchy for `Parameterizing` and `Mutating` interfaces as they are tightly related. http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ae796378/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java index dc14cc6..f1fc54a 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java @@ -300,7 +300,10 @@ public interface Graph extends AutoCloseable, Host { * For those graphs that do not need to register any custom serializers, the default implementation should suffice. * If the default is overridden, take care to register the current graph via the * {@link org.apache.tinkerpop.gremlin.structure.io.Io.Builder#graph(Graph)} method. + * + * @deprecated As of release 3.4.0, replaced by {@link GraphTraversalSource#io(String)}. */ +@Deprecated public default I io(final Io.Builder builder) { return (I) builder.graph(this).create(); } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ae796378/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java index bae56c5..0971e31 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Io.java @@ -18,6 +18,7 @@ */ package org.apache.tinkerpop.gremlin.structure.io; +import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; import org.apache.tinkerpop.gremlin.structure.Graph; import java.io.IOException; @@ -31,7 +32,9 @@ import java.util.function.Consumer; * internal {@link Mapper} (if the format has such capability). * * @author Stephen Mallette (http://stephen.genoprime.com) + * @deprecated As of release 3.4.0, replaced by {@link GraphTraversalSource#io(String)}. */ +@Deprecated public interface Io { /** http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ae796378/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCore.java index 9d9ad60..a357184 100644 ---
[46/50] [abbrv] tinkerpop git commit: TINKERPOP-1990 Implemented `ShortestPathVertexProgram` and `ShortestPathVertexProgramStep`.
TINKERPOP-1990 Implemented `ShortestPathVertexProgram` and `ShortestPathVertexProgramStep`. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/e1ba81b4 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/e1ba81b4 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/e1ba81b4 Branch: refs/heads/TINKERPOP-1990 Commit: e1ba81b4b25c562050fae10b52ab67c5dec96232 Parents: 69b6f96 Author: Daniel Kuppitz Authored: Wed May 23 08:46:34 2018 -0400 Committer: Daniel Kuppitz Committed: Wed Aug 1 12:26:30 2018 -0700 -- CHANGELOG.asciidoc | 1 + docs/src/reference/the-graphcomputer.asciidoc | 51 ++ docs/src/reference/the-traversal.asciidoc | 56 ++ .../tinkerpop/gremlin/jsr223/CoreImports.java | 4 + .../search/path/ShortestPathVertexProgram.java | 581 +++ .../traversal/step/map/ShortestPath.java| 108 .../step/map/ShortestPathVertexProgramStep.java | 174 ++ .../gremlin/process/remote/RemoteGraph.java | 8 + .../gremlin/process/traversal/Traversal.java| 16 +- .../traversal/dsl/graph/GraphTraversal.java | 20 + .../traversal/lambda/ColumnTraversal.java | 8 + .../traversal/lambda/ConstantTraversal.java | 9 +- .../traversal/lambda/ElementValueTraversal.java | 10 +- .../traversal/lambda/IdentityTraversal.java | 7 +- .../process/traversal/lambda/LoopTraversal.java | 6 + .../traversal/lambda/TokenTraversal.java| 8 + .../process/traversal/lambda/TrueTraversal.java | 5 + .../structure/io/graphson/GraphSONModule.java | 2 +- .../gremlin/structure/io/gryo/GryoVersion.java | 12 +- .../structure/io/gryo/UtilSerializers.java | 15 + .../gremlin/structure/util/Attachable.java | 3 +- .../traversal/dsl/graph/GraphTraversalTest.java | 2 +- .../Process/Traversal/GraphTraversal.cs | 11 +- .../lib/process/graph-traversal.js | 10 + .../gremlin_python/process/graph_traversal.py | 4 + .../tinkerpop/gremlin/AbstractGremlinTest.java | 6 +- .../process/AbstractGremlinProcessTest.java | 11 +- .../gremlin/process/ProcessComputerSuite.java | 5 + .../search/path/ShortestPathTestHelper.java | 100 .../path/ShortestPathVertexProgramTest.java | 297 ++ .../traversal/step/map/ShortestPathTest.java| 353 +++ pom.xml | 3 + 32 files changed, 1880 insertions(+), 26 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e1ba81b4/CHANGELOG.asciidoc -- diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 430f52d..4ffc6b7 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -25,6 +25,7 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima This release also includes changes from <>. +* Implemented `ShortestPathVertexProgram` and the `shortestPath()` step. * `AbstractGraphProvider` uses `g.io()` for loading test data. * Added the `io()` start step and `read()` and `write()` termination steps to the Gremlin language. * Added `GraphFeatures.supportsIoRead()` and `GraphFeatures.supportsIoWrite()`. http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e1ba81b4/docs/src/reference/the-graphcomputer.asciidoc -- diff --git a/docs/src/reference/the-graphcomputer.asciidoc b/docs/src/reference/the-graphcomputer.asciidoc index 1d7586c..3d7ec00 100644 --- a/docs/src/reference/the-graphcomputer.asciidoc +++ b/docs/src/reference/the-graphcomputer.asciidoc @@ -403,6 +403,57 @@ g.V().peerPressure().by('cluster').valueMap() g.V().peerPressure().by(outE('knows')).by('cluster').valueMap() +[[shortestpathvertexprogram]] +=== ShortestPathVertexProgram + +The `ShortestPathVertexProram` provides an easy way to find shortest non-cyclic paths in the graph. It provides several options to configure +the output format, the start- and end-vertices, the direction, a custom distance function, as well as a distance limitation. By default it just +finds all undirected, shortest paths in the graph. + +[gremlin-groovy,modern] + +spvp = ShortestPathVertexProgram.build().create() <1> +result = graph.compute().program(spvp).submit().get() <2> +result.memory().get(ShortestPathVertexProgram.SHORTEST_PATHS) <3> + + +<1> Create a `ShortestPathVertexProgram` with its default configuration. +<2> Execute the `ShortestPathVertexProgram`. +<3> Get all shortest paths from the results memory. + +[gremlin-groovy,modern] + +spvp = ShortestPathVertexProgram.build().includeEdges(true).create() <1> +result = graph.compute().program(spvp).submit().get() <2>
[11/50] [abbrv] tinkerpop git commit: TINKERPOP-1996 Enabled feature coverage checks for GLV tests on read()/write()
TINKERPOP-1996 Enabled feature coverage checks for GLV tests on read()/write() Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/a580b6fd Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/a580b6fd Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/a580b6fd Branch: refs/heads/TINKERPOP-1990 Commit: a580b6fda232ce6dd035e4c261a53d0f0dc69f83 Parents: 048ea21 Author: Stephen Mallette Authored: Thu Jul 19 14:08:25 2018 -0400 Committer: Stephen Mallette Committed: Thu Jul 19 14:08:25 2018 -0400 -- .../apache/tinkerpop/gremlin/process/FeatureCoverageTest.java | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/a580b6fd/gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/process/FeatureCoverageTest.java -- diff --git a/gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/process/FeatureCoverageTest.java b/gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/process/FeatureCoverageTest.java index 503df77..5739629 100644 --- a/gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/process/FeatureCoverageTest.java +++ b/gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/process/FeatureCoverageTest.java @@ -56,11 +56,13 @@ import org.apache.tinkerpop.gremlin.process.traversal.step.map.OrderTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.PathTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.ProjectTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.PropertiesTest; +import org.apache.tinkerpop.gremlin.process.traversal.step.map.ReadTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.SumTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.UnfoldTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.ValueMapTest; import org.apache.tinkerpop.gremlin.process.traversal.step.map.VertexTest; +import org.apache.tinkerpop.gremlin.process.traversal.step.map.WriteTest; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.AggregateTest; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.GroupCountTest; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.GroupTest; @@ -158,10 +160,12 @@ public class FeatureCoverageTest { GroupCountTest.class, GroupTest.class, InjectTest.class, +ReadTest.class, SackTest.class, SideEffectCapTest.class, //SideEffectTest.class, -StoreTest.class); +StoreTest.class, +WriteTest.class); // SubgraphTest.class, // TreeTest.class);
[1/3] tinkerpop git commit: CTR: Fixed build by adding python3-dev dependency to the base image Dockerfile
Repository: tinkerpop Updated Branches: refs/heads/master b183edd10 -> 69b6f96d7 CTR: Fixed build by adding python3-dev dependency to the base image Dockerfile Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/7fce1377 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/7fce1377 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/7fce1377 Branch: refs/heads/master Commit: 7fce1377f70190954f858901c7a3698dbd17cf82 Parents: e3018fb Author: Daniel Kuppitz Authored: Wed Aug 1 11:26:51 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 11:26:51 2018 -0700 -- docker/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7fce1377/docker/Dockerfile -- diff --git a/docker/Dockerfile b/docker/Dockerfile index dbc06ef..9ae74aa 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -30,7 +30,7 @@ RUN apt-get update \ && apt-get update \ && apt-get install -y oracle-java8-installer curl gawk git maven openssh-server subversion zip \ && sh -c 'curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > /etc/apt/trusted.gpg.d/microsoft.gpg' \ -&& apt-get install -y --force-yes dotnet-sdk-2.1.101 python python-dev python-pip build-essential mono-devel \ +&& apt-get install -y --force-yes dotnet-sdk-2.1.101 python python-dev python3-dev python-pip build-essential mono-devel \ && pip install virtualenv virtualenvwrapper \ && pip install --upgrade pip \ && rm -rf /var/lib/apt/lists/* /var/cache/oracle-jdk8-installer
[2/2] tinkerpop git commit: Merge branch 'tp32' into tp33
Merge branch 'tp32' into tp33 Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/8af78372 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/8af78372 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/8af78372 Branch: refs/heads/tp33 Commit: 8af7837286c7eba82c831576a3291c77bfb727e9 Parents: dfe79c2 7fce137 Author: Daniel Kuppitz Authored: Wed Aug 1 11:27:04 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 11:27:04 2018 -0700 -- docker/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --
[3/3] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/69b6f96d Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/69b6f96d Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/69b6f96d Branch: refs/heads/master Commit: 69b6f96d7ac8aa706ce65dee436d7038b8b3eaac Parents: b183edd 8af7837 Author: Daniel Kuppitz Authored: Wed Aug 1 11:27:17 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 11:27:17 2018 -0700 -- docker/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --
tinkerpop git commit: CTR: Fixed build by adding python3-dev dependency to the base image Dockerfile
Repository: tinkerpop Updated Branches: refs/heads/tp32 e3018fb2e -> 7fce1377f CTR: Fixed build by adding python3-dev dependency to the base image Dockerfile Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/7fce1377 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/7fce1377 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/7fce1377 Branch: refs/heads/tp32 Commit: 7fce1377f70190954f858901c7a3698dbd17cf82 Parents: e3018fb Author: Daniel Kuppitz Authored: Wed Aug 1 11:26:51 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 11:26:51 2018 -0700 -- docker/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7fce1377/docker/Dockerfile -- diff --git a/docker/Dockerfile b/docker/Dockerfile index dbc06ef..9ae74aa 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -30,7 +30,7 @@ RUN apt-get update \ && apt-get update \ && apt-get install -y oracle-java8-installer curl gawk git maven openssh-server subversion zip \ && sh -c 'curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > /etc/apt/trusted.gpg.d/microsoft.gpg' \ -&& apt-get install -y --force-yes dotnet-sdk-2.1.101 python python-dev python-pip build-essential mono-devel \ +&& apt-get install -y --force-yes dotnet-sdk-2.1.101 python python-dev python3-dev python-pip build-essential mono-devel \ && pip install virtualenv virtualenvwrapper \ && pip install --upgrade pip \ && rm -rf /var/lib/apt/lists/* /var/cache/oracle-jdk8-installer
[1/2] tinkerpop git commit: CTR: Fixed build by adding python3-dev dependency to the base image Dockerfile
Repository: tinkerpop Updated Branches: refs/heads/tp33 dfe79c24f -> 8af783728 CTR: Fixed build by adding python3-dev dependency to the base image Dockerfile Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/7fce1377 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/7fce1377 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/7fce1377 Branch: refs/heads/tp33 Commit: 7fce1377f70190954f858901c7a3698dbd17cf82 Parents: e3018fb Author: Daniel Kuppitz Authored: Wed Aug 1 11:26:51 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 11:26:51 2018 -0700 -- docker/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7fce1377/docker/Dockerfile -- diff --git a/docker/Dockerfile b/docker/Dockerfile index dbc06ef..9ae74aa 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -30,7 +30,7 @@ RUN apt-get update \ && apt-get update \ && apt-get install -y oracle-java8-installer curl gawk git maven openssh-server subversion zip \ && sh -c 'curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > /etc/apt/trusted.gpg.d/microsoft.gpg' \ -&& apt-get install -y --force-yes dotnet-sdk-2.1.101 python python-dev python-pip build-essential mono-devel \ +&& apt-get install -y --force-yes dotnet-sdk-2.1.101 python python-dev python3-dev python-pip build-essential mono-devel \ && pip install virtualenv virtualenvwrapper \ && pip install --upgrade pip \ && rm -rf /var/lib/apt/lists/* /var/cache/oracle-jdk8-installer
[2/3] tinkerpop git commit: Merge branch 'tp32' into tp33
Merge branch 'tp32' into tp33 Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/8af78372 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/8af78372 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/8af78372 Branch: refs/heads/master Commit: 8af7837286c7eba82c831576a3291c77bfb727e9 Parents: dfe79c2 7fce137 Author: Daniel Kuppitz Authored: Wed Aug 1 11:27:04 2018 -0700 Committer: Daniel Kuppitz Committed: Wed Aug 1 11:27:04 2018 -0700 -- docker/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --
[18/50] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/f50aeb6d Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/f50aeb6d Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/f50aeb6d Branch: refs/heads/TINKERPOP-1878 Commit: f50aeb6d62009845f42d9558b7cc8e576c91cf4e Parents: a2db39c dfe79c2 Author: Robert Dale Authored: Tue Jul 31 13:12:40 2018 -0400 Committer: Robert Dale Committed: Tue Jul 31 13:12:40 2018 -0400 -- pom.xml | 16 +--- 1 file changed, 1 insertion(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/f50aeb6d/pom.xml --
[32/50] tinkerpop git commit: TINKERPOP-1878 Added the plugin for sparql-gremlin
TINKERPOP-1878 Added the plugin for sparql-gremlin Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/a816dec7 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/a816dec7 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/a816dec7 Branch: refs/heads/TINKERPOP-1878 Commit: a816dec72b4d0c1b2cc7c8882e9df8e3bdc44650 Parents: 37f4df3 Author: Stephen Mallette Authored: Mon Jan 29 09:37:15 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- .../sparql/jsr223/SparqlGremlinPlugin.java | 63 ...pache.tinkerpop.gremlin.jsr223.GremlinPlugin | 1 + 2 files changed, 64 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/a816dec7/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/jsr223/SparqlGremlinPlugin.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/jsr223/SparqlGremlinPlugin.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/jsr223/SparqlGremlinPlugin.java new file mode 100644 index 000..8d0115b --- /dev/null +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/jsr223/SparqlGremlinPlugin.java @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.tinkerpop.gremlin.sparql.jsr223; + +import org.apache.tinkerpop.gremlin.jsr223.AbstractGremlinPlugin; +import org.apache.tinkerpop.gremlin.jsr223.DefaultImportCustomizer; +import org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin; +import org.apache.tinkerpop.gremlin.jsr223.ImportCustomizer; +import org.apache.tinkerpop.gremlin.sparql.process.traversal.dsl.sparql.DefaultSparqlTraversal; +import org.apache.tinkerpop.gremlin.sparql.process.traversal.dsl.sparql.SparqlTraversal; +import org.apache.tinkerpop.gremlin.sparql.process.traversal.dsl.sparql.SparqlTraversalSource; +import org.apache.tinkerpop.gremlin.sparql.process.traversal.strategy.SparqlStrategy; + +/** + * {@link GremlinPlugin} implementation for {@code sparql-gremlin} that imports the key classes and interfaces required + * to use SPARQL in TinkerPop. + * + * @author Stephen Mallette (http://stephen.genoprime.com) + */ +public final class SparqlGremlinPlugin extends AbstractGremlinPlugin { + +private static final String NAME = "tinkerpop.sparql"; + +private static final ImportCustomizer imports; + +static { +try { +imports = DefaultImportCustomizer.build().addClassImports( +SparqlTraversalSource.class, +SparqlTraversal.class, +DefaultSparqlTraversal.class, +SparqlStrategy.class).create(); +} catch (Exception ex) { +throw new RuntimeException(ex); +} +} + +private static final SparqlGremlinPlugin instance = new SparqlGremlinPlugin(); + +public SparqlGremlinPlugin() { +super(NAME, imports); +} + +public static SparqlGremlinPlugin instance() { +return instance; +} +} http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/a816dec7/sparql-gremlin/src/main/resources/META-INF/services/org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin -- diff --git a/sparql-gremlin/src/main/resources/META-INF/services/org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin b/sparql-gremlin/src/main/resources/META-INF/services/org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin new file mode 100644 index 000..882054c --- /dev/null +++ b/sparql-gremlin/src/main/resources/META-INF/services/org.apache.tinkerpop.gremlin.jsr223.GremlinPlugin @@ -0,0 +1 @@ +org.apache.tinkerpop.gremlin.sparql.jsr223.SparqlGremlinPlugin \ No newline at end of file
[10/50] tinkerpop git commit: Added a better reason for some OptOut on RemoteGraph CTR
Added a better reason for some OptOut on RemoteGraph CTR Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/00a085aa Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/00a085aa Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/00a085aa Branch: refs/heads/TINKERPOP-1878 Commit: 00a085aa15cfc877222712f8572c89f1c096597c Parents: 2b045f3 Author: Stephen Mallette Authored: Mon Jul 30 11:40:52 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 11:40:52 2018 -0400 -- .../org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/00a085aa/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java index 8a1ab02..4b23475 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java @@ -55,11 +55,11 @@ import java.util.Iterator; @Graph.OptOut( test = "org.apache.tinkerpop.gremlin.process.traversal.step.map.PeerPressureTest", method = "*", -reason = "h") +reason = "https://issues.apache.org/jira/browse/TINKERPOP-1976;) @Graph.OptOut( test = "org.apache.tinkerpop.gremlin.process.traversal.step.map.PageRankTest", method = "*", -reason = "h") +reason = "https://issues.apache.org/jira/browse/TINKERPOP-1976;) @Graph.OptOut( test = "org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.TranslationStrategyProcessTest", method = "*",
[48/50] tinkerpop git commit: Update compilers.asciidoc
Update compilers.asciidoc Updating documentation: - fixed optional clause - updated optional clause limitation while in nesting with union & order-by - went through other examples and limitations Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/fafda957 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/fafda957 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/fafda957 Branch: refs/heads/TINKERPOP-1878 Commit: fafda957f8d22c399f8c463ae249bdb5bcbfab8d Parents: 4a08ffc Author: Harsh Thakkar Authored: Wed Aug 1 11:27:59 2018 +0200 Committer: Stephen Mallette Committed: Wed Aug 1 09:37:35 2018 -0400 -- docs/src/reference/compilers.asciidoc | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/fafda957/docs/src/reference/compilers.asciidoc -- diff --git a/docs/src/reference/compilers.asciidoc b/docs/src/reference/compilers.asciidoc index e015cc8..5b6ab2e 100644 --- a/docs/src/reference/compilers.asciidoc +++ b/docs/src/reference/compilers.asciidoc @@ -206,9 +206,8 @@ WHERE { ?person v:label "person" . ?person v:age ?age . ?person v:name ?name . } The the variable encountered first will be the ordering decider, i.e. since we have `?person` being encountered first, the result set will be ordered according to the `?person` variable (which are vertex id). * OPTIONAL - -currently fixing this... - +In the current implementation, `OPTIONAL` clause doesn't work under nesting with `UNION` clause (i.e. multiple optional clauses with in a union clause) and `ORDER-By` clause (i.e. declaring ordering over triple patterns within optional clauses). Everything else with SPARQL `OPTIONAL` works just fine. + [[examples]] === Examples
[46/50] tinkerpop git commit: TINKERPOP-1878 Minor transpiler doc fixes
TINKERPOP-1878 Minor transpiler doc fixes Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c5b3c4c5 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c5b3c4c5 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c5b3c4c5 Branch: refs/heads/TINKERPOP-1878 Commit: c5b3c4c534c4223f23d980cd13a81a7ebcc3e415 Parents: c322416 Author: Stephen Mallette Authored: Fri Jun 15 08:16:03 2018 -0400 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:07 2018 -0400 -- docs/src/reference/transpilers.asciidoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c5b3c4c5/docs/src/reference/transpilers.asciidoc -- diff --git a/docs/src/reference/transpilers.asciidoc b/docs/src/reference/transpilers.asciidoc index 1bb7728..0483a5d 100644 --- a/docs/src/reference/transpilers.asciidoc +++ b/docs/src/reference/transpilers.asciidoc @@ -131,7 +131,7 @@ The current implementation of SPARQL-Gremlin transpiler (i.e. SPARQL-Gremlin) do * SPARQL queries with variables in the predicate position are not currently covered, with an exception of the following case: -[source,text] +[source,groovy] g.sparql("""SELECT * WHERE { ?x ?y ?z . }""") @@ -141,7 +141,7 @@ SPARQL query has the same number of patterns on both the side of the union opera SPARQL query cannot be mapped using Gremlinator, since a union is executed between different number of graph patterns (two patterns `union` 1 pattern). -[source,text] +[source,groovy] g.sparql("""SELECT * WHERE { {?person e:created ?software . @@ -163,7 +163,7 @@ g.sparql("""SELECT ?age WHERE { Whereas, the following SPARQL query will be invalid: -[source,text] +[source,groovy] g.sparql("""SELECT ?person WHERE { ?person v:label "person" .
[17/50] tinkerpop git commit: dedup build-helper-maven-plugin - CTR
dedup build-helper-maven-plugin - CTR Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/dfe79c24 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/dfe79c24 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/dfe79c24 Branch: refs/heads/TINKERPOP-1878 Commit: dfe79c24f4decb667ad20c729928a50f8cb1ec0b Parents: c49c0cc Author: Robert Dale Authored: Tue Jul 31 13:12:31 2018 -0400 Committer: Robert Dale Committed: Tue Jul 31 13:12:31 2018 -0400 -- pom.xml | 16 +--- 1 file changed, 1 insertion(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/dfe79c24/pom.xml -- diff --git a/pom.xml b/pom.xml index b669872..f2629ad 100644 --- a/pom.xml +++ b/pom.xml @@ -305,6 +305,7 @@ limitations under the License. false + parse-version @@ -407,21 +408,6 @@ limitations under the License. - - -org.codehaus.mojo -build-helper-maven-plugin - 3.0.0 - - -parse-version - -parse-version - -validate - - - org.revapi revapi-maven-plugin
[23/50] tinkerpop git commit: TINKERPOP-1878 Added some Traversal infrastructure
TINKERPOP-1878 Added some Traversal infrastructure This is the start of getting Sparql working as a DSL in TinkerPop Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/eadb8b94 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/eadb8b94 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/eadb8b94 Branch: refs/heads/TINKERPOP-1878 Commit: eadb8b947823878748b09b27e2e5f06a7379ecc2 Parents: b0f8a61 Author: Stephen Mallette Authored: Wed Jan 24 17:04:27 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:32:58 2018 -0400 -- .../dsl/sparql/DefaultSparqlTraversal.java | 55 .../traversal/dsl/sparql/SparqlTraversal.java | 68 + .../dsl/sparql/SparqlTraversalSource.java | 138 +++ .../traversal/strategy/SparqlStrategy.java | 25 4 files changed, 286 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/eadb8b94/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/DefaultSparqlTraversal.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/DefaultSparqlTraversal.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/DefaultSparqlTraversal.java new file mode 100644 index 000..133d4be --- /dev/null +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/DefaultSparqlTraversal.java @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.tinkerpop.gremlin.sparql.process.traversal.dsl.sparql; + +import org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal; +import org.apache.tinkerpop.gremlin.structure.Graph; + +/** + * @author Stephen Mallette (http://stephen.genoprime.com) + */ +public class DefaultSparqlTraversal extends DefaultTraversal implements SparqlTraversal.Admin { + +public DefaultSparqlTraversal() { +super(); +} + +public DefaultSparqlTraversal(final SparqlTraversalSource sparqlTraversalSource) { +super(sparqlTraversalSource); +} + +public DefaultSparqlTraversal(final Graph graph) { +super(graph); +} + +@Override +public SparqlTraversal.Admin asAdmin() { +return this; +} + +@Override +public SparqlTraversal iterate() { +return SparqlTraversal.Admin.super.iterate(); +} + +@Override +public DefaultSparqlTraversal clone() { +return (DefaultSparqlTraversal) super.clone(); +} +} http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/eadb8b94/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversal.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversal.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversal.java new file mode 100644 index 000..efa9489 --- /dev/null +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversal.java @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
[21/50] tinkerpop git commit: TINKERPOP-1878 sparql-gremlin is now building
TINKERPOP-1878 sparql-gremlin is now building Fixed some basic pom.xml issues and resolved dependency conflicts. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/b0f8a612 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/b0f8a612 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/b0f8a612 Branch: refs/heads/TINKERPOP-1878 Commit: b0f8a612c5026421e3f67311e039afcce0f64964 Parents: c4244ce Author: Stephen Mallette Authored: Wed Jan 24 15:23:57 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:32:58 2018 -0400 -- pom.xml| 18 sparql-gremlin/pom.xml | 52 ++--- 2 files changed, 39 insertions(+), 31 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/b0f8a612/pom.xml -- diff --git a/pom.xml b/pom.xml index 4badaa1..197d8dc 100644 --- a/pom.xml +++ b/pom.xml @@ -108,6 +108,18 @@ limitations under the License. twil...@gmail.com https://github.com/twilmes + +Harsh Thakkar +hars...@gmail.com +University of Bonn +http://harshthakkar.in + + +Dharmen Punjani +dharmen.punj...@gmail.com +National and Kapodistrian University of Athens + http://wdaqua.eu/students/dharmen-punjani + 3.1.0 @@ -124,6 +136,7 @@ limitations under the License. hadoop-gremlin spark-gremlin neo4j-gremlin +sparql-gremlin gremlin-driver gremlin-console gremlin-server @@ -680,6 +693,11 @@ limitations under the License. +org.apache.httpcomponents +httpclient +4.5.1 + + commons-httpclient commons-httpclient 3.1 http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/b0f8a612/sparql-gremlin/pom.xml -- diff --git a/sparql-gremlin/pom.xml b/sparql-gremlin/pom.xml index e134d31..d38a85d 100644 --- a/sparql-gremlin/pom.xml +++ b/sparql-gremlin/pom.xml @@ -2,55 +2,46 @@ http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 tinkerpop org.apache.tinkerpop 3.3.2-SNAPSHOT -4.0.0 - sparql-gremlin - -WDAQUA-ITN -http://wdaqua.eu - - - -Harsh Thakkar -hars...@gmail.com -University of Bonn -http://harshthakkar.in - - -Dharmen Punjani -dharmen.punj...@gmail.com -National and Kapodistrian University of Athens - http://wdaqua.eu/students/dharmen-punjani - - +Apache TinkerPop :: SPARQL Gremlin -UTF-8 -3.0.0 src/test/resources/sparql/queries - -org.apache.jena -apache-jena-libs -pom -${jena.version} - + +org.apache.jena +apache-jena-libs +pom +3.0.0 + + + +org.apache.httpcomponents +httpclient + + + + +org.apache.httpcomponents +httpclient + org.apache.tinkerpop gremlin-core -3.3.2-SNAPSHOT +${project.version} org.apache.tinkerpop tinkergraph-gremlin -3.3.2-SNAPSHOT +${project.version} org.apache.tinkerpop @@ -59,5 +50,4 @@ test - \ No newline at end of file
[33/50] tinkerpop git commit: TINKERPOP-1878 Major refactoring of sparql-gremlin classes
TINKERPOP-1878 Major refactoring of sparql-gremlin classes Renamed the "compiler" to a "transpiler" since it's really transforming sparql to a language of a similar level of abstraction in Gremlin. Lots of code reformatting and dead code pruning. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/456cdcfc Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/456cdcfc Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/456cdcfc Branch: refs/heads/TINKERPOP-1878 Commit: 456cdcfce5a0397d08a079c7e145e7233e7f6de9 Parents: fe14b3c Author: Stephen Mallette Authored: Fri Jan 26 13:36:29 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- .../gremlin/sparql/SparqlToGremlinCompiler.java | 381 --- .../sparql/SparqlToGremlinTranspiler.java | 286 ++ .../gremlin/sparql/TraversalBuilder.java| 7 +- .../gremlin/sparql/WhereTraversalBuilder.java | 66 ++-- .../traversal/strategy/SparqlStrategy.java | 5 +- 5 files changed, 322 insertions(+), 423 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/456cdcfc/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinCompiler.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinCompiler.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinCompiler.java deleted file mode 100644 index adb36e9..000 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinCompiler.java +++ /dev/null @@ -1,381 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ - -package org.apache.tinkerpop.gremlin.sparql; - -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -import org.apache.jena.graph.Triple; -import org.apache.jena.query.Query; -import org.apache.jena.query.QueryFactory; -import org.apache.jena.query.SortCondition; -import org.apache.jena.query.Syntax; -import org.apache.jena.sparql.algebra.Algebra; -import org.apache.jena.sparql.algebra.Op; -import org.apache.jena.sparql.algebra.OpVisitorBase; -import org.apache.jena.sparql.algebra.OpWalker; -import org.apache.jena.sparql.algebra.op.OpBGP; -import org.apache.jena.sparql.algebra.op.OpFilter; -import org.apache.jena.sparql.algebra.op.OpLeftJoin; -import org.apache.jena.sparql.algebra.op.OpUnion; -import org.apache.jena.sparql.core.Var; -import org.apache.jena.sparql.core.VarExprList; -import org.apache.jena.sparql.expr.Expr; -import org.apache.jena.sparql.expr.ExprAggregator; -import org.apache.tinkerpop.gremlin.process.traversal.Order; -import org.apache.tinkerpop.gremlin.process.traversal.Scope; -import org.apache.tinkerpop.gremlin.process.traversal.Traversal; -import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal; -import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource; -import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__; -import org.apache.tinkerpop.gremlin.structure.Graph; -import org.apache.tinkerpop.gremlin.structure.Vertex; - -// TODO: implement OpVisitor, don't extend OpVisitorBase -public class SparqlToGremlinCompiler extends OpVisitorBase { - - private GraphTraversal traversal; - - List traversalList = new ArrayList(); - - String groupVariable = ""; - int sortingDirection = 0; - long offsetLimit = 0; - String sortingVariable = ""; - - GraphTraversalSource temp; - Graph graph; - - private SparqlToGremlinCompiler(final GraphTraversal traversal) { - this.traversal = traversal; - } - - private SparqlToGremlinCompiler(final GraphTraversalSource g) { - this(g.V()); - temp = g; - - } - - private SparqlToGremlinCompiler(final Graph g) { - this.traversal = (GraphTraversal) g.traversal(); -
[31/50] tinkerpop git commit: TINKERPOP-1878 Added basic docs for sparql-gremlin
TINKERPOP-1878 Added basic docs for sparql-gremlin Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/8e6acf43 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/8e6acf43 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/8e6acf43 Branch: refs/heads/TINKERPOP-1878 Commit: 8e6acf43d0bb5cf6ae7e546d40cb5422f8d17f53 Parents: a816dec Author: Stephen Mallette Authored: Mon Jan 29 11:04:23 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- docs/preprocessor/install-plugins.sh| 2 +- docs/src/reference/index.asciidoc | 2 + docs/src/reference/transpilers.asciidoc | 66 3 files changed, 69 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/8e6acf43/docs/preprocessor/install-plugins.sh -- diff --git a/docs/preprocessor/install-plugins.sh b/docs/preprocessor/install-plugins.sh index 0a7ca31..f1da197 100755 --- a/docs/preprocessor/install-plugins.sh +++ b/docs/preprocessor/install-plugins.sh @@ -25,7 +25,7 @@ TMP_DIR=$3 INSTALL_TEMPLATE="docs/preprocessor/install-plugins.groovy" INSTALL_FILE="${TMP_DIR}/install-plugins.groovy" -plugins=("hadoop-gremlin" "spark-gremlin" "neo4j-gremlin") +plugins=("hadoop-gremlin" "spark-gremlin" "neo4j-gremlin", "sparql-gremlin") # plugins=() pluginsCount=${#plugins[@]} http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/8e6acf43/docs/src/reference/index.asciidoc -- diff --git a/docs/src/reference/index.asciidoc b/docs/src/reference/index.asciidoc index 6e81bf2..02a218b 100644 --- a/docs/src/reference/index.asciidoc +++ b/docs/src/reference/index.asciidoc @@ -45,6 +45,8 @@ include::implementations-hadoop-end.asciidoc[] include::gremlin-variants.asciidoc[] +include::transpilers.asciidoc[] + include::conclusion.asciidoc[] include::acknowledgements.asciidoc[] http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/8e6acf43/docs/src/reference/transpilers.asciidoc -- diff --git a/docs/src/reference/transpilers.asciidoc b/docs/src/reference/transpilers.asciidoc new file mode 100644 index 000..305e3dc --- /dev/null +++ b/docs/src/reference/transpilers.asciidoc @@ -0,0 +1,66 @@ + +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +[[transpilers]] += Gremlin Transpilers + +There are many languages built to query data. SQL is typically used to query relational data. There is SPARQL for RDF +data. Cypher is used to do pattern matching in graph data. The list could go on. Transpilers convert languages like +these to Gremlin so that it becomes possible to use them in any context that Gremlin is used. In other words, a +Gremlin Transpiler enables a particular query language to work on any TinkerPop-enabled graph system. + +== SPARQL-Gremlin + +[source,xml] + + + org.apache.tinkerpop + sparql-gremlin + x.y.z + + + +The SPARQL-Gremlin transpiler converts link:https://en.wikipedia.org/wiki/SPARQL[SPARQL] queries into Gremlin so that +they can be executed across any TinkerPop-enabled graph system. To use this transpiler in the Gremlin Console, first +install and activate the "tinkerpop.sparql" plugin: + +[source,text] + +gremlin> :install org.apache.tinkerpop sparql-gremlin x.y.z +==>Loaded: [org.apache.tinkerpop, sparql-gremlin, x.y.z] +gremlin> :plugin use tinkerpop.sparql +==>tinkerpop.sparql activated + + +Installing this plugin will download appropriate dependencies and import certain classes to the console so that they +may be used as follows: + +[gremlin-groovy,modern] + +graph = TinkerFactory.createModern() +g = graph.traversal(SparqlTraversalSource) <1> +g.sparql("""SELECT ?name ?age +WHERE { ?person v:name ?name . ?person v:age ?age } +ORDER BY ASC(?age)""")
[12/50] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/7d21ee0b Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/7d21ee0b Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/7d21ee0b Branch: refs/heads/TINKERPOP-1878 Commit: 7d21ee0b8b8770a68e6b5fb6f84500170f6c2a82 Parents: 4dccf14 ddc6694 Author: Stephen Mallette Authored: Mon Jul 30 11:41:28 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 11:41:28 2018 -0400 -- .../org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --
[28/50] tinkerpop git commit: TINKERPOP-1878 Test cleanup for sparql-gremlin
TINKERPOP-1878 Test cleanup for sparql-gremlin Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/3bf92605 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/3bf92605 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/3bf92605 Branch: refs/heads/TINKERPOP-1878 Commit: 3bf92605e42c62c566e8062ebc0207a6d9adafe5 Parents: 492db18 Author: Stephen Mallette Authored: Fri Jan 26 08:10:48 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:32:58 2018 -0400 -- sparql-gremlin/pom.xml | 10 +- .../dsl/sparql/SparqlTraversalSource.java | 4 +- .../gremlin/sparql/ResourceHelper.java | 37 --- .../sparql/SparqlToGremlinCompilerTest.java | 223 --- .../dsl/sparql/SparqlTraversalSourceTest.java | 31 ++- 5 files changed, 36 insertions(+), 269 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/3bf92605/sparql-gremlin/pom.xml -- diff --git a/sparql-gremlin/pom.xml b/sparql-gremlin/pom.xml index 0e3da38..731b822 100644 --- a/sparql-gremlin/pom.xml +++ b/sparql-gremlin/pom.xml @@ -41,9 +41,13 @@ test -org.apache.tinkerpop -gremlin-test -${project.version} +junit +junit +test + + +org.hamcrest +hamcrest-all test http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/3bf92605/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java index deb16f7..cc54aa5 100644 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java @@ -28,6 +28,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal; import org.apache.tinkerpop.gremlin.process.traversal.step.map.ConstantStep; import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.InjectStep; +import org.apache.tinkerpop.gremlin.sparql.process.traversal.strategy.SparqlStrategy; import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.Transaction; import org.apache.tinkerpop.gremlin.structure.util.StringFactory; @@ -46,6 +47,7 @@ public class SparqlTraversalSource implements TraversalSource { public SparqlTraversalSource(final Graph graph, final TraversalStrategies traversalStrategies) { this.graph = graph; this.strategies = traversalStrategies; +this.strategies.addStrategies(SparqlStrategy.instance()); } public SparqlTraversalSource(final Graph graph) { @@ -122,7 +124,7 @@ public class SparqlTraversalSource implements TraversalSource { /** * The start step for a SPARQL based traversal that accepts a string representation of the query to execute. */ -public SparqlTraversal sparql(final String query) { +public SparqlTraversal sparql(final String query) { final SparqlTraversalSource clone = this.clone(); // this is a bit of a hack to get remote traversals to work cleanly. on the remote side, we'd expect a http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/3bf92605/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/ResourceHelper.java -- diff --git a/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/ResourceHelper.java b/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/ResourceHelper.java deleted file mode 100644 index 4f58e61..000 --- a/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/ResourceHelper.java +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the
[25/50] tinkerpop git commit: TINKERPOP-1878 Dropped the console app
TINKERPOP-1878 Dropped the console app This allows TinkerGraph to remain a test dependency only and the console app was more of a test rig than something that a user of this library would use, so it seemed like something that could be dropped. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/ad8bba60 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/ad8bba60 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/ad8bba60 Branch: refs/heads/TINKERPOP-1878 Commit: ad8bba6097782f35d9ad7e626ed23e364532a7b1 Parents: c940117 Author: Stephen Mallette Authored: Thu Jan 25 14:49:18 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:32:58 2018 -0400 -- sparql-gremlin/pom.xml | 1 + .../gremlin/sparql/ConsoleCompiler.java | 165 --- 2 files changed, 1 insertion(+), 165 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ad8bba60/sparql-gremlin/pom.xml -- diff --git a/sparql-gremlin/pom.xml b/sparql-gremlin/pom.xml index d38a85d..0dceb85 100644 --- a/sparql-gremlin/pom.xml +++ b/sparql-gremlin/pom.xml @@ -42,6 +42,7 @@ org.apache.tinkerpop tinkergraph-gremlin ${project.version} +test org.apache.tinkerpop http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ad8bba60/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/ConsoleCompiler.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/ConsoleCompiler.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/ConsoleCompiler.java deleted file mode 100644 index 0d50475..000 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/ConsoleCompiler.java +++ /dev/null @@ -1,165 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ - -package org.apache.tinkerpop.gremlin.sparql; - -import java.io.BufferedReader; -import java.io.FileInputStream; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.io.StringReader; -import java.util.Map; -import java.util.stream.Collectors; - -import org.apache.commons.cli.CommandLine; -import org.apache.commons.cli.CommandLineParser; -import org.apache.commons.cli.DefaultParser; -import org.apache.commons.cli.Options; -import org.apache.commons.cli.ParseException; -import org.apache.tinkerpop.gremlin.jsr223.JavaTranslator; -import org.apache.tinkerpop.gremlin.process.traversal.Bytecode; -import org.apache.tinkerpop.gremlin.process.traversal.Traversal; -import org.apache.tinkerpop.gremlin.structure.Graph; -import org.apache.tinkerpop.gremlin.structure.Vertex; -import org.apache.tinkerpop.gremlin.structure.io.IoCore; -import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerFactory; -import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph; - -import org.apache.tinkerpop.gremlin.sparql.SparqlToGremlinCompiler; - -class ConsoleCompiler { - -public static void main(final String[] args) throws IOException { - //args = "/examples/modern1.sparql"; -final Options options = new Options(); -options.addOption("f", "file", true, "a file that contains a SPARQL query"); -options.addOption("g", "graph", true, "the graph that's used to execute the query [classic|modern|crew|kryo file]"); -// TODO: add an OLAP option (perhaps: "--olap spark"?) - -final CommandLineParser parser = new DefaultParser(); -final CommandLine commandLine; - -try { -commandLine = parser.parse(options, args); -} catch (ParseException e) { -System.out.println(e.getMessage()); -printHelp(1); -return; -} - -final InputStream inputStream = commandLine.hasOption("file") -? new
tinkerpop git commit: TINKERPOP-1878 Minor formatting fix
Repository: tinkerpop Updated Branches: refs/heads/TINKERPOP-1878 10eed31bf -> 1b2f121ae TINKERPOP-1878 Minor formatting fix Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/1b2f121a Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/1b2f121a Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/1b2f121a Branch: refs/heads/TINKERPOP-1878 Commit: 1b2f121aea97b550b7c01f1321fed275c16fafa9 Parents: 10eed31 Author: Stephen Mallette Authored: Wed Aug 1 10:01:51 2018 -0400 Committer: Stephen Mallette Committed: Wed Aug 1 10:01:51 2018 -0400 -- .../apache/tinkerpop/gremlin/sparql/SparqlToGremlinCompiler.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/1b2f121a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinCompiler.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinCompiler.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinCompiler.java index e3b4ecb..4e00045 100644 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinCompiler.java +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinCompiler.java @@ -35,7 +35,7 @@ import org.apache.jena.sparql.algebra.OpVisitorBase; import org.apache.jena.sparql.algebra.OpWalker; import org.apache.jena.sparql.algebra.op.OpBGP; import org.apache.jena.sparql.algebra.op.OpFilter; -import org.apache.jena.sparql.algebra.op.OpLeftJoin; +import org.apache.jena.sparql.algebra.op.OpLeftJoin; import org.apache.jena.sparql.algebra.op.OpUnion; import org.apache.jena.sparql.core.Var; import org.apache.jena.sparql.core.VarExprList;
tinkerpop git commit: TINKERPOP-1878 Minor formatting/edits to compilers doc for sparql-gremlin
Repository: tinkerpop Updated Branches: refs/heads/TINKERPOP-1878 fafda957f -> 10eed31bf TINKERPOP-1878 Minor formatting/edits to compilers doc for sparql-gremlin Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/10eed31b Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/10eed31b Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/10eed31b Branch: refs/heads/TINKERPOP-1878 Commit: 10eed31bfc2204e683d1fb8555dd8cb4b851a1e6 Parents: fafda95 Author: Stephen Mallette Authored: Wed Aug 1 09:59:19 2018 -0400 Committer: Stephen Mallette Committed: Wed Aug 1 09:59:19 2018 -0400 -- docs/src/reference/compilers.asciidoc | 129 +++-- 1 file changed, 48 insertions(+), 81 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/10eed31b/docs/src/reference/compilers.asciidoc -- diff --git a/docs/src/reference/compilers.asciidoc b/docs/src/reference/compilers.asciidoc index 5b6ab2e..2a8d561 100644 --- a/docs/src/reference/compilers.asciidoc +++ b/docs/src/reference/compilers.asciidoc @@ -18,9 +18,9 @@ limitations under the License. = Gremlin Compilers There are many languages built to query data. SQL is typically used to query relational data. There is SPARQL for RDF -data. Cypher is used to do pattern matching in graph data. The list could go on. compilers convert languages like +data. Cypher is used to do pattern matching in graph data. The list could go on. Compilers convert languages like these to Gremlin so that it becomes possible to use them in any context that Gremlin is used. In other words, a -Gremlin Cranspiler enables a particular query language to work on any TinkerPop-enabled graph system. +Gremlin Compiler enables a particular query language to work on any TinkerPop-enabled graph system. [[sparql-gremlin]] == SPARQL-Gremlin @@ -139,83 +139,90 @@ g.sparql("""SELECT * WHERE { ?x ?y ?z . }""") * A SPARQL Union query with un-balanced patterns, i.e. a gremlin union traversal can only be generated if the input SPARQL query has the same number of patterns on both the side of the union operator. For instance, the following -SPARQL query cannot be mapped using Gremlinator, since a union is executed between different number of graph patterns -(two patterns `union` 1 pattern). +SPARQL query cannot be mapped, since a union is executed between different number of graph patterns (two patterns +`union` 1 pattern). [source,groovy] -g.sparql("""SELECT * WHERE { - {?person e:created ?software . - ?person v:name "josh" .} - UNION - {?software v:lang "java" .} }""") +g.sparql("""SELECT * +WHERE { +{?person e:created ?software . +?person v:name "josh" .} +UNION +{?software v:lang "java" .} }""") -* A non-Group key variable cannot be projected in a SPARQL query. This is a SPARQL langauge limitation rather than that of Gremlin/TinkerPop. Apache Jena throws the exception " Non-group key variable in SELECT" if this occurs. -For instance, in a SPARQL query with GROUP-BY clause, only the variable on which the grouping is declared, can be projected. The following query is valid: +* A non-Group key variable cannot be projected in a SPARQL query. This is a SPARQL language limitation rather than +that of Gremlin/TinkerPop. Apache Jena throws the exception "Non-group key variable in SELECT" if this occurs. +For instance, in a SPARQL query with GROUP-BY clause, only the variable on which the grouping is declared, can be +projected. The following query is valid: [source,groovy] -g.sparql("""SELECT ?age WHERE { -?person v:label "person" . -?person v:age ?age . -?person v:name ?name .} GROUP BY (?age)""") +g.sparql("""SELECT ?age +WHERE { +?person v:label "person" . +?person v:age ?age . +?person v:name ?name .} GROUP BY (?age)""") Whereas, the following SPARQL query will be invalid: [source,groovy] -g.sparql("""SELECT ?person WHERE { - ?person v:label "person" . - ?person v:age ?age . - ?person v:name ?name .} GROUP BY (?age)""") +g.sparql("""SELECT ?person +WHERE { + ?person v:label "person" . + ?person v:age ?age . + ?person v:name ?name .} GROUP BY (?age)""") - -* In a SPARQL query with an ORDER-BY clause, the ordering occurs wrt to the first projected variable in the query. You may choose any number of variable to be projected, however, the first variable in the selection will be the ordering decider. -For instance, in the query: - +* In a SPARQL query with an ORDER-BY clause,
[50/50] tinkerpop git commit: TINKERPOP-1878 Renamed "transpiler" to "compiler"
TINKERPOP-1878 Renamed "transpiler" to "compiler" Since we're converting to Gremlin bytecode it's really compiling and not transpiling (as in converting to other source code) Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/55c991b2 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/55c991b2 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/55c991b2 Branch: refs/heads/TINKERPOP-1878 Commit: 55c991b2d87d5a1676cec1cae190eefbd17ea9a3 Parents: c5b3c4c Author: Stephen Mallette Authored: Wed Jun 20 07:25:17 2018 -0400 Committer: Stephen Mallette Committed: Wed Aug 1 09:37:35 2018 -0400 -- docs/site/home/index.html | 1 + docs/src/reference/compilers.asciidoc | 450 +++ docs/src/reference/index.asciidoc | 2 +- docs/src/reference/transpilers.asciidoc | 449 -- .../gremlin/sparql/SparqlToGremlinCompiler.java | 266 +++ .../sparql/SparqlToGremlinTranspiler.java | 266 --- .../traversal/strategy/SparqlStrategy.java | 4 +- 7 files changed, 720 insertions(+), 718 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/55c991b2/docs/site/home/index.html -- diff --git a/docs/site/home/index.html b/docs/site/home/index.html index 4ff692c..b17fedc 100644 --- a/docs/site/home/index.html +++ b/docs/site/home/index.html @@ -258,6 +258,7 @@ limitations under the License. https://github.com/pm-dev/kotlin-gremlin-ogm;>kotlin-gremlin-ogm (kotlin/dsl) - An Object Graph Mapping Library for Kotlin and Gremlin. http://ogre.clojurewerkz.org/;>ogre (clojure/variant) - A Clojure language wrapper for TinkerPop3. http://bayofmany.github.io/;>Peapod (java/dsl) - An object-graph-wrapper. +http://tinkerpop.apache.org/docs/current/reference/#sparql-gremlin;>sparql-gremlin (sparql/distinct) - A SPARQL to Gremlin traversal compiler. https://github.com/LITMUS-Benchmark-Suite/sparql-to-gremlin;>sparql-gremlin (sparql/distinct) - A SPARQL to Gremlin traversal compiler. https://github.com/Microsoft/spring-data-gremlin;>spring-data-gremlin (java/dsl) - Spring Data support for TinkerPop-enabled graph systems. https://github.com/twilmes/sql-gremlin;>sql-gremlin (sql/distinct) - An SQL to Gremlin traversal compiler. http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/55c991b2/docs/src/reference/compilers.asciidoc -- diff --git a/docs/src/reference/compilers.asciidoc b/docs/src/reference/compilers.asciidoc new file mode 100644 index 000..e015cc8 --- /dev/null +++ b/docs/src/reference/compilers.asciidoc @@ -0,0 +1,450 @@ + +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +[[compilers]] += Gremlin Compilers + +There are many languages built to query data. SQL is typically used to query relational data. There is SPARQL for RDF +data. Cypher is used to do pattern matching in graph data. The list could go on. compilers convert languages like +these to Gremlin so that it becomes possible to use them in any context that Gremlin is used. In other words, a +Gremlin Cranspiler enables a particular query language to work on any TinkerPop-enabled graph system. + +[[sparql-gremlin]] +== SPARQL-Gremlin + +image::gremlintron.png[width=225] + +The SPARQL-Gremlin compiler, transforms link:https://en.wikipedia.org/wiki/SPARQL[SPARQL] queries into Gremlin +traversals. It uses the https://jena.apache.org/index.html[Apache Jena] SPARQL processor +link:https://jena.apache.org/documentation/query/index.html[ARQ], which provides access to a syntax tree of a +SPARQL query. + +The goal of this work is to bridge the query interoperability gap between the two famous, yet fairly disconnected, +graph communities: Semantic Web (which relies on the RDF data model) and Graph database (which relies on property graph +data model). + +NOTE: The foundational
[43/50] tinkerpop git commit: TINKERPOP-1878 Clean up docs a bit more.
TINKERPOP-1878 Clean up docs a bit more. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/5c82aa1d Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/5c82aa1d Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/5c82aa1d Branch: refs/heads/TINKERPOP-1878 Commit: 5c82aa1d7629bfd3d64fd76257fe19c848c67ebb Parents: 9d153af Author: Stephen Mallette Authored: Sun May 13 10:53:56 2018 -0400 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- docs/src/reference/transpilers.asciidoc | 36 +--- 1 file changed, 16 insertions(+), 20 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/5c82aa1d/docs/src/reference/transpilers.asciidoc -- diff --git a/docs/src/reference/transpilers.asciidoc b/docs/src/reference/transpilers.asciidoc index b05be68..390d4f4 100644 --- a/docs/src/reference/transpilers.asciidoc +++ b/docs/src/reference/transpilers.asciidoc @@ -24,23 +24,20 @@ Gremlin Transpiler enables a particular query language to work on any TinkerPop- == SPARQL-Gremlin -image:https://raw.githubusercontent.com/LITMUS-Benchmark-Suite/sparql-to-gremlin/master/docs/images/sparql-gremlin-logo.png[gremlinator] +image::gremlin-standing.png[] -The SPARQL-Gremlin transpiler, transforms link:https://en.wikipedia.org/wiki/SPARQL[SPARQL] queries into Gremlin traversals. It is based on the https://jena.apache.org/index.html[Apache Jena] SPARQL processor https://jena.apache.org/documentation/query/index.html[ARQ], which provides access to a syntax tree of a SPARQL query. +The SPARQL-Gremlin transpiler, transforms link:https://en.wikipedia.org/wiki/SPARQL[SPARQL] queries into Gremlin +traversals. It uses the https://jena.apache.org/index.html[Apache Jena] SPARQL processor +link:https://jena.apache.org/documentation/query/index.html[ARQ], which provides access to a syntax tree of a +SPARQL query. +The goal of this work is to bridge the query interoperability gap between the two famous, yet fairly disconnected, +graph communities: Semantic Web (which relies on the RDF data model) and Graph database (which relies on property graph +data model). -The goal of this work is to bridge the query interoperability gap between the two famous, yet fairly disconnected, graph communities: Semantic Web (which relies on the RDF data model) and Graph database -(which relies on Property graph data model). - -NOTE: The foundational research work on SPARQL-Gremlin transpiler (aka Gremlinator) can be found from - -https://arxiv.org/pdf/1801.02911.pdf[Gremlinator full paper]. In this -paper, we present and discuss the notion of graph query language -semantics of SPARQL and Gremlin, and a formal mapping between SPARQL -pattern matching graph patterns and Gremlin traversals. Furthermore, we -point the interested reader to the following resourcesfor a better -understanding: (1) Gremlinator demonstration - -(http://gremlinator.iai.uni-bonn.de:8080/Demo/[Public Demo Mirror 1]) -and (http://195.201.31.31:8080/Demo/[Public Demo Mirror 2]); (2) A short video tutorial on how to use the demonstration - https://www.youtube.com/watch?v=Z0ETx2IBamw[here] +NOTE: The foundational research work on SPARQL-Gremlin transpiler (aka Gremlinator) can be found in the +link:https://arxiv.org/pdf/1801.02911.pdf[Gremlinator paper]. This paper presents the graph query language semantics of +SPARQL and Gremlin, and a formal mapping between SPARQL pattern matching graph patterns and Gremlin traversals. [source,xml] @@ -111,9 +108,9 @@ g.sparql("""SELECT ?name ?id ?label [[supported-queries]] === Supported Queries -The SPARQL-Gremlin transpiler is currently an on-going effort with an aim to cover the entire SPARQL 1.1 query feature -spectrum, however we currently only support translation of the SPARQL 1.0 specification, especially _SELECT_ queries. -The supported SPARQL query types are: +The SPARQL-Gremlin transpiler currently supports translation of the SPARQL 1.0 specification, especially `SELECT` +queries, though there is an on-going effort to cover the entire SPARQL 1.1 query feature spectrum. The supported +SPARQL query types are: * Union * Optional @@ -129,8 +126,7 @@ The supported SPARQL query types are: [[limitations]] === Limitations -The current implementation of SPARQL-Gremlin transpiler (i.e. SPARQL-Gremlin) does -not support the following cases: +The current implementation of SPARQL-Gremlin transpiler (i.e. SPARQL-Gremlin) does not support the following cases: * SPARQL queries with variables in the predicate position are not currently covered, with an exception of the following case: @@ -143,7 +139,7 @@ g.sparql("""SELECT * WHERE { ?x ?y ?z . }""")
[22/50] tinkerpop git commit: Initial commit for sparql-gremlin changes
Initial commit for sparql-gremlin changes Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c4244ce4 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c4244ce4 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c4244ce4 Branch: refs/heads/TINKERPOP-1878 Commit: c4244ce47387fbe45e7ce26915a5ba418fc30725 Parents: b183edd Author: harsh9t Authored: Thu Jan 18 01:33:29 2018 +0100 Committer: Stephen Mallette Committed: Wed Aug 1 09:32:58 2018 -0400 -- sparql-gremlin/pom.xml | 63 +++ .../gremlin/sparql/ConsoleCompiler.java | 165 .../tinkerpop/gremlin/sparql/Prefixes.java | 64 .../gremlin/sparql/SparqlToGremlinCompiler.java | 381 +++ .../gremlin/sparql/TraversalBuilder.java| 82 .../gremlin/sparql/WhereTraversalBuilder.java | 134 +++ .../tinkerpop/gremlin/sparql/PrefixesTest.java | 67 .../gremlin/sparql/ResourceHelper.java | 37 ++ .../sparql/SparqlToGremlinCompilerTest.java | 223 +++ 9 files changed, 1216 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c4244ce4/sparql-gremlin/pom.xml -- diff --git a/sparql-gremlin/pom.xml b/sparql-gremlin/pom.xml new file mode 100644 index 000..e134d31 --- /dev/null +++ b/sparql-gremlin/pom.xml @@ -0,0 +1,63 @@ + +http://maven.apache.org/POM/4.0.0; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> + +tinkerpop +org.apache.tinkerpop +3.3.2-SNAPSHOT + +4.0.0 + +sparql-gremlin + +WDAQUA-ITN +http://wdaqua.eu + + + +Harsh Thakkar +hars...@gmail.com +University of Bonn +http://harshthakkar.in + + +Dharmen Punjani +dharmen.punj...@gmail.com +National and Kapodistrian University of Athens + http://wdaqua.eu/students/dharmen-punjani + + + + +UTF-8 +3.0.0 +src/test/resources/sparql/queries + + + + +org.apache.jena +apache-jena-libs +pom +${jena.version} + + +org.apache.tinkerpop +gremlin-core +3.3.2-SNAPSHOT + + +org.apache.tinkerpop +tinkergraph-gremlin +3.3.2-SNAPSHOT + + +org.apache.tinkerpop +gremlin-test +${project.version} +test + + + + \ No newline at end of file http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c4244ce4/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/ConsoleCompiler.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/ConsoleCompiler.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/ConsoleCompiler.java new file mode 100644 index 000..0d50475 --- /dev/null +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/ConsoleCompiler.java @@ -0,0 +1,165 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.tinkerpop.gremlin.sparql; + +import java.io.BufferedReader; +import java.io.FileInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.io.StringReader; +import java.util.Map; +import java.util.stream.Collectors; + +import org.apache.commons.cli.CommandLine; +import org.apache.commons.cli.CommandLineParser; +import org.apache.commons.cli.DefaultParser; +import org.apache.commons.cli.Options; +import org.apache.commons.cli.ParseException; +import org.apache.tinkerpop.gremlin.jsr223.JavaTranslator; +import
[27/50] tinkerpop git commit: TINKERPOP-1878 sparql() step works over remoting
TINKERPOP-1878 sparql() step works over remoting At this point it is only tested to work with Gryo 3.0. Fixed a bug with how dynamic gryo registrations work - they failed if you didn't have a specific serializer specified. Introduced a bit of a hack to make SparqlTraversalSource behave like a GraphTraversalSource and thus more cleanly follow the DSL pattern. Not sure if that's a great approach yet or not, but it seems harmless at this point. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/7171f37d Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/7171f37d Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/7171f37d Branch: refs/heads/TINKERPOP-1878 Commit: 7171f37de7f2ffe501e907b7fc922aaf31263a51 Parents: 48ae8aa Author: Stephen Mallette Authored: Thu Jan 25 17:00:54 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:32:58 2018 -0400 -- .../apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java | 5 - sparql-gremlin/pom.xml | 4 .../process/traversal/dsl/sparql/SparqlTraversalSource.java | 3 +++ .../sparql/process/traversal/strategy/SparqlStrategy.java | 5 +++-- 4 files changed, 10 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7171f37d/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java index e7dfd93..45d2970 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java @@ -403,6 +403,9 @@ public enum GryoVersion { "org.apache.tinkerpop.gremlin.driver.ser.RequestMessageGryoSerializer", 167); tryAddDynamicType(this, "org.apache.tinkerpop.gremlin.driver.message.ResponseMessage", "org.apache.tinkerpop.gremlin.driver.ser.ResponseMessageGryoSerializer", 169); + +tryAddDynamicType(this, "org.apache.tinkerpop.gremlin.sparql.process.traversal.strategy.SparqlStrategy", +null, 1690); }}; } @@ -591,7 +594,7 @@ public enum GryoVersion { final String serializer, final int registrationId) { try { final Class typeClass = Class.forName(type); -final Optional> serializerInstance = Optional.of(serializer) +final Optional> serializerInstance = Optional.ofNullable(serializer) .map(FunctionUtils.wrapFunction(Class::forName)) .map(FunctionUtils.wrapFunction(c -> (SerializerShim) c.getConstructor().newInstance())); if (serializerInstance.isPresent()) { http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7171f37d/sparql-gremlin/pom.xml -- diff --git a/sparql-gremlin/pom.xml b/sparql-gremlin/pom.xml index 0dceb85..0e3da38 100644 --- a/sparql-gremlin/pom.xml +++ b/sparql-gremlin/pom.xml @@ -11,10 +11,6 @@ sparql-gremlin Apache TinkerPop :: SPARQL Gremlin - -src/test/resources/sparql/queries - - org.apache.jena http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/7171f37d/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java index b156ed1..3889d95 100644 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java @@ -27,6 +27,7 @@ import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal; import org.apache.tinkerpop.gremlin.process.traversal.step.map.ConstantStep; +import org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.InjectStep; import
[47/50] tinkerpop git commit: TINKERPOP-1878 Fixed up formatting of sparql-gremlin image
TINKERPOP-1878 Fixed up formatting of sparql-gremlin image Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/8732c6b5 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/8732c6b5 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/8732c6b5 Branch: refs/heads/TINKERPOP-1878 Commit: 8732c6b5505646ab8d22902176c2afe20f4e7d17 Parents: 5c82aa1 Author: Stephen Mallette Authored: Sun May 13 11:02:26 2018 -0400 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:07 2018 -0400 -- docs/src/reference/transpilers.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/8732c6b5/docs/src/reference/transpilers.asciidoc -- diff --git a/docs/src/reference/transpilers.asciidoc b/docs/src/reference/transpilers.asciidoc index 390d4f4..3211857 100644 --- a/docs/src/reference/transpilers.asciidoc +++ b/docs/src/reference/transpilers.asciidoc @@ -24,7 +24,7 @@ Gremlin Transpiler enables a particular query language to work on any TinkerPop- == SPARQL-Gremlin -image::gremlin-standing.png[] +image::gremlintron.png[width=225] The SPARQL-Gremlin transpiler, transforms link:https://en.wikipedia.org/wiki/SPARQL[SPARQL] queries into Gremlin traversals. It uses the https://jena.apache.org/index.html[Apache Jena] SPARQL processor
[45/50] tinkerpop git commit: Update transpilers.asciidoc
Update transpilers.asciidoc Changes: - fixed non-working queries (queries which returned no results) - updated documentation with more examples in the limitations section - updated execution tags for queries which are for reference only - commented out the pattern filter examples for now To Do: - fix OPTIONAL queries - fix Pattern filter queries - inspect and patch special cases in code Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c322416e Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c322416e Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c322416e Branch: refs/heads/TINKERPOP-1878 Commit: c322416e27712df797c566622f6272fef2d63565 Parents: 4fc2b0d Author: Harsh Thakkar Authored: Fri Jun 15 11:24:57 2018 +0200 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:07 2018 -0400 -- docs/src/reference/transpilers.asciidoc | 94 +--- 1 file changed, 72 insertions(+), 22 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c322416e/docs/src/reference/transpilers.asciidoc -- diff --git a/docs/src/reference/transpilers.asciidoc b/docs/src/reference/transpilers.asciidoc index 3211857..1bb7728 100644 --- a/docs/src/reference/transpilers.asciidoc +++ b/docs/src/reference/transpilers.asciidoc @@ -131,7 +131,7 @@ The current implementation of SPARQL-Gremlin transpiler (i.e. SPARQL-Gremlin) do * SPARQL queries with variables in the predicate position are not currently covered, with an exception of the following case: -[source,groovy] +[source,text] g.sparql("""SELECT * WHERE { ?x ?y ?z . }""") @@ -141,24 +141,72 @@ SPARQL query has the same number of patterns on both the side of the union opera SPARQL query cannot be mapped using Gremlinator, since a union is executed between different number of graph patterns (two patterns `union` 1 pattern). -[source,groovy] +[source,text] -g.sparql("""SELECT * - WHERE { - {?person e:created ?software . - ?person v:name "daniel" .} +g.sparql("""SELECT * WHERE { + {?person e:created ?software . + ?person v:name "josh" .} UNION {?software v:lang "java" .} }""") -* order by - -Adding more here... - +* A non-Group key variable cannot be projected in a SPARQL query. This is a SPARQL langauge limitation rather than that of Gremlin/TinkerPop. Apache Jena throws the exception " Non-group key variable in SELECT" if this occurs. +For instance, in a SPARQL query with GROUP-BY clause, only the variable on which the grouping is declared, can be projected. The following query is valid: + +[source,groovy] + +g.sparql("""SELECT ?age WHERE { +?person v:label "person" . +?person v:age ?age . +?person v:name ?name .} GROUP BY (?age)""") + + +Whereas, the following SPARQL query will be invalid: + +[source,text] + +g.sparql("""SELECT ?person WHERE { + ?person v:label "person" . + ?person v:age ?age . + ?person v:name ?name .} GROUP BY (?age)""") + + + +* In a SPARQL query with an ORDER-BY clause, the ordering occurs wrt to the first projected variable in the query. You may choose any number of variable to be projected, however, the first variable in the selection will be the ordering decider. +For instance, in the query: + + +[source,groovy] + +g.sparql("""SELECT ?name ?age WHERE { +?person v:label "person" . +?person v:age ?age . +?person v:name ?name . } ORDER BY (?age)""") + -* group by +the result set will be ordered according to the `?name` variable (in ascending order by default) despite having passed `?age` in the order by. Whereas, for the following query: + +[source,groovy] + +g.sparql("""SELECT ?age ?name WHERE { +?person v:label "person" . +?person v:age ?age . +?person v:name ?name . } ORDER BY (?age)""") + + +the result set will be ordered according to the `?age` (as it is the first projected variable). Finally, for the select all case (`SELECT *`): + +[source,groovy] + +g.sparql("""SELECT * +WHERE { ?person v:label "person" . ?person v:age ?age . ?person v:name ?name . } ORDER BY (?age)""") + + +The the variable encountered first will be the ordering decider, i.e. since we have `?person` being encountered first, the result set will be ordered according to the `?person` variable (which are vertex id). + +* OPTIONAL -Adding more here... +currently fixing this... [[examples]] @@ -264,6 +312,7 @@ WHERE { FILTER (?age > 30 && ?lang = "java") }""") + [[pattern-filters]] Pattern Filter(s) @@ -288,6 +337,7 @@ WHERE { ?person v:name ?name . FILTER NOT EXISTS { ?person e:created ?project } }""")
[44/50] tinkerpop git commit: TINKERPOP-1878 Bump sparql-gremlin to 3.4.0-SNAPSHOT
TINKERPOP-1878 Bump sparql-gremlin to 3.4.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/4fc2b0de Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/4fc2b0de Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/4fc2b0de Branch: refs/heads/TINKERPOP-1878 Commit: 4fc2b0de9bce26cde30efc93e5b26e8fc396789a Parents: 8732c6b Author: Stephen Mallette Authored: Sun May 13 11:07:10 2018 -0400 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:07 2018 -0400 -- sparql-gremlin/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/4fc2b0de/sparql-gremlin/pom.xml -- diff --git a/sparql-gremlin/pom.xml b/sparql-gremlin/pom.xml index 731b822..8608cc6 100644 --- a/sparql-gremlin/pom.xml +++ b/sparql-gremlin/pom.xml @@ -6,7 +6,7 @@ tinkerpop org.apache.tinkerpop -3.3.2-SNAPSHOT +3.4.0-SNAPSHOT sparql-gremlin Apache TinkerPop :: SPARQL Gremlin
[34/50] tinkerpop git commit: TINKERPOP-1878 Fixed up formatting of traspiler section
TINKERPOP-1878 Fixed up formatting of traspiler section There were some examples that didn't return results or generated errors. These will need some review. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/9d153af9 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/9d153af9 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/9d153af9 Branch: refs/heads/TINKERPOP-1878 Commit: 9d153af999298d546bb6feacfc08be1038c4d50e Parents: 781bfdc Author: Stephen Mallette Authored: Tue Apr 3 10:52:17 2018 -0400 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- docs/src/reference/transpilers.asciidoc | 156 --- 1 file changed, 71 insertions(+), 85 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/9d153af9/docs/src/reference/transpilers.asciidoc -- diff --git a/docs/src/reference/transpilers.asciidoc b/docs/src/reference/transpilers.asciidoc index 386b4a5..b05be68 100644 --- a/docs/src/reference/transpilers.asciidoc +++ b/docs/src/reference/transpilers.asciidoc @@ -84,8 +84,7 @@ start step. and then when finally iterated, executes that against the TinkerGraph. [[prefixes]] -Prefixes -~ +=== Prefixes The SPARQL-Gremlin transpiler supports the following prefixes to traverse the graph: @@ -110,10 +109,11 @@ g.sparql("""SELECT ?name ?id ?label [[supported-queries]] -Supported Queries -~~ +=== Supported Queries -The SPARQL-Gremlin transpiler is currently an on-going effort with an aim to cover the entire SPARQL 1.1 query feature spectrum, however we currently only support translation of the SPARQL 1.0 specification, especially _SELECT_ queries. The supported SPARQL query types are: +The SPARQL-Gremlin transpiler is currently an on-going effort with an aim to cover the entire SPARQL 1.1 query feature +spectrum, however we currently only support translation of the SPARQL 1.0 specification, especially _SELECT_ queries. +The supported SPARQL query types are: * Union * Optional @@ -127,29 +127,25 @@ The SPARQL-Gremlin transpiler is currently an on-going effort with an aim to cov ** OFFSET [[limitations]] -Limitations - +=== Limitations The current implementation of SPARQL-Gremlin transpiler (i.e. SPARQL-Gremlin) does not support the following cases: -* SPARQL queries with variables in the -predicate position are not currently covered, with an exception of the -following case: +* SPARQL queries with variables in the predicate position are not currently covered, with an exception of the following +case: -[gremlin-groovy,existing] +[source,groovy] g.sparql("""SELECT * WHERE { ?x ?y ?z . }""") -* A SPARQL Union query with un-balanced patterns, i.e. a gremlin union -traversal can only be generated if the unput SPARQL query has the same -number of patterns on both the side of the union operator. For instance, -the following SPARQL query cannot be mapped using Gremlinator, since a -union is executed between different number of graph patterns (two -patterns _union_ 1 pattern). +* A SPARQL Union query with un-balanced patterns, i.e. a gremlin union traversal can only be generated if the input +SPARQL query has the same number of patterns on both the side of the union operator. For instance, the following +SPARQL query cannot be mapped using Gremlinator, since a union is executed between different number of graph patterns +(two patterns _union_ 1 pattern). -[gremlin-groovy,existing] +[source,groovy] g.sparql("""SELECT * WHERE { @@ -170,16 +166,15 @@ Adding more here... [[examples]] -Examples -~ +=== Examples -The following section presents a comprehensive examples of SPARQL queries that are currently covered by the SPARQL-Gremlin transpiler. +The following section presents a comprehensive examples of SPARQL queries that are currently covered by the +SPARQL-Gremlin transpiler. [[select-all]] -Select All -^^ + Select All -.Select all vertices in the graph. +Select all vertices in the graph. [gremlin-groovy,existing] @@ -187,10 +182,9 @@ g.sparql("""SELECT * WHERE { }""") [[match-constant-values]] -Match Constant Values -^ + Match Constant Values -.Select all vertices with the label `person`. +Select all vertices with the label `person`. [gremlin-groovy,existing] @@ -198,10 +192,9 @@ g.sparql("""SELECT * WHERE { ?person v:label "person" .}""") [[select-specific-elements]] -Select Specific Elements - + Select Specific Elements -.Select the values of the properties `name` and `age` for each `person` vertex. +Select the values of
[14/50] tinkerpop git commit: Added dev notes for python environment CTR
Added dev notes for python environment CTR Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/e3018fb2 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/e3018fb2 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/e3018fb2 Branch: refs/heads/TINKERPOP-1878 Commit: e3018fb2e98df62a21272446f63cea5ee550408e Parents: 00a085a Author: Stephen Mallette Authored: Tue Jul 31 08:35:06 2018 -0400 Committer: Stephen Mallette Committed: Tue Jul 31 08:35:06 2018 -0400 -- docs/src/dev/developer/development-environment.asciidoc | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/e3018fb2/docs/src/dev/developer/development-environment.asciidoc -- diff --git a/docs/src/dev/developer/development-environment.asciidoc b/docs/src/dev/developer/development-environment.asciidoc index cde81c1..779f642 100644 --- a/docs/src/dev/developer/development-environment.asciidoc +++ b/docs/src/dev/developer/development-environment.asciidoc @@ -74,13 +74,14 @@ The site will be generated to the `target/site/home` directory. [[python-environment]] === Python Environment -As of TinkerPop 3.2.2, the build optionally requires link:https://www.python.org/[Python 2.x] to work with the -`gremlin-python` module. If Python is not installed, TinkerPop will still build with Maven, but native Python tests and +As of TinkerPop 3.2.2, the build optionally requires link:https://www.python.org/[Python] to build the `gremlin-python` +module. If Python is not installed, TinkerPop will still build with Maven, but native Python tests and Java tests that require Python code will be skipped. Developers should also install link:https://pypi.python.org/pypi/pip[pip] and link:https://virtualenv.pypa.io/en/stable/[virtualenv] (version 15.0.2 - older versions may cause build failures). -Once the Python environment is established, the full building and testing of `gremlin-python` may commence. It can be -done manually from the command line with: +The build expects two Python executables `python` and `python3` where `python` maps to 2.7.6 and `python3` is 3.4.3 or +higher. Once the Python environment is established, the full building and testing of `gremlin-python` may commence. It +can be done manually from the command line with: [source,text] mvn clean install -Pglv-python
[30/50] tinkerpop git commit: TINKERPOP-1878 Added a test for GROUP
TINKERPOP-1878 Added a test for GROUP Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/37f4df31 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/37f4df31 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/37f4df31 Branch: refs/heads/TINKERPOP-1878 Commit: 37f4df31f7f20af15d80e5efe952e5442d3e2d7d Parents: 960b045 Author: Stephen Mallette Authored: Mon Jan 29 09:09:07 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- .../dsl/sparql/SparqlTraversalSourceTest.java | 16 1 file changed, 16 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/37f4df31/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java -- diff --git a/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java b/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java index 56d62c9..0e56347 100644 --- a/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java +++ b/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java @@ -110,6 +110,7 @@ public class SparqlTraversalSourceTest { assertEquals(x.get("a"), _g.V(1).next()); assertEquals(x.get("b"), _g.V(4).next()); assertEquals(x.get("c"), _g.V(3).next()); +assertEquals(3, x.size()); } @Test @@ -134,4 +135,19 @@ public class SparqlTraversalSourceTest { "ORDER BY ?name").toList(); assertThat(x, contains("josh", "marko", "peter")); } + +@Test +public void shouldGroup() { +final Map x = (Map) g.sparql( +"SELECT ?name (COUNT(?name) AS ?name_count)\n" + +"WHERE {\n" + +"?a e:created ?b .\n" + +"?a v:name ?name .\n" + +"}" + +"GROUP BY ?name").next(); +assertEquals(new Long(2), x.get("josh")); +assertEquals(new Long(1), x.get("peter")); +assertEquals(new Long(1), x.get("marko")); +assertEquals(3, x.size()); +} }
[40/50] tinkerpop git commit: TINKERPOP-1878 General refactoring and javadoc on Prefixes class
TINKERPOP-1878 General refactoring and javadoc on Prefixes class Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/fe14b3cb Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/fe14b3cb Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/fe14b3cb Branch: refs/heads/TINKERPOP-1878 Commit: fe14b3cbe0b6663b34a4379da90507a1acf41ff7 Parents: 3f1b7bf Author: Stephen Mallette Authored: Fri Jan 26 09:39:28 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- .../tinkerpop/gremlin/sparql/Prefixes.java | 23 ++-- 1 file changed, 12 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/fe14b3cb/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/Prefixes.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/Prefixes.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/Prefixes.java index 452bf68..32768a7 100644 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/Prefixes.java +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/Prefixes.java @@ -22,15 +22,16 @@ package org.apache.tinkerpop.gremlin.sparql; import java.util.Arrays; import java.util.List; +/** + * Helper methods for working with prefix lines in SPARQL queries. + */ +class Prefixes { -public class Prefixes { - - // public final static String BASE_URI = "http://northwind.com/model/;; -public final static String BASE_URI = "http://www.tinkerpop.com/traversal/;; +final static String BASE_URI = "http://tinkerpop.apache.org/traversal/;; -final static List PREFIXES = Arrays.asList("edge", "property", "value"); +private final static List PREFIXES = Arrays.asList("edge", "property", "value"); -final static String PREFIX_DEFINITIONS; +private final static String PREFIX_DEFINITIONS; static { final StringBuilder builder = new StringBuilder(); @@ -41,24 +42,24 @@ public class Prefixes { PREFIX_DEFINITIONS = builder.toString(); } -public static String getURI(final String prefix) { +static String getURI(final String prefix) { return BASE_URI + prefix + "#"; } -public static String getURIValue(final String uri) { +static String getURIValue(final String uri) { return uri.substring(uri.indexOf("#") + 1); } -public static String getPrefix(final String uri) { +static String getPrefix(final String uri) { final String tmp = uri.substring(0, uri.indexOf("#")); return tmp.substring(tmp.lastIndexOf("/") + 1); } -public static String prepend(final String script) { +static String prepend(final String script) { return PREFIX_DEFINITIONS + script; } -public static StringBuilder prepend(final StringBuilder scriptBuilder) { +static StringBuilder prepend(final StringBuilder scriptBuilder) { return scriptBuilder.insert(0, PREFIX_DEFINITIONS); } }
[29/50] tinkerpop git commit: TINKERPOP-1878 Testing for DISTINCT
TINKERPOP-1878 Testing for DISTINCT Refactoring around the distinct and ordering - much less code there now. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/960b0455 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/960b0455 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/960b0455 Branch: refs/heads/TINKERPOP-1878 Commit: 960b0455e4d320a600dae9642ca371abacc12ddd Parents: 35f5fc5 Author: Stephen Mallette Authored: Fri Jan 26 16:56:14 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- .../sparql/SparqlToGremlinTranspiler.java | 44 .../dsl/sparql/SparqlTraversalSourceTest.java | 23 ++ 2 files changed, 41 insertions(+), 26 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/960b0455/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java index 1b28a1d..67ac1e1 100644 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java @@ -107,39 +107,28 @@ public class SparqlToGremlinTranspiler { final List vars = query.getResultVars(); if (!query.isQueryResultStar() && !query.hasGroupBy()) { -// the result sizes have special handling to get the right signatures of select() called. perhaps this -// could be refactored to work more nicely -switch (vars.size()) { +final String[] all = new String[vars.size()]; +vars.toArray(all); +if (query.isDistinct()) { +traversal = traversal.dedup(all); +} + +// apply ordering from ORDER BY +orderingIndex.forEach((k, v) -> traversal = traversal.order().by(__.select(k), v)); + +// the result sizes have special handling to get the right signatures of select() called. +switch (all.length) { case 0: throw new IllegalStateException(); case 1: -if (query.isDistinct()) -traversal = traversal.dedup(vars.get(0)); - -orderingIndex.forEach((k, v) -> traversal = traversal.order().by(__.select(k), v)); -traversal = traversal.select(vars.get(0)); - +traversal = traversal.select(all[0]); break; case 2: -if (query.isDistinct()) -traversal = traversal.dedup(vars.get(0), vars.get(1)); - -orderingIndex.forEach((k, v) -> traversal = traversal.order().by(__.select(k), v)); -traversal = traversal.select(vars.get(0), vars.get(1)); - +traversal = traversal.select(all[0], all[1]); break; default: -final String[] all = new String[vars.size()]; -vars.toArray(all); -if (query.isDistinct()) -traversal = traversal.dedup(all); - -orderingIndex.forEach((k, v) -> traversal = traversal.order().by(__.select(k), v)); - -// just some shenanigans to get the right signature of select() called final String[] others = Arrays.copyOfRange(all, 2, vars.size()); -traversal = traversal.select(vars.get(0), vars.get(1), others); - +traversal = traversal.select(all[0], all[1], others); break; } } @@ -207,7 +196,10 @@ public class SparqlToGremlinTranspiler { for (SortCondition sortCondition : sortingConditions) { final Expr expr = sortCondition.getExpression(); -orderingIndex.put(expr.getVarName(), sortCondition.getDirection() == 1 ? Order.incr : Order.decr); + +// by default, the sort will be ascending. getDirection() returns -2 if the DESC/ASC isn't +// supplied - weird +orderingIndex.put(expr.getVarName(), sortCondition.getDirection() == -1 ? Order.decr : Order.incr); } } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/960b0455/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java
[36/50] tinkerpop git commit: testing logo
testing logo Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/8e60e914 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/8e60e914 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/8e60e914 Branch: refs/heads/TINKERPOP-1878 Commit: 8e60e914402a1190726ea281550a46e31f28541a Parents: 8e6acf4 Author: Harsh Thakkar Authored: Wed Mar 28 10:29:43 2018 +0200 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- docs/src/reference/transpilers.asciidoc | 4 1 file changed, 4 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/8e60e914/docs/src/reference/transpilers.asciidoc -- diff --git a/docs/src/reference/transpilers.asciidoc b/docs/src/reference/transpilers.asciidoc index 305e3dc..9100f84 100644 --- a/docs/src/reference/transpilers.asciidoc +++ b/docs/src/reference/transpilers.asciidoc @@ -17,6 +17,10 @@ limitations under the License. [[transpilers]] = Gremlin Transpilers +![gremlinator][SPARQL-Gremlin] + +[SPARQL-Gremlin]: https://raw.githubusercontent.com/LITMUS-Benchmark-Suite/sparql-to-gremlin/master/docs/images/sparql-gremlin-logo.png + There are many languages built to query data. SQL is typically used to query relational data. There is SPARQL for RDF data. Cypher is used to do pattern matching in graph data. The list could go on. Transpilers convert languages like these to Gremlin so that it becomes possible to use them in any context that Gremlin is used. In other words, a
[41/50] tinkerpop git commit: TINKERPOP-1878 Added a test for ordering and corrected some problems in logic
TINKERPOP-1878 Added a test for ordering and corrected some problems in logic Ordering didn't work - at least not completely. The test, as it is written in this commit, that failed to sort properly. Changed the logic for ordering to track all of the keys specified to ORDER with their appropriate ASC/DESC operators and added all of them to the traversal. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/87ccf583 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/87ccf583 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/87ccf583 Branch: refs/heads/TINKERPOP-1878 Commit: 87ccf5833e84cc490d2aa94d3eac472f4a4288ab Parents: 456cdcf Author: Stephen Mallette Authored: Fri Jan 26 15:41:10 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- .../sparql/SparqlToGremlinTranspiler.java | 55 ++-- .../dsl/sparql/SparqlTraversalSourceTest.java | 31 +-- 2 files changed, 44 insertions(+), 42 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/87ccf583/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java index 9db7d82..ea3f828 100644 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java @@ -21,9 +21,10 @@ package org.apache.tinkerpop.gremlin.sparql; import java.util.ArrayList; import java.util.Arrays; +import java.util.HashMap; import java.util.List; +import java.util.Map; -import org.apache.jena.graph.Triple; import org.apache.jena.query.Query; import org.apache.jena.query.QueryFactory; import org.apache.jena.query.SortCondition; @@ -58,8 +59,6 @@ public class SparqlToGremlinTranspiler { private List traversalList = new ArrayList<>(); -private String sortingVariable = ""; - private SparqlToGremlinTranspiler(final GraphTraversal traversal) { this.traversal = traversal; } @@ -96,31 +95,18 @@ public class SparqlToGremlinTranspiler { final int numberOfTraversal = traversalList.size(); final Traversal arrayOfAllTraversals[] = new Traversal[numberOfTraversal]; - if (query.hasOrderBy() && !query.hasGroupBy()) { -final List sortingConditions = query.getOrderBy(); - - for (SortCondition sortCondition : sortingConditions) { -final Expr expr = sortCondition.getExpression(); - sortingVariable = expr.getVarName(); - } - } - for (Traversal tempTrav : traversalList) { arrayOfAllTraversals[traversalIndex++] = tempTrav; } - Order orderDirection = Order.incr; + final Map orderingIndex = new HashMap<>(); if (query.hasOrderBy()) { -int directionOfSort = 0; final List sortingConditions = query.getOrderBy(); for (SortCondition sortCondition : sortingConditions) { final Expr expr = sortCondition.getExpression(); - directionOfSort = sortCondition.getDirection(); - sortingVariable = expr.getVarName(); +orderingIndex.put(expr.getVarName(), sortCondition.getDirection() == 1 ? Order.incr : Order.decr); } - - if (directionOfSort == -1) orderDirection = Order.decr; } if (traversalList.size() > 0) @@ -128,6 +114,8 @@ public class SparqlToGremlinTranspiler { final List vars = query.getResultVars(); if (!query.isQueryResultStar() && !query.hasGroupBy()) { + // the result sizes have special handling to get the right signatures of select() called. perhaps this +// could be refactored to work more nicely switch (vars.size()) { case 0: throw new IllegalStateException(); @@ -135,20 +123,16 @@ public class SparqlToGremlinTranspiler { if (query.isDistinct()) traversal = traversal.dedup(vars.get(0)); -if (query.hasOrderBy()) -traversal = traversal.order().by(sortingVariable, orderDirection); -else -
[24/50] tinkerpop git commit: TINKERPOP-1878 Minor javadoc and code cleanup
TINKERPOP-1878 Minor javadoc and code cleanup Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/48ae8aa2 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/48ae8aa2 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/48ae8aa2 Branch: refs/heads/TINKERPOP-1878 Commit: 48ae8aa2bd5db3f8a060f11bab8bf5d848d12c27 Parents: ad8bba6 Author: Stephen Mallette Authored: Thu Jan 25 15:36:53 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:32:58 2018 -0400 -- .../process/traversal/dsl/sparql/SparqlTraversal.java | 12 +++- .../traversal/dsl/sparql/SparqlTraversalSource.java | 8 +++- .../process/traversal/strategy/SparqlStrategy.java | 11 --- 3 files changed, 18 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/48ae8aa2/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversal.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversal.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversal.java index efa9489..843267d 100644 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversal.java +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversal.java @@ -22,6 +22,9 @@ import org.apache.tinkerpop.gremlin.process.traversal.Step; import org.apache.tinkerpop.gremlin.process.traversal.Traversal; /** + * The {@code SparqlTraversal} has no additional traversal steps. The only step available for "SPARQL" is the + * {@link SparqlTraversalSource#sparql(String)} start step. + * * @author Stephen Mallette (http://stephen.genoprime.com) */ public interface SparqlTraversal extends Traversal { @@ -56,13 +59,4 @@ public interface SparqlTraversal extends Traversal { Traversal.super.iterate(); return this; } - -public static final class Symbols { - -private Symbols() { -// static fields only -} - -public static final String sparql = "sparql"; -} } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/48ae8aa2/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java index 3596716..b156ed1 100644 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSource.java @@ -25,12 +25,15 @@ import org.apache.tinkerpop.gremlin.process.traversal.Bytecode; import org.apache.tinkerpop.gremlin.process.traversal.TraversalSource; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategies; import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; +import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal; import org.apache.tinkerpop.gremlin.process.traversal.step.map.ConstantStep; import org.apache.tinkerpop.gremlin.structure.Graph; import org.apache.tinkerpop.gremlin.structure.Transaction; import org.apache.tinkerpop.gremlin.structure.util.StringFactory; /** + * A {@link TraversalSource} implementation that spawns {@link SparqlTraversal} instances. + * * @author Stephen Mallette (http://stephen.genoprime.com) */ public class SparqlTraversalSource implements TraversalSource { @@ -115,9 +118,12 @@ public class SparqlTraversalSource implements TraversalSource { return (SparqlTraversalSource) clone; } +/** + * The start step for a SPARQL based traversal that accepts a string representation of the query to execute. + */ public SparqlTraversal sparql(final String query) { final SparqlTraversalSource clone = this.clone(); -clone.bytecode.addStep(SparqlTraversal.Symbols.sparql, query); +clone.bytecode.addStep(GraphTraversal.Symbols.constant, query); final SparqlTraversal.Admin traversal = new DefaultSparqlTraversal<>(clone); return traversal.addStep(new ConstantStep(traversal, query)); }
[37/50] tinkerpop git commit: updating the reference documentation
updating the reference documentation added the following: - fix the logo - added prefixes - added limitations - added examples todo: - cover all limitations and special cases with examples Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/781bfdca Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/781bfdca Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/781bfdca Branch: refs/heads/TINKERPOP-1878 Commit: 781bfdca4659510792cd2991b3595cbe99a682c8 Parents: 8e60e91 Author: Harsh Thakkar Authored: Wed Mar 28 17:27:32 2018 +0200 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- docs/src/reference/transpilers.asciidoc | 355 ++- 1 file changed, 351 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/781bfdca/docs/src/reference/transpilers.asciidoc -- diff --git a/docs/src/reference/transpilers.asciidoc b/docs/src/reference/transpilers.asciidoc index 9100f84..386b4a5 100644 --- a/docs/src/reference/transpilers.asciidoc +++ b/docs/src/reference/transpilers.asciidoc @@ -17,10 +17,6 @@ limitations under the License. [[transpilers]] = Gremlin Transpilers -![gremlinator][SPARQL-Gremlin] - -[SPARQL-Gremlin]: https://raw.githubusercontent.com/LITMUS-Benchmark-Suite/sparql-to-gremlin/master/docs/images/sparql-gremlin-logo.png - There are many languages built to query data. SQL is typically used to query relational data. There is SPARQL for RDF data. Cypher is used to do pattern matching in graph data. The list could go on. Transpilers convert languages like these to Gremlin so that it becomes possible to use them in any context that Gremlin is used. In other words, a @@ -28,6 +24,24 @@ Gremlin Transpiler enables a particular query language to work on any TinkerPop- == SPARQL-Gremlin +image:https://raw.githubusercontent.com/LITMUS-Benchmark-Suite/sparql-to-gremlin/master/docs/images/sparql-gremlin-logo.png[gremlinator] + +The SPARQL-Gremlin transpiler, transforms link:https://en.wikipedia.org/wiki/SPARQL[SPARQL] queries into Gremlin traversals. It is based on the https://jena.apache.org/index.html[Apache Jena] SPARQL processor https://jena.apache.org/documentation/query/index.html[ARQ], which provides access to a syntax tree of a SPARQL query. + + +The goal of this work is to bridge the query interoperability gap between the two famous, yet fairly disconnected, graph communities: Semantic Web (which relies on the RDF data model) and Graph database +(which relies on Property graph data model). + +NOTE: The foundational research work on SPARQL-Gremlin transpiler (aka Gremlinator) can be found from - +https://arxiv.org/pdf/1801.02911.pdf[Gremlinator full paper]. In this +paper, we present and discuss the notion of graph query language +semantics of SPARQL and Gremlin, and a formal mapping between SPARQL +pattern matching graph patterns and Gremlin traversals. Furthermore, we +point the interested reader to the following resourcesfor a better +understanding: (1) Gremlinator demonstration - +(http://gremlinator.iai.uni-bonn.de:8080/Demo/[Public Demo Mirror 1]) +and (http://195.201.31.31:8080/Demo/[Public Demo Mirror 2]); (2) A short video tutorial on how to use the demonstration - https://www.youtube.com/watch?v=Z0ETx2IBamw[here] + [source,xml] @@ -68,3 +82,336 @@ start step. <2> Execute a SPARQL query against the TinkerGraph instance. The `SparqlTraversalSource` uses a <> to transparently converts that SPARQL query into a standard Gremlin traversal and then when finally iterated, executes that against the TinkerGraph. + +[[prefixes]] +Prefixes +~ + +The SPARQL-Gremlin transpiler supports the following prefixes to traverse the graph: + +[cols=",",options="header",] +| +|Prefix |Purpose +|`v:` |label-access traversal +|`e:` |out-edge traversal +|`p:` |property traversal +|`v:` |property-value traversal +| + +Note that element IDs and labels are treated like normal properties, hence they can be accessed using the same pattern: + +[gremlin-groovy,existing] + +g.sparql("""SELECT ?name ?id ?label + WHERE { + ?element v:name ?name . + ?element v:id ?id . + ?element v:label ?label .}""") + + +[[supported-queries]] +Supported Queries +~~ + +The SPARQL-Gremlin transpiler is currently an on-going effort with an aim to cover the entire SPARQL 1.1 query feature spectrum, however we currently only support translation of the SPARQL 1.0 specification, especially _SELECT_ queries. The supported SPARQL query types are: + +* Union +* Optional +* Order-By +* Group-By +*
[15/50] tinkerpop git commit: Merge branch 'tp32' into tp33
Merge branch 'tp32' into tp33 Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c49c0ccb Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c49c0ccb Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c49c0ccb Branch: refs/heads/TINKERPOP-1878 Commit: c49c0ccb3bce76caabb4b0a9c7efe9df43801b39 Parents: ddc6694 e3018fb Author: Stephen Mallette Authored: Tue Jul 31 08:35:24 2018 -0400 Committer: Stephen Mallette Committed: Tue Jul 31 08:35:24 2018 -0400 -- docs/src/dev/developer/development-environment.asciidoc | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c49c0ccb/docs/src/dev/developer/development-environment.asciidoc --
[02/50] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/13ff64c8 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/13ff64c8 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/13ff64c8 Branch: refs/heads/TINKERPOP-1878 Commit: 13ff64c8b1230cb886b28d294f31cce064f4b52d Parents: a17bb40 9b693c0 Author: Stephen Mallette Authored: Fri Jul 27 07:18:45 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 27 07:18:45 2018 -0400 -- docs/src/dev/developer/for-committers.asciidoc| 2 +- docs/src/recipes/between-vertices.asciidoc| 6 +++--- docs/src/recipes/collections.asciidoc | 18 +- docs/src/recipes/pagination.asciidoc | 2 +- .../recipes/traversal-component-reuse.asciidoc| 4 ++-- docs/src/recipes/tree.asciidoc| 2 +- docs/src/reference/the-traversal.asciidoc | 2 +- docs/src/tutorials/getting-started/index.asciidoc | 10 +- .../src/tutorials/gremlins-anatomy/index.asciidoc | 16 .../tutorials/the-gremlin-console/index.asciidoc | 2 +- docs/src/upgrade/release-3.3.x.asciidoc | 4 ++-- 11 files changed, 34 insertions(+), 34 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/13ff64c8/docs/src/reference/the-traversal.asciidoc -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/13ff64c8/docs/src/tutorials/getting-started/index.asciidoc -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/13ff64c8/docs/src/tutorials/the-gremlin-console/index.asciidoc --
[06/50] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/6358715e Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/6358715e Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/6358715e Branch: refs/heads/TINKERPOP-1878 Commit: 6358715e9dfa69627a425e4b930636f271141022 Parents: 9ab5efa 7985106 Author: Stephen Mallette Authored: Fri Jul 27 16:29:50 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 27 16:29:50 2018 -0400 -- docs/src/recipes/element-existence.asciidoc | 91 docs/src/recipes/index.asciidoc | 2 + 2 files changed, 93 insertions(+) --
[20/50] tinkerpop git commit: TINKERPOP-1878 Added a basics for sparql execution in a traversal
TINKERPOP-1878 Added a basics for sparql execution in a traversal Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/c9401171 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/c9401171 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/c9401171 Branch: refs/heads/TINKERPOP-1878 Commit: c9401171ccf923661295f4c7feddb7df94925d66 Parents: eadb8b9 Author: Stephen Mallette Authored: Thu Jan 25 14:46:31 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:32:58 2018 -0400 -- .../traversal/step/map/ConstantStep.java| 4 ++ .../traversal/strategy/SparqlStrategy.java | 56 +++- .../dsl/sparql/SparqlTraversalSourceTest.java | 39 ++ .../src/test/resources/log4j-silent.properties | 23 .../src/test/resources/log4j-test.properties| 23 5 files changed, 144 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c9401171/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ConstantStep.java -- diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ConstantStep.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ConstantStep.java index 5d02e28..749de31 100644 --- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ConstantStep.java +++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/ConstantStep.java @@ -36,6 +36,10 @@ public class ConstantStep extends MapStep { this.constant = constant; } +public E getConstant() { +return this.constant; +} + @Override protected E map(final Traverser.Admin traverser) { return this.constant; http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/c9401171/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/strategy/SparqlStrategy.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/strategy/SparqlStrategy.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/strategy/SparqlStrategy.java index 6440127..07ac4cf 100644 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/strategy/SparqlStrategy.java +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/strategy/SparqlStrategy.java @@ -18,8 +18,62 @@ */ package org.apache.tinkerpop.gremlin.sparql.process.traversal.strategy; +import org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration.VertexProgramStrategy; +import org.apache.tinkerpop.gremlin.process.traversal.Traversal; +import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy; +import org.apache.tinkerpop.gremlin.process.traversal.step.map.ConstantStep; +import org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalMapStep; +import org.apache.tinkerpop.gremlin.process.traversal.step.util.EmptyStep; +import org.apache.tinkerpop.gremlin.process.traversal.strategy.AbstractTraversalStrategy; +import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalHelper; +import org.apache.tinkerpop.gremlin.sparql.SparqlToGremlinCompiler; +import org.apache.tinkerpop.gremlin.structure.Vertex; + +import java.util.Collections; +import java.util.Set; + /** * @author Stephen Mallette (http://stephen.genoprime.com) */ -public class SparqlStrategy { +public class SparqlStrategy extends AbstractTraversalStrategy +implements TraversalStrategy.DecorationStrategy { +private static final SparqlStrategy INSTANCE = new SparqlStrategy(); + +private static final Set> POSTS = Collections.singleton(VertexProgramStrategy.class); + +private SparqlStrategy() {} + +public static SparqlStrategy instance() { +return INSTANCE; +} + +@Override +public Set> applyPost() { +return POSTS; +} + + +@Override +public void apply(final Traversal.Admin traversal) { +if (!(traversal.getParent() instanceof EmptyStep)) +return; + +if (traversal.getSteps().size() == 1 && traversal.getEndStep() instanceof ConstantStep) { +final ConstantStep stepWithSparql = (ConstantStep) traversal.getEndStep(); +final Object constant = stepWithSparql.getConstant(); +if (constant instanceof String) { +final String sparql = (String) constant; +final Traversal sparqlTraversal = SparqlToGremlinCompiler.convertToGremlinTraversal( +
[11/50] tinkerpop git commit: Merge branch 'tp32' into tp33
Merge branch 'tp32' into tp33 Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/ddc66941 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/ddc66941 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/ddc66941 Branch: refs/heads/TINKERPOP-1878 Commit: ddc66941ed6340edd9009f0aa277aa8326255c68 Parents: 88b6e14 00a085a Author: Stephen Mallette Authored: Mon Jul 30 11:41:21 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 11:41:21 2018 -0400 -- .../org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/ddc66941/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteGraph.java --
[42/50] tinkerpop git commit: TINKERPOP-1878 More tests for ordering
TINKERPOP-1878 More tests for ordering Factored the order index creation into its own method to tidy up a bit. Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/81526247 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/81526247 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/81526247 Branch: refs/heads/TINKERPOP-1878 Commit: 81526247d63564e7054068989426d9e38d2cfd02 Parents: 87ccf58 Author: Stephen Mallette Authored: Fri Jan 26 15:56:55 2018 -0500 Committer: Stephen Mallette Committed: Wed Aug 1 09:35:06 2018 -0400 -- .../sparql/SparqlToGremlinTranspiler.java | 29 ++-- .../dsl/sparql/SparqlTraversalSourceTest.java | 6 2 files changed, 26 insertions(+), 9 deletions(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/81526247/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java -- diff --git a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java index ea3f828..c3bc907 100644 --- a/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java +++ b/sparql-gremlin/src/main/java/org/apache/tinkerpop/gremlin/sparql/SparqlToGremlinTranspiler.java @@ -99,15 +99,8 @@ public class SparqlToGremlinTranspiler { arrayOfAllTraversals[traversalIndex++] = tempTrav; } - final Map orderingIndex = new HashMap<>(); - if (query.hasOrderBy()) { -final List sortingConditions = query.getOrderBy(); - - for (SortCondition sortCondition : sortingConditions) { -final Expr expr = sortCondition.getExpression(); -orderingIndex.put(expr.getVarName(), sortCondition.getDirection() == 1 ? Order.incr : Order.decr); - } - } + // creates a map of ordering keys and their ordering direction +final Map orderingIndex = createOrderIndexFromQuery(query); if (traversalList.size() > 0) traversal = traversal.match(arrayOfAllTraversals); @@ -203,6 +196,24 @@ public class SparqlToGremlinTranspiler { return traversal; } +/** + * Extracts any {@code SortCondition} instances from the SPARQL query and holds them in an index of their keys + * where the value is that keys sorting direction. + */ +private static Map createOrderIndexFromQuery(final Query query) { +final Map orderingIndex = new HashMap<>(); +if (query.hasOrderBy()) { +final List sortingConditions = query.getOrderBy(); + +for (SortCondition sortCondition : sortingConditions) { +final Expr expr = sortCondition.getExpression(); +orderingIndex.put(expr.getVarName(), sortCondition.getDirection() == 1 ? Order.incr : Order.decr); +} +} + +return orderingIndex; +} + private static GraphTraversal transpile(final GraphTraversalSource g, final Query query) { return new SparqlToGremlinTranspiler(g).transpile(query); } http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/81526247/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java -- diff --git a/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java b/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java index 2743255..1b39813 100644 --- a/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java +++ b/sparql-gremlin/src/test/java/org/apache/tinkerpop/gremlin/sparql/process/traversal/dsl/sparql/SparqlTraversalSourceTest.java @@ -82,4 +82,10 @@ public class SparqlTraversalSourceTest { }} )); } + +@Test +public void shouldFindAllNamesOrdered() { +final List x = g.sparql("SELECT ?name WHERE { ?person v:name ?name } ORDER BY DESC(?name)").toList(); +assertThat(x, contains("vadas", "ripple", "peter", "marko", "lop", "josh")); +} }
[05/50] tinkerpop git commit: Merge branch 'tp32' into tp33
Merge branch 'tp32' into tp33 Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/79851067 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/79851067 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/79851067 Branch: refs/heads/TINKERPOP-1878 Commit: 798510679b5af84ffc1ecdfa5ef86d52229cbc99 Parents: 9b693c0 3b8c828 Author: Stephen Mallette Authored: Fri Jul 27 16:29:37 2018 -0400 Committer: Stephen Mallette Committed: Fri Jul 27 16:29:37 2018 -0400 -- docs/src/recipes/element-existence.asciidoc | 91 docs/src/recipes/index.asciidoc | 2 + 2 files changed, 93 insertions(+) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/79851067/docs/src/recipes/index.asciidoc --
[09/50] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/4dccf144 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/4dccf144 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/4dccf144 Branch: refs/heads/TINKERPOP-1878 Commit: 4dccf144c71603612ce976827fd24c3eead4fb24 Parents: 6358715 88b6e14 Author: Stephen Mallette Authored: Mon Jul 30 07:16:33 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 07:16:33 2018 -0400 -- gremlin-server/src/main/bin/gremlin-server.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --
[16/50] tinkerpop git commit: Merge branch 'tp33'
Merge branch 'tp33' Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/a2db39cc Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/a2db39cc Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/a2db39cc Branch: refs/heads/TINKERPOP-1878 Commit: a2db39cc710dd2809bab3fb6b2db44257458dcc8 Parents: edd8234 c49c0cc Author: Stephen Mallette Authored: Tue Jul 31 08:35:34 2018 -0400 Committer: Stephen Mallette Committed: Tue Jul 31 08:35:34 2018 -0400 -- docs/src/dev/developer/development-environment.asciidoc | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) --
[08/50] tinkerpop git commit: Merge branch 'tp32' into tp33
Merge branch 'tp32' into tp33 Conflicts: gremlin-server/src/main/bin/gremlin-server.sh Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/88b6e143 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/88b6e143 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/88b6e143 Branch: refs/heads/TINKERPOP-1878 Commit: 88b6e143dd81134074c17e5fa098f03972514273 Parents: 7985106 2b045f3 Author: Stephen Mallette Authored: Mon Jul 30 07:16:27 2018 -0400 Committer: Stephen Mallette Committed: Mon Jul 30 07:16:27 2018 -0400 -- gremlin-server/src/main/bin/gremlin-server.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/88b6e143/gremlin-server/src/main/bin/gremlin-server.sh -- diff --cc gremlin-server/src/main/bin/gremlin-server.sh index fec465c,06b065a..a09e10c --- a/gremlin-server/src/main/bin/gremlin-server.sh +++ b/gremlin-server/src/main/bin/gremlin-server.sh @@@ -90,191 -49,14 +90,191 @@@ els fi # Set Java options -if [ "$JAVA_OPTIONS" = "" ] ; then +if [[ "$JAVA_OPTIONS" = "" ]] ; then - JAVA_OPTIONS="-Xms32m -Xmx512m" + JAVA_OPTIONS="-Xms512m -Xmx4096m" fi -# Execute the application and return its exit code -if [ "$1" = "-i" ]; then - shift - exec $JAVA -Dlog4j.configuration=conf/log4j-server.properties $JAVA_OPTIONS -cp $CP:$CLASSPATH org.apache.tinkerpop.gremlin.server.util.GremlinServerInstall "$@" -else - exec $JAVA -Dlog4j.configuration=conf/log4j-server.properties $JAVA_OPTIONS -cp $CP:$CLASSPATH org.apache.tinkerpop.gremlin.server.GremlinServer "$@" -fi +# Build Java CLASSPATH +CP="$GREMLIN_HOME/conf/" +CP="$CP":$( echo $GREMLIN_HOME/lib/*.jar . | sed 's/ /:/g') +CP="$CP":$( find -L "$GREMLIN_HOME"/ext -mindepth 1 -maxdepth 1 -type d | \ +sort | sed 's/$/\/plugin\/*/' | tr '\n' ':' ) + +CLASSPATH="${CLASSPATH:-}:$CP" + +GREMLIN_SERVER_CMD=org.apache.tinkerpop.gremlin.server.GremlinServer +GREMLIN_INSTALL_CMD=org.apache.tinkerpop.gremlin.server.util.GremlinServerInstall + + +isRunning() { + if [[ -r "$PID_FILE" ]] ; then +PID=$(cat "$PID_FILE") +ps -p "$PID" &> /dev/null +return $? + else +return 1 + fi +} + +status() { + isRunning + RUNNING=$? +if [[ $RUNNING -gt 0 ]]; then + echo Server not running +else + echo Server running with PID $(cat "$PID_FILE") +fi +} + +stop() { + isRunning + RUNNING=$? + if [[ $RUNNING -gt 0 ]]; then +echo Server not running +rm -f "$PID_FILE" + else +kill "$PID" &> /dev/null || { echo "Unable to kill server [$PID]"; exit 1; } +for i in $(seq 1 60); do + ps -p "$PID" &> /dev/null || { echo "Server stopped [$PID]"; rm -f "$PID_FILE"; return 0; } + [[ $i -eq 30 ]] && kill "$PID" &> /dev/null + sleep 1 +done +echo "Unable to kill server [$PID]"; +exit 1; + fi +} + +start() { + isRunning + RUNNING=$? + if [[ $RUNNING -eq 0 ]]; then +echo Server already running with PID $(cat "$PID_FILE"). +exit 1 + fi + + if [[ -z "$RUNAS" ]]; then + +mkdir -p "$LOG_DIR" &>/dev/null +if [[ ! -d "$LOG_DIR" ]]; then + echo ERROR: LOG_DIR $LOG_DIR does not exist and could not be created. + exit 1 +fi + +mkdir -p "$PID_DIR" &>/dev/null +if [[ ! -d "$PID_DIR" ]]; then + echo ERROR: PID_DIR $PID_DIR does not exist and could not be created. + exit 1 +fi + +$JAVA -Dlog4j.configuration=$LOG4J_CONF $JAVA_OPTIONS -cp $CP:$CLASSPATH $GREMLIN_SERVER_CMD "$GREMLIN_YAML" >> "$LOG_FILE" 2>&1 & +PID=$! +disown $PID +echo $PID > "$PID_FILE" + else + +su -c "mkdir -p $LOG_DIR &>/dev/null" "$RUNAS" +if [[ ! -d "$LOG_DIR" ]]; then + echo ERROR: LOG_DIR $LOG_DIR does not exist and could not be created. + exit 1 +fi + +su -c "mkdir -p $PID_DIR &>/dev/null" "$RUNAS" +if [[ ! -d "$PID_DIR" ]]; then + echo ERROR: PID_DIR $PID_DIR does not exist and could not be created. + exit 1 +fi + +su -c "$JAVA -Dlog4j.configuration=$LOG4J_CONF $JAVA_OPTIONS -cp $CP:$CLASSPATH $GREMLIN_SERVER_CMD \"$GREMLIN_YAML\" >> \"$LOG_FILE\" 2>&1 & echo \$! " "$RUNAS" > "$PID_FILE" +chown "$RUNAS" "$PID_FILE" + fi + + isRunning + RUNNING=$? + if [[ $RUNNING -eq 0 ]]; then +echo Server started $(cat "$PID_FILE"). +exit 0 + else +echo Server failed +exit 1 + fi + +} + +startForeground() { + isRunning + RUNNING=$? + if [[ $RUNNING -eq 0 ]]; then +echo Server already running with PID $(cat "$PID_FILE"). +exit 1 + fi + + if [[ -z "$RUNAS" ]]; then +$JAVA