[jira] [Commented] (TINKERPOP-1643) Something's broke in OLAP
[ https://issues.apache.org/jira/browse/TINKERPOP-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860010#comment-16860010 ] Marc de Lignie commented on TINKERPOP-1643: --- The original example including as/select and dedup() can be simplified. All output is from gremlin-console-3.3.7 g = TinkerFactory.createModern().traversal() With emit() in front: gremlin> g.withComputer().V(6).emit().repeat(both()).times(1).count() ==>2 gremlin> g.withComputer().V(6).emit().repeat(both().dedup()).times(1).count() ==>1 ==>1 gremlin> g.withComputer().V(6).emit().repeat(both()).times(1).barrier().count() ==>1 ==>1 With emit() behind: gremlin> g.withComputer().V(6).repeat(both()).times(2).emit().count() ==>4 gremlin> g.withComputer().V(6).repeat(both()).times(2).emit().barrier().count() ==>1 ==>3 The OLAP issue also appears with other barrier() steps in combination with repeat/emit. So, aggregate() displays the issue while store() does not: gremlin> g.withComputer().V(6).emit().repeat(both().aggregate("v")).times(1).cap("v") ==>[v[3]] ==>[v[3]] gremlin> g.withComputer().V(6).emit().repeat(both().store("v")).times(1).cap("v") ==>[v[3]] > Something's broke in OLAP > - > > Key: TINKERPOP-1643 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1643 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.3.0, 3.2.5 >Reporter: Daniel Kuppitz >Assignee: stephen mallette >Priority: Major > > I don't know what it's related to, hence the open title on this ticket. > *Base Query (all good)*: > {noformat} > gremlin> g.V().as("v").emit().repeat(both().as("v").dedup()).select("v") > ==>v[1] > ==>v[2] > ==>v[3] > ==>v[4] > ==>v[5] > ==>v[6] > ==>[v[1],v[4]] > ==>[v[4],v[5]] > ==>[v[3],v[6]] > ==>[v[2],v[1]] > ==>[v[1],v[2]] > ==>[v[1],v[3]] > {noformat} > *Issues*: > {noformat} > gremlin> > g.V().as("v").emit().repeat(both().as("v").dedup()).select("v").count() > ==>6 > ==>6 > {noformat} > {noformat} > gremlin> > g.V().as("v").emit().repeat(both().as("v").dedup()).select("v").order().by(count(local)) > ==>v[1] > ==>v[1] > ==>v[2] > ==>v[2] > ==>v[3] > ==>v[3] > ==>v[4] > ==>v[4] > ==>v[5] > ==>v[5] > ==>v[6] > ==>v[6] > ==>[v[1],v[4]] > ==>[v[4],v[5]] > ==>[v[3],v[6]] > ==>[v[2],v[1]] > ==>[v[1],v[2]] > ==>[v[1],v[3]] > {noformat} > I'm going to create a dev branch and test cases tomorrow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1306) TraversalVertexPrograms don't store halted traversers
[ https://issues.apache.org/jira/browse/TINKERPOP-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16870534#comment-16870534 ] Marc de Lignie commented on TINKERPOP-1306: --- BulkDumperVertexProgram was deprecated in TinkerPop 3.4 so the workaround presented above will not be available anymore in v3.5. > TraversalVertexPrograms don't store halted traversers > - > > Key: TINKERPOP-1306 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1306 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.0-incubating >Reporter: Daniel Kuppitz >Assignee: Marko A. Rodriguez >Priority: Major > > Traversal vertex programs only store their halted traversers if the traversal > contains another vertex program (job chaining). > *Simple traversal (wrong result):* > {noformat} > gremlin> graph = GraphFactory.open("conf/hadoop/hadoop-gryo.properties") > ==>hadoopgraph[gryoinputformat->gryooutputformat] > gremlin> g = > graph.traversal().withComputer(compute().persist(GraphComputer.Persist.EDGES).result(GraphComputer.ResultGraph.NEW)) > ==>graphtraversalsource[hadoopgraph[gryoinputformat->gryooutputformat], > graphcomputer] > gremlin> g.withSack(1f).V().hasLabel("person").as("p"). > gremlin> outE("created").sack(mult).by("weight").inV(). > gremlin> inE("created").sack(mult).by("weight").outV().as("c"). > gremlin> select("p","c").by(id).where("p", lt("c")). > gremlin> sack().as("weight"). > gremlin> select("p").values("name").as("name"). > gremlin> select("c").values("name").as("coworker") > ==>josh > ==>peter > ==>peter > gremlin> hdfs.head("output/~g", GryoInputFormat).each {println > it.property(TraversalVertexProgram.HALTED_TRAVERSERS)} > vp[empty] > vp[empty] > vp[empty] > vp[empty] > vp[empty] > vp[empty] > gremlin> > {noformat} > *With job chaining (correct result):* > {noformat} > gremlin> graph = GraphFactory.open("conf/hadoop/hadoop-gryo.properties") > ==>hadoopgraph[gryoinputformat->gryooutputformat] > gremlin> g = graph.traversal().withComputer() > ==>graphtraversalsource[hadoopgraph[gryoinputformat->gryooutputformat], > graphcomputer] > gremlin> g.withSack(1f).V().hasLabel("person").as("p"). > gremlin> outE("created").sack(mult).by("weight").inV(). > gremlin> inE("created").sack(mult).by("weight").outV().as("c"). > gremlin> select("p","c").by(id).where("p", lt("c")). > gremlin> sack().as("weight"). > gremlin> select("p").values("name").as("name"). > gremlin> select("c").values("name").as("coworker"). > gremlin> program(BulkDumperVertexProgram.build().create(graph)) > ==>josh > ==>peter > ==>peter > gremlin> hdfs.head("output/~g", GryoInputFormat).each {println > it.property(TraversalVertexProgram.HALTED_TRAVERSERS)} > vp[gremlin.traversalVertexProgram.haltedTraversers->[josh]] > vp[empty] > vp[gremlin.traversalVertexProgram.haltedTraversers->[peter, peter]] > vp[empty] > vp[empty] > vp[empty] > gremlin> > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1306) TraversalVertexPrograms don't store halted traversers
[ https://issues.apache.org/jira/browse/TINKERPOP-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16874428#comment-16874428 ] Marc de Lignie commented on TINKERPOP-1306: --- [~spmallette] Yes, I verified that the CloneVertexProgram works allright for the workaround given in the issue above. I just forgot about the CloneVertexProgram but it does not hurt that it is mentioned now in this place. > TraversalVertexPrograms don't store halted traversers > - > > Key: TINKERPOP-1306 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1306 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.0-incubating >Reporter: Daniel Kuppitz >Assignee: Marko A. Rodriguez >Priority: Major > > Traversal vertex programs only store their halted traversers if the traversal > contains another vertex program (job chaining). > *Simple traversal (wrong result):* > {noformat} > gremlin> graph = GraphFactory.open("conf/hadoop/hadoop-gryo.properties") > ==>hadoopgraph[gryoinputformat->gryooutputformat] > gremlin> g = > graph.traversal().withComputer(compute().persist(GraphComputer.Persist.EDGES).result(GraphComputer.ResultGraph.NEW)) > ==>graphtraversalsource[hadoopgraph[gryoinputformat->gryooutputformat], > graphcomputer] > gremlin> g.withSack(1f).V().hasLabel("person").as("p"). > gremlin> outE("created").sack(mult).by("weight").inV(). > gremlin> inE("created").sack(mult).by("weight").outV().as("c"). > gremlin> select("p","c").by(id).where("p", lt("c")). > gremlin> sack().as("weight"). > gremlin> select("p").values("name").as("name"). > gremlin> select("c").values("name").as("coworker") > ==>josh > ==>peter > ==>peter > gremlin> hdfs.head("output/~g", GryoInputFormat).each {println > it.property(TraversalVertexProgram.HALTED_TRAVERSERS)} > vp[empty] > vp[empty] > vp[empty] > vp[empty] > vp[empty] > vp[empty] > gremlin> > {noformat} > *With job chaining (correct result):* > {noformat} > gremlin> graph = GraphFactory.open("conf/hadoop/hadoop-gryo.properties") > ==>hadoopgraph[gryoinputformat->gryooutputformat] > gremlin> g = graph.traversal().withComputer() > ==>graphtraversalsource[hadoopgraph[gryoinputformat->gryooutputformat], > graphcomputer] > gremlin> g.withSack(1f).V().hasLabel("person").as("p"). > gremlin> outE("created").sack(mult).by("weight").inV(). > gremlin> inE("created").sack(mult).by("weight").outV().as("c"). > gremlin> select("p","c").by(id).where("p", lt("c")). > gremlin> sack().as("weight"). > gremlin> select("p").values("name").as("name"). > gremlin> select("c").values("name").as("coworker"). > gremlin> program(BulkDumperVertexProgram.build().create(graph)) > ==>josh > ==>peter > ==>peter > gremlin> hdfs.head("output/~g", GryoInputFormat).each {println > it.property(TraversalVertexProgram.HALTED_TRAVERSERS)} > vp[gremlin.traversalVertexProgram.haltedTraversers->[josh]] > vp[empty] > vp[gremlin.traversalVertexProgram.haltedTraversers->[peter, peter]] > vp[empty] > vp[empty] > vp[empty] > gremlin> > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1306) TraversalVertexPrograms don't store halted traversers
[ https://issues.apache.org/jira/browse/TINKERPOP-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001733#comment-17001733 ] Marc de Lignie commented on TINKERPOP-1306: --- It was noted on the gremlin users list that the workaround with CloneVertexProgram for keeping the halted traversers does not work for PersistedOutputRDD output, see: [https://groups.google.com/forum/#!topic/gremlin-users/NAkYtzXtxZg] > TraversalVertexPrograms don't store halted traversers > - > > Key: TINKERPOP-1306 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1306 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.0-incubating >Reporter: Daniel Kuppitz >Assignee: Marko A. Rodriguez >Priority: Major > > Traversal vertex programs only store their halted traversers if the traversal > contains another vertex program (job chaining). > *Simple traversal (wrong result):* > {noformat} > gremlin> graph = GraphFactory.open("conf/hadoop/hadoop-gryo.properties") > ==>hadoopgraph[gryoinputformat->gryooutputformat] > gremlin> g = > graph.traversal().withComputer(compute().persist(GraphComputer.Persist.EDGES).result(GraphComputer.ResultGraph.NEW)) > ==>graphtraversalsource[hadoopgraph[gryoinputformat->gryooutputformat], > graphcomputer] > gremlin> g.withSack(1f).V().hasLabel("person").as("p"). > gremlin> outE("created").sack(mult).by("weight").inV(). > gremlin> inE("created").sack(mult).by("weight").outV().as("c"). > gremlin> select("p","c").by(id).where("p", lt("c")). > gremlin> sack().as("weight"). > gremlin> select("p").values("name").as("name"). > gremlin> select("c").values("name").as("coworker") > ==>josh > ==>peter > ==>peter > gremlin> hdfs.head("output/~g", GryoInputFormat).each {println > it.property(TraversalVertexProgram.HALTED_TRAVERSERS)} > vp[empty] > vp[empty] > vp[empty] > vp[empty] > vp[empty] > vp[empty] > gremlin> > {noformat} > *With job chaining (correct result):* > {noformat} > gremlin> graph = GraphFactory.open("conf/hadoop/hadoop-gryo.properties") > ==>hadoopgraph[gryoinputformat->gryooutputformat] > gremlin> g = graph.traversal().withComputer() > ==>graphtraversalsource[hadoopgraph[gryoinputformat->gryooutputformat], > graphcomputer] > gremlin> g.withSack(1f).V().hasLabel("person").as("p"). > gremlin> outE("created").sack(mult).by("weight").inV(). > gremlin> inE("created").sack(mult).by("weight").outV().as("c"). > gremlin> select("p","c").by(id).where("p", lt("c")). > gremlin> sack().as("weight"). > gremlin> select("p").values("name").as("name"). > gremlin> select("c").values("name").as("coworker"). > gremlin> program(BulkDumperVertexProgram.build().create(graph)) > ==>josh > ==>peter > ==>peter > gremlin> hdfs.head("output/~g", GryoInputFormat).each {println > it.property(TraversalVertexProgram.HALTED_TRAVERSERS)} > vp[gremlin.traversalVertexProgram.haltedTraversers->[josh]] > vp[empty] > vp[gremlin.traversalVertexProgram.haltedTraversers->[peter, peter]] > vp[empty] > vp[empty] > vp[empty] > gremlin> > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (TINKERPOP-1641) Kerberos authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075096#comment-17075096 ] Marc de Lignie edited comment on TINKERPOP-1641 at 4/4/20, 8:50 AM: Circumstances allowed me to revisit this long standing issue. Initial results with kerberos authentication in gremlin-python can be previewed at [https://github.com/vtslab/incubator-tinkerpop/commits/TINKERPOP-1641]. So, I left kerberos hell behind, but work is still to be done to make this into a complete PR. was (Author: hadoopmarc): Circumstances allowed me to revisit this long standing issue. Initial results with kerberos authentication in gremlin-python can be previewed at [https://github.com/vtslab/incubator-tinkerpop/commits/TINKERPOP-1641|[https://github.com/vtslab/incubator-tinkerpop/commits/TINKERPOP-1641].] So, I left kerberos hell behind, but work is still to be done to make this into a complete PR. > Kerberos authentication for gremlin-python > -- > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Enable kerberos authentication for gremlin-python. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-1641) Kerberos authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075096#comment-17075096 ] Marc de Lignie commented on TINKERPOP-1641: --- Circumstances allowed me to revisit this long standing issue. Initial results with kerberos authentication in gremlin-python can be previewed at [https://github.com/vtslab/incubator-tinkerpop/commits/TINKERPOP-1641|[https://github.com/vtslab/incubator-tinkerpop/commits/TINKERPOP-1641].] So, I left kerberos hell behind, but work is still to be done to make this into a complete PR. > Kerberos authentication for gremlin-python > -- > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Enable kerberos authentication for gremlin-python. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2335) Drop support for older GLV runtimes
[ https://issues.apache.org/jira/browse/TINKERPOP-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075132#comment-17075132 ] Marc de Lignie commented on TINKERPOP-2335: --- Did a fun code search on "print " language:python with one hit: [https://github.com/apache/tinkerpop/blob/master/bin/gephi-mock.py] And setup.py might mention additional newer python versions; I suspect the current limit on python-3.5 is not on purpose. > Drop support for older GLV runtimes > --- > > Key: TINKERPOP-2335 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2335 > Project: TinkerPop > Issue Type: Improvement > Components: dotnet, javascript, python >Reporter: Jorge Bay >Priority: Major > Fix For: 3.5.0 > > > As [discussed on the mailing > list|https://lists.apache.org/thread.html/r25901cdb66b913b32b025970a6cd435eae87609e5c5af3c8a4ce630a%40%3Cdev.tinkerpop.apache.org%3E], > we should take the opportunity of the new revision to drop support for EOL'd > runtimes for javascript, python and .NET. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2335) Drop support for older GLV runtimes
[ https://issues.apache.org/jira/browse/TINKERPOP-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075138#comment-17075138 ] Marc de Lignie commented on TINKERPOP-2335: --- And searching on sys.version_info gives the following python2 remnant (two instances): [https://github.com/apache/tinkerpop/blob/master/gremlin-python/src/main/python/runtest.py] > Drop support for older GLV runtimes > --- > > Key: TINKERPOP-2335 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2335 > Project: TinkerPop > Issue Type: Improvement > Components: dotnet, javascript, python >Reporter: Jorge Bay >Priority: Major > Fix For: 3.5.0 > > > As [discussed on the mailing > list|https://lists.apache.org/thread.html/r25901cdb66b913b32b025970a6cd435eae87609e5c5af3c8a4ce630a%40%3Cdev.tinkerpop.apache.org%3E], > we should take the opportunity of the new revision to drop support for EOL'd > runtimes for javascript, python and .NET. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-1641) Kerberos authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081715#comment-17081715 ] Marc de Lignie commented on TINKERPOP-1641: --- While working on this ticket I realized that python-kerberos does not install on Microsoft Windows, so I made it into an "extra_requires" python dependency. While some other projects offer a winkerberos alternative, I think this is out of scope for TinkerPop because we have no Windows testing infra. Additionally, there is now the Docker alternative, already provided for Gremlin-Console. I am hesitant, however, to create a ticket for a Gremlin-Python docker image, because as a Docker noob I am not going to do this work myself. Anyone else to do this or shall we just wait until someone pops up who supports Microsoft Windows end users? > Kerberos authentication for gremlin-python > -- > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Enable kerberos authentication for gremlin-python. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (TINKERPOP-1641) Kerberos authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081715#comment-17081715 ] Marc de Lignie edited comment on TINKERPOP-1641 at 4/12/20, 9:41 AM: - While working on this ticket I realized that python-kerberos does not install on Microsoft Windows, so I made it into an "extra_require" python dependency. While some other projects offer a winkerberos alternative, I think this is out of scope for TinkerPop because we have no Windows testing infra. Additionally, there is now the Docker alternative, already provided for Gremlin-Console. I am hesitant, however, to create a ticket for a Gremlin-Python docker image, because as a Docker noob I am not going to do this work myself. Anyone else to do this or shall we just wait until someone pops up who supports Microsoft Windows end users? was (Author: hadoopmarc): While working on this ticket I realized that python-kerberos does not install on Microsoft Windows, so I made it into an "extra_requires" python dependency. While some other projects offer a winkerberos alternative, I think this is out of scope for TinkerPop because we have no Windows testing infra. Additionally, there is now the Docker alternative, already provided for Gremlin-Console. I am hesitant, however, to create a ticket for a Gremlin-Python docker image, because as a Docker noob I am not going to do this work myself. Anyone else to do this or shall we just wait until someone pops up who supports Microsoft Windows end users? > Kerberos authentication for gremlin-python > -- > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Enable kerberos authentication for gremlin-python. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-1641) Kerberos authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082382#comment-17082382 ] Marc de Lignie commented on TINKERPOP-1641: --- Indeed I try to say multiple things: * Kerberos for Gremlin-Python won't work on Windows unless you have it install and import the winkerberos python package instead of python-kerberos, depending on the host OS * we have no means to test winkerberos on Windows, so providing a winkerberos option seems too uncomfortable to do * one could still use Kerberos for Gremlin-Python on a Windows machine using a Docker image that runs Gremlin-Python Do you think anyone would want a Docker image running Gremlin-Python if this would provide a way to authenticate to Gremlin-Server using Kerberos AD infrastructure? If so, we can create the ticket as a reminder. > Kerberos authentication for gremlin-python > -- > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Enable kerberos authentication for gremlin-python. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2335) Drop support for older GLV runtimes
[ https://issues.apache.org/jira/browse/TINKERPOP-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082991#comment-17082991 ] Marc de Lignie commented on TINKERPOP-2335: --- The bin/gephi-mocj.py still has a python3 problem. The import statement should read: {color:#80}from {color}http.server {color:#80}import {color}BaseHTTPRequestHandler, HTTPServer I guess that on its turn will break the docker build as it apparently still relies on python2. > Drop support for older GLV runtimes > --- > > Key: TINKERPOP-2335 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2335 > Project: TinkerPop > Issue Type: Improvement > Components: dotnet, javascript, python >Reporter: Jorge Bay >Priority: Major > Fix For: 3.5.0 > > > As [discussed on the mailing > list|https://lists.apache.org/thread.html/r25901cdb66b913b32b025970a6cd435eae87609e5c5af3c8a4ce630a%40%3Cdev.tinkerpop.apache.org%3E], > we should take the opportunity of the new revision to drop support for EOL'd > runtimes for javascript, python and .NET. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2335) Drop support for older GLV runtimes
[ https://issues.apache.org/jira/browse/TINKERPOP-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17085736#comment-17085736 ] Marc de Lignie commented on TINKERPOP-2335: --- Can I bother you once more on this one. I still have problems with "{{docker/build.sh -d"}} after your recent patch. Maven inside docker crashes on the wrong JAVA_HOME. I can only get maven start building in docker if I add "{color:#0073bf}export {color}JAVA_HOME=/usr/lib/jvm/java-8-oracle/jre" to docker/script/build.sh. What is going on here, how can it work for you? Before all this, the .travis.install-maven.sh, called from docker/build/Dockerfile.template, prints to the console during a run: Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) Maven home: /root/mvn-home Java version: 1.8.0_111, vendor: Oracle Corporation Java home: {color:#de350b}*/usr/lib/jvm/java-8-oracle/jre*{color} Default locale: en_US, platform encoding: ANSI_X3.4-1968 OS name: "linux", version: "4.15.0-96-generic", arch: "amd64", family: "unix" So, I guess this maven install gets the wrong JAVA_HOME already. > Drop support for older GLV runtimes > --- > > Key: TINKERPOP-2335 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2335 > Project: TinkerPop > Issue Type: Improvement > Components: dotnet, javascript, python >Reporter: Jorge Bay >Priority: Major > Fix For: 3.5.0 > > > As [discussed on the mailing > list|https://lists.apache.org/thread.html/r25901cdb66b913b32b025970a6cd435eae87609e5c5af3c8a4ce630a%40%3Cdev.tinkerpop.apache.org%3E], > we should take the opportunity of the new revision to drop support for EOL'd > runtimes for javascript, python and .NET. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2335) Drop support for older GLV runtimes
[ https://issues.apache.org/jira/browse/TINKERPOP-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17086009#comment-17086009 ] Marc de Lignie commented on TINKERPOP-2335: --- Thanks, that pointed me in the right direction. > Drop support for older GLV runtimes > --- > > Key: TINKERPOP-2335 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2335 > Project: TinkerPop > Issue Type: Improvement > Components: dotnet, javascript, python >Reporter: Jorge Bay >Priority: Major > Fix For: 3.5.0 > > > As [discussed on the mailing > list|https://lists.apache.org/thread.html/r25901cdb66b913b32b025970a6cd435eae87609e5c5af3c8a4ce630a%40%3Cdev.tinkerpop.apache.org%3E], > we should take the opportunity of the new revision to drop support for EOL'd > runtimes for javascript, python and .NET. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-1641) Kerberos authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087636#comment-17087636 ] Marc de Lignie commented on TINKERPOP-1641: --- You asked me to check docker changes regarding gremlin-server, but I saw some issues there: # docker/gremlin-server.sh 3.4.5 runs fine, but 3.4.6 and 3.5.0-SNAPSHOT warn about the failing gremlin-python install on gremlin-server (which was removed after 3.4.5, I guess) # docker/gremlin-server.sh also starts server-secure on port 45941, but port 45941 was never exposed in the gremlin-server docker-template If you agree, I suppose you rather patch them separately from TINKERPOP-1641. > Kerberos authentication for gremlin-python > -- > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Enable kerberos authentication for gremlin-python. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-1641) Kerberos authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087788#comment-17087788 ] Marc de Lignie commented on TINKERPOP-1641: --- The alternative maven build steps (run after removing all docker images) make no difference. Relevant part of the stacktrace: Installing dependency org.apache.tinkerpop gremlin-python 3.5.0-SNAPSHOT Could not install the dependency: Error grabbing grapes -- /root/.groovy/grapes/resolved-caller-all-caller-working29.) java.lang.RuntimeException: Error grabbing grapes -- /root/.groovy/grapes/resolved-caller-all-caller-working29.xml (N) It seems the groovy error really is due to gremlin-python 3.5.0-SNAPSHOT being non-existent (I also checked that in my local .m2 and .groovy). Actually, this does not kill the container (because of a missing "set -e" bash line) . And 45940 just starts without gremlin-python as it should for this branch. I do not mind if we leave it as is for now (but a comment in the docker-entrypoint.sh would help).-- > Kerberos authentication for gremlin-python > -- > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Enable kerberos authentication for gremlin-python. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (TINKERPOP-1641) Kerberos authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087788#comment-17087788 ] Marc de Lignie edited comment on TINKERPOP-1641 at 4/20/20, 2:35 PM: - The alternative maven build steps (run after removing all docker images) make no difference. Relevant part of the stacktrace: Installing dependency org.apache.tinkerpop gremlin-python 3.5.0-SNAPSHOT Could not install the dependency: Error grabbing grapes -- /root/.groovy/grapes/resolved-caller-all-caller-working29.) java.lang.RuntimeException: Error grabbing grapes -- /root/.groovy/grapes/resolved-caller-all-caller-working29.xml (N) It seems the groovy error really is due to gremlin-python 3.5.0-SNAPSHOT being non-existent (I also checked that in my local .m2 and .groovy). Actually, this does not kill the container (because of a missing "set -e" bash line) . And 45940 just starts without gremlin-python as it should for this branch. I do not mind if we leave it as is for now (but a comment in the docker-entrypoint.sh would help). was (Author: hadoopmarc): The alternative maven build steps (run after removing all docker images) make no difference. Relevant part of the stacktrace: Installing dependency org.apache.tinkerpop gremlin-python 3.5.0-SNAPSHOT Could not install the dependency: Error grabbing grapes -- /root/.groovy/grapes/resolved-caller-all-caller-working29.) java.lang.RuntimeException: Error grabbing grapes -- /root/.groovy/grapes/resolved-caller-all-caller-working29.xml (N) It seems the groovy error really is due to gremlin-python 3.5.0-SNAPSHOT being non-existent (I also checked that in my local .m2 and .groovy). Actually, this does not kill the container (because of a missing "set -e" bash line) . And 45940 just starts without gremlin-python as it should for this branch. I do not mind if we leave it as is for now (but a comment in the docker-entrypoint.sh would help).-- > Kerberos authentication for gremlin-python > -- > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Enable kerberos authentication for gremlin-python. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-1641) Kerberos authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087824#comment-17087824 ] Marc de Lignie commented on TINKERPOP-1641: --- OK. I make the changes (remove install and expose ports) in my branch and will do a final rebase against master before making the PR. I do not know, however, when the PR will be ready. Had also troubles with the recent groovy bump 2.5.7 -> 2.5.11 with groovy 2.5.7 remnants hanging around and causing conflicts. Also, the python3 docs generation is still giving troubles with the build ending without error but the output singlehtml for the ref docs being truncated after the gephi plugin. I will come back to these issues when I have better understanding of what is going on. > Kerberos authentication for gremlin-python > -- > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Enable kerberos authentication for gremlin-python. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-1641) Kerberos authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092218#comment-17092218 ] Marc de Lignie commented on TINKERPOP-1641: --- OK, another and hopefully the last python3 issue in bin/gephi-mock.py. The last line from the repond method should read: {color:#94558d} self{color}.wfile.write({color:#008080}"{}"{color}.encode({color:#008080}'utf-8'{color})) Now, both the python2 and python3 versions always return {} independent of what you hit them with curl. > Kerberos authentication for gremlin-python > -- > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Enable kerberos authentication for gremlin-python. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (TINKERPOP-2462) Duplicated BytecodeUtil and BytecodeHelper classes
Marc de Lignie created TINKERPOP-2462: - Summary: Duplicated BytecodeUtil and BytecodeHelper classes Key: TINKERPOP-2462 URL: https://issues.apache.org/jira/browse/TINKERPOP-2462 Project: TinkerPop Issue Type: Task Components: process Affects Versions: 3.4.8 Reporter: Marc de Lignie Fix For: 3.4.9 The BytecodeUtil and BytecodeHelper classes seem to server the same purpose. Further: * no tests are available for BytecodeHelper * BytecodeHelper.getLambdaLanguage() gives the wrong result for embedded java lambdas because it only detects Lambda.function() instances. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2473) Allow TraversalStrategy instances to be merged
[ https://issues.apache.org/jira/browse/TINKERPOP-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236644#comment-17236644 ] Marc de Lignie commented on TINKERPOP-2473: --- Another indication that some action is needed regarding multiple additions of the same strategy is the inconsistency in behaviour. While SubgraphStrategy has superseding semantics (the last added Strategy takes precedence) the PartionStrategy has ignore semantics (the first added Strategy takes precedence). So, looking back it would have been to just throw an error as default behavior. In the present stage, though, this would be a breaking change. > Allow TraversalStrategy instances to be merged > -- > > Key: TINKERPOP-2473 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2473 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.4.8 >Reporter: Stephen Mallette >Priority: Major > > Not sure if this is a great idea but it came up as part of TINKERPOP-2389 > where there might be a need to assign both a client-side and server-side > {{SubgraphStrategy}}. Currently, while not explicit, you can't assign more > than one strategy of a particular type using {{withStrategies()}}. This task > would make that explicit and provide a direct way for mergeable strategies to > be pushed together into one. If we did that it would also simplify > {{OptionsStrategy}} usage in {{with()}} step which currently finds an > existing one first if present and then adds to it (otherwise creates it new). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (TINKERPOP-2473) Allow TraversalStrategy instances to be merged
[ https://issues.apache.org/jira/browse/TINKERPOP-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236644#comment-17236644 ] Marc de Lignie edited comment on TINKERPOP-2473 at 11/21/20, 11:32 AM: --- Another indication that some action is needed regarding multiple additions of the same strategy is the inconsistency in behavior. While SubgraphStrategy has superseding semantics (the last added Strategy takes precedence) the PartionStrategy has ignore semantics (the first added Strategy takes precedence). So, looking back it would have been better to just throw an error as default behavior. In the present stage, though, this would be a breaking change. was (Author: hadoopmarc): Another indication that some action is needed regarding multiple additions of the same strategy is the inconsistency in behaviour. While SubgraphStrategy has superseding semantics (the last added Strategy takes precedence) the PartionStrategy has ignore semantics (the first added Strategy takes precedence). So, looking back it would have been to just throw an error as default behavior. In the present stage, though, this would be a breaking change. > Allow TraversalStrategy instances to be merged > -- > > Key: TINKERPOP-2473 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2473 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.4.8 >Reporter: Stephen Mallette >Priority: Major > > Not sure if this is a great idea but it came up as part of TINKERPOP-2389 > where there might be a need to assign both a client-side and server-side > {{SubgraphStrategy}}. Currently, while not explicit, you can't assign more > than one strategy of a particular type using {{withStrategies()}}. This task > would make that explicit and provide a direct way for mergeable strategies to > be pushed together into one. If we did that it would also simplify > {{OptionsStrategy}} usage in {{with()}} step which currently finds an > existing one first if present and then adds to it (otherwise creates it new). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2473) Allow TraversalStrategy instances to be merged
[ https://issues.apache.org/jira/browse/TINKERPOP-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237377#comment-17237377 ] Marc de Lignie commented on TINKERPOP-2473: --- OK, great, I already have some documentation written for the Traversal Strategies section (with the current situation) which I expect to push this Friday. If you use that as a basis, I can rebase my PR later on. B.t.w., my PR does not really depend on TINKERPOP-2473, but it would certainly be better if we set this straight right now and end up with consistent behaviour that authorization can rely on. > Allow TraversalStrategy instances to be merged > -- > > Key: TINKERPOP-2473 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2473 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.4.8 >Reporter: Stephen Mallette >Priority: Major > > Not sure if this is a great idea but it came up as part of TINKERPOP-2389 > where there might be a need to assign both a client-side and server-side > {{SubgraphStrategy}}. Currently, while not explicit, you can't assign more > than one strategy of a particular type using {{withStrategies()}}. This task > would make that explicit and provide a direct way for mergeable strategies to > be pushed together into one. If we did that it would also simplify > {{OptionsStrategy}} usage in {{with()}} step which currently finds an > existing one first if present and then adds to it (otherwise creates it new). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2473) Allow TraversalStrategy instances to be merged
[ https://issues.apache.org/jira/browse/TINKERPOP-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17239641#comment-17239641 ] Marc de Lignie commented on TINKERPOP-2473: --- Another issue with the application of traversal strategies is that a restricting strategy such a ReadOnlyStrategy can simply be removed remotely with a withoutStrategies(ReadOnlyStrategy). Although I can imagine cases where you would allow this on purpose, it clearly was not the intention of the authors because there even is a gremlin-server-modern-readonly.yaml example file that suggests that the graph is exposed as readonly. So, I feel there should be a list of restricting strategies (currently {color:#008000}"ReadOnlyStrategy"{color}, {color:#008000}"SubgraphStrategy"{color}, {color:#008000}"PartitionStrategy"{color},{color:#008000}"LambdaRestrictionStrategy"{color}, {color:#008000}"EdgeLabelVerificationStrategy"{color}) that cannot be undone. If admins want to expose a graph with Gremlin Server in both the restricted and the unrestricted way, they can simply provide two differently named GraphTraversalSource instances. I think this is best resolved as part of this issue (TINKERPOP-2473) and implies some changes in the application logic of the traversal strategies. What do you think? The documentation I will push this afternoon as part of TINKERPOP-2389 adds some description about the application of traversal strategies (but covers the present situation), please feel free to use it for documentation of the present issue. > Allow TraversalStrategy instances to be merged > -- > > Key: TINKERPOP-2473 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2473 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.4.8 >Reporter: Stephen Mallette >Priority: Major > > Not sure if this is a great idea but it came up as part of TINKERPOP-2389 > where there might be a need to assign both a client-side and server-side > {{SubgraphStrategy}}. Currently, while not explicit, you can't assign more > than one strategy of a particular type using {{withStrategies()}}. This task > would make that explicit and provide a direct way for mergeable strategies to > be pushed together into one. If we did that it would also simplify > {{OptionsStrategy}} usage in {{with()}} step which currently finds an > existing one first if present and then adds to it (otherwise creates it new). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2473) Allow TraversalStrategy instances to be merged
[ https://issues.apache.org/jira/browse/TINKERPOP-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17239694#comment-17239694 ] Marc de Lignie commented on TINKERPOP-2473: --- Yes, indeed, "must-be-present" is close to "must-not-be-removed". You are also right about the workaround, although the Gremlin Server docs now describe "implicit", "passive" and "active" authorization strategies, where the implicit one does not use the Authorizer interface but relies on restricting traversal strategies that can not be removed or replaced. So, it is does not seem consistent to release an Authorizer feature without resolving TINKERPOP-2473/TINKERPOP-669. > Allow TraversalStrategy instances to be merged > -- > > Key: TINKERPOP-2473 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2473 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.4.8 >Reporter: Stephen Mallette >Priority: Major > > Not sure if this is a great idea but it came up as part of TINKERPOP-2389 > where there might be a need to assign both a client-side and server-side > {{SubgraphStrategy}}. Currently, while not explicit, you can't assign more > than one strategy of a particular type using {{withStrategies()}}. This task > would make that explicit and provide a direct way for mergeable strategies to > be pushed together into one. If we did that it would also simplify > {{OptionsStrategy}} usage in {{with()}} step which currently finds an > existing one first if present and then adds to it (otherwise creates it new). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (TINKERPOP-2485) Invalid http tests with ?gremlin=1-1
Marc de Lignie created TINKERPOP-2485: - Summary: Invalid http tests with ?gremlin=1-1 Key: TINKERPOP-2485 URL: https://issues.apache.org/jira/browse/TINKERPOP-2485 Project: TinkerPop Issue Type: Task Components: server Affects Versions: 3.4.8, 3.3.11 Reporter: Marc de Lignie Various tests with ?gremlin=1-1 are compared against node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).intValue() or node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}) .get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}) .intValue() which accidentally result in the 0 value. Replace with ?gremlin=2-1 and node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).intValue() -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TINKERPOP-2485) Invalid http tests with ?gremlin=1-1
[ https://issues.apache.org/jira/browse/TINKERPOP-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc de Lignie updated TINKERPOP-2485: -- Description: Various tests with ?gremlin=1-1 are compared against node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).intValue() or node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}) .intValue() which accidentally result in the 0 value. Replace with ?gremlin=2-1 and node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).intValue() was: Various tests with ?gremlin=1-1 are compared against node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).intValue() or node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}) .get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}) .intValue() which accidentally result in the 0 value. Replace with ?gremlin=2-1 and node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).intValue() > Invalid http tests with ?gremlin=1-1 > > > Key: TINKERPOP-2485 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2485 > Project: TinkerPop > Issue Type: Task > Components: server >Affects Versions: 3.3.11, 3.4.8 >Reporter: Marc de Lignie >Priority: Trivial > > Various tests with ?gremlin=1-1 are compared against > node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).intValue() > or > node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}) > .intValue() which accidentally result in the 0 value. Replace with > ?gremlin=2-1 and > node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).intValue() -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TINKERPOP-2485) Invalid http tests with ?gremlin=1-1
[ https://issues.apache.org/jira/browse/TINKERPOP-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc de Lignie updated TINKERPOP-2485: -- Description: Various tests with ?gremlin=1-1 are compared against node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).intValue() or node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}).intValue() which accidentally result in the 0 value. Replace with ?gremlin=2-1 and node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).intValue() was: Various tests with ?gremlin=1-1 are compared against node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).intValue() or node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}) .intValue() which accidentally result in the 0 value. Replace with ?gremlin=2-1 and node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).intValue() > Invalid http tests with ?gremlin=1-1 > > > Key: TINKERPOP-2485 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2485 > Project: TinkerPop > Issue Type: Task > Components: server >Affects Versions: 3.3.11, 3.4.8 >Reporter: Marc de Lignie >Priority: Trivial > > Various tests with ?gremlin=1-1 are compared against > > node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).intValue() > or > > node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}).intValue() > which accidentally result in the 0 value. > Replace with ?gremlin=2-1 and > > node.get({color:#008000}"result"{color}).get({color:#008000}"data"{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).get({color:#ff}0{color}).get(GraphSONTokens.{color:#660e7a}VALUEPROP{color}).intValue() -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2504) Intermittently failing server/driver integration tests
[ https://issues.apache.org/jira/browse/TINKERPOP-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314909#comment-17314909 ] Marc de Lignie commented on TINKERPOP-2504: --- OK, I can contribute in investigating this issue. First, I asked myself why failures are more common on Travis, so I added a docker command line option --cpus={color:#008000}".9"{color}{color:#172b4d} to the docker/build.sh. Without Stephen's thread.sleep(), this has a "docker/build.sh -t" fail consistently in the gremlin groovy module (see two example stack traces below), while it runs fine with my four available CPU cores. I will move over to the Krb5Authenticator part and try and give it the same treat. I leave gremlin groovy to you.{color} Example 1 [ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 17.613 s <<< FAILURE! - in org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutorOverGraphTest [ERROR] shouldAllowTraversalToIterateInDifferentThreadThanOriginallyEvaluatedWithAutoCommit(org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutorOverGraphTest) Time elapsed: 9.827 s <<< ERROR! java.util.concurrent.CompletionException: java.util.concurrent.TimeoutException: Evaluation exceeded the configured 'evaluationTimeout' threshold of 8000 ms or evaluation was otherwise cancelled directly for request [g.V().out()] at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346) at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:704) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$1(GremlinExecutor.java:320) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.util.concurrent.TimeoutException: Evaluation exceeded the configured 'evaluationTimeout' threshold of 8000 ms or evaluation was otherwise cancelled directly for request [g.V().out()] at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$1(GremlinExecutor.java:321) ... 6 more Example 2 [ERROR] Tests run: 37, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 68.996 s <<< FAILURE! - in org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutorTest [ERROR] shouldEvalScriptWithBindings(org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutorTest) Time elapsed: 8.893 s <<< ERROR! java.util.concurrent.ExecutionException: java.util.concurrent.TimeoutException: Evaluation exceeded the configured 'evaluationTimeout' threshold of 8000 ms or evaluation was otherwise cancelled directly for request [1+x] at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999) at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutorTest.shouldEvalScriptWithBindings(GremlinExecutorTest.java:170) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.Pa
[jira] [Comment Edited] (TINKERPOP-2504) Intermittently failing server/driver integration tests
[ https://issues.apache.org/jira/browse/TINKERPOP-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17318834#comment-17318834 ] Marc de Lignie edited comment on TINKERPOP-2504 at 4/11/21, 3:34 PM: - I pushed [some changes|[https://github.com/vtslab/incubator-tinkerpop/tree/TINKERPOP-2504]] that make the GremlinServerAuthKrb5IntegrateTest, GremlinServerAuditLogIntegrateTest and GremlinServerAuditLogDeprecatedIntegrateTest succeed in a cocker container with --cpus=".9" . This still does not complete the gremlin-server build, so their may be other problems (see below). Also, my changes only work around some other issue with gremlin-driver that has a blocking thread on connection failure or does not have the right timeout configs (but I guess we do not have to solve this here). For completeness I list the failsafe-report from the docker container while it hangs on some other issue: root@50b4e89febc5:/usr/src/tinkerpop/gremlin-server/target# ls failsafe-reports/*.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinResultSetIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogDeprecatedIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthKrb5IntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthzIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerSessionIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerSslIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinSessionTxIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.HttpChannelizerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizerIntegrateTest.txt root@50b4e89febc5:/usr/src/tinkerpop/gremlin-server/target# Probably it fails on GremlinDriverIntegrateTest which also uses the addContactPoint() method. was (Author: hadoopmarc): I pushed [some changes|[https://github.com/vtslab/incubator-tinkerpop/tree/TINKERPOP-2504]|https://github.com/vtslab/incubator-tinkerpop/tree/TINKERPOP-2504] that make the GremlinServerAuthKrb5IntegrateTest, GremlinServerAuditLogIntegrateTest and GremlinServerAuditLogDeprecatedIntegrateTest succeed in a cocker container with --cpus=".9" . This still does not complete the gremlin-server build, so their may be other problems (see below). Also, my changes only work around some other issue with gremlin-driver that has a blocking thread on connection failure or does not have the right timeout configs (but I guess we do not have to solve this here). For completeness I list the failsafe-report from the docker container while it hangs on some other issue: root@50b4e89febc5:/usr/src/tinkerpop/gremlin-server/target# ls failsafe-reports/*.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinResultSetIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogDeprecatedIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthKrb5IntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthzIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerSessionIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerSslIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinSessionTxIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.HttpChannelizerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizerIntegrateTest.txt root@50b4e89febc5:/usr/src/tinkerpop/gremlin-server/target# Probably it fails on GremlinDriverIntegrateTest which also uses the addContactPoint() method. > Intermittently failing server/driver integration tests > -- > > Key: TINKERPOP-2504 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2504 > Project: TinkerPop >
[jira] [Commented] (TINKERPOP-2504) Intermittently failing server/driver integration tests
[ https://issues.apache.org/jira/browse/TINKERPOP-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17318834#comment-17318834 ] Marc de Lignie commented on TINKERPOP-2504: --- I pushed [some changes|[https://github.com/vtslab/incubator-tinkerpop/tree/TINKERPOP-2504]|https://github.com/vtslab/incubator-tinkerpop/tree/TINKERPOP-2504] that make the GremlinServerAuthKrb5IntegrateTest, GremlinServerAuditLogIntegrateTest and GremlinServerAuditLogDeprecatedIntegrateTest succeed in a cocker container with --cpus=".9" . This still does not complete the gremlin-server build, so their may be other problems (see below). Also, my changes only work around some other issue with gremlin-driver that has a blocking thread on connection failure or does not have the right timeout configs (but I guess we do not have to solve this here). For completeness I list the failsafe-report from the docker container while it hangs on some other issue: root@50b4e89febc5:/usr/src/tinkerpop/gremlin-server/target# ls failsafe-reports/*.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinResultSetIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogDeprecatedIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthKrb5IntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthzIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerSessionIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerSslIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinSessionTxIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.HttpChannelizerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizerIntegrateTest.txt root@50b4e89febc5:/usr/src/tinkerpop/gremlin-server/target# Probably it fails on GremlinDriverIntegrateTest which also uses the addContactPoint() method. > Intermittently failing server/driver integration tests > -- > > Key: TINKERPOP-2504 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2504 > Project: TinkerPop > Issue Type: Improvement > Components: driver, server >Affects Versions: 3.4.9 >Reporter: Stephen Mallette >Priority: Minor > > I've noticed this test failing intermittently on Travis and more consistently > on the {{master}} branch with Docker. It fails with a > {{ConcurrentModificationException}} (haven't been able to easily get the > entire stack trace thanks to the docker issue and i've not caught it > happening in the last few days on Travis). Adding this line of code seems to > have made the test pass more consistently: > https://github.com/apache/tinkerpop/commit/4b099b3c84a350aae953cdf517aa11c7017eb2ae > which would indicate something perhaps fishy with how hosts are being marked > dead and iterated. Would be nice to get rid of that little hack. > Another common failure that is fairly consistent is with > {{GremlinServerAuditLogIntegrateTest.shouldAuditLogWithKrb5Authenticator}} > {code} > [ERROR] > shouldAuditLogWithKrb5Authenticator(org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest) > Time elapsed: 3.16 s <<< ERROR! > java.util.concurrent.ExecutionException: > org.apache.tinkerpop.gremlin.driver.exception.ResponseException: > Authenticator is not ready to handle requests > at > org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest.shouldAuditLogWithKrb5Authenticator(GremlinServerAuditLogIntegrateTest.java:222) > Caused by: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: > Authenticator is not ready to handle requests > {code} > cc/ [~divijvaidya] [~HadoopMarc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (TINKERPOP-2504) Intermittently failing server/driver integration tests
[ https://issues.apache.org/jira/browse/TINKERPOP-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17318834#comment-17318834 ] Marc de Lignie edited comment on TINKERPOP-2504 at 4/11/21, 3:36 PM: - I pushed [some changes|[https://github.com/vtslab/incubator-tinkerpop/tree/TINKERPOP-2504]] that make the GremlinServerAuthKrb5IntegrateTest, GremlinServerAuditLogIntegrateTest and GremlinServerAuditLogDeprecatedIntegrateTest succeed in a docker container with --cpus=".9" . This still does not complete the gremlin-server build, so their may be other problems (see below). Also, my changes only work around some other issue with gremlin-driver that has a blocking thread on connection failure or does not have the right timeout configs (but I guess we do not have to solve this here). For completeness I list the failsafe-report from the docker container while it hangs on some other issue: root@50b4e89febc5:/usr/src/tinkerpop/gremlin-server/target# ls failsafe-reports/*.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinResultSetIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogDeprecatedIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthKrb5IntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthzIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerSessionIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerSslIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinSessionTxIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.HttpChannelizerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizerIntegrateTest.txt root@50b4e89febc5:/usr/src/tinkerpop/gremlin-server/target# Probably it fails on GremlinDriverIntegrateTest which also uses the addContactPoint() method. was (Author: hadoopmarc): I pushed [some changes|[https://github.com/vtslab/incubator-tinkerpop/tree/TINKERPOP-2504]] that make the GremlinServerAuthKrb5IntegrateTest, GremlinServerAuditLogIntegrateTest and GremlinServerAuditLogDeprecatedIntegrateTest succeed in a cocker container with --cpus=".9" . This still does not complete the gremlin-server build, so their may be other problems (see below). Also, my changes only work around some other issue with gremlin-driver that has a blocking thread on connection failure or does not have the right timeout configs (but I guess we do not have to solve this here). For completeness I list the failsafe-report from the docker container while it hangs on some other issue: root@50b4e89febc5:/usr/src/tinkerpop/gremlin-server/target# ls failsafe-reports/*.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinResultSetIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogDeprecatedIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthKrb5IntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerAuthzIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerSessionIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinServerSslIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.GremlinSessionTxIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.HttpChannelizerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizerIntegrateTest.txt failsafe-reports/org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizerIntegrateTest.txt root@50b4e89febc5:/usr/src/tinkerpop/gremlin-server/target# Probably it fails on GremlinDriverIntegrateTest which also uses the addContactPoint() method. > Intermittently failing server/driver integration tests > -- > > Key: TINKERPOP-2504 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2504 > Project: TinkerPop > Issue Type: Improvement >
[jira] [Commented] (TINKERPOP-2504) Intermittently failing server/driver integration tests
[ https://issues.apache.org/jira/browse/TINKERPOP-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17319354#comment-17319354 ] Marc de Lignie commented on TINKERPOP-2504: --- Hi [~spmallette] , my assumption is that the routine travis failures coincide with difficulties to handle low resources, at least that seemed the case for the integration tests I investigated above. Indeed, my suggestion is to handle the present issue with workarounds where possible and to generates new tickets for the underlying issues. "Low resource test failures" might be too general. The specific issue I mentioned relates to the gremlin server TestClientFactory (based on the gremlin-driver Cluster client) blocking an entire cpu core per destination that is not reachable. > Intermittently failing server/driver integration tests > -- > > Key: TINKERPOP-2504 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2504 > Project: TinkerPop > Issue Type: Improvement > Components: driver, server >Affects Versions: 3.4.9 >Reporter: Stephen Mallette >Priority: Minor > > I've noticed this test failing intermittently on Travis and more consistently > on the {{master}} branch with Docker. It fails with a > {{ConcurrentModificationException}} (haven't been able to easily get the > entire stack trace thanks to the docker issue and i've not caught it > happening in the last few days on Travis). Adding this line of code seems to > have made the test pass more consistently: > https://github.com/apache/tinkerpop/commit/4b099b3c84a350aae953cdf517aa11c7017eb2ae > which would indicate something perhaps fishy with how hosts are being marked > dead and iterated. Would be nice to get rid of that little hack. > Another common failure that is fairly consistent is with > {{GremlinServerAuditLogIntegrateTest.shouldAuditLogWithKrb5Authenticator}} > {code} > [ERROR] > shouldAuditLogWithKrb5Authenticator(org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest) > Time elapsed: 3.16 s <<< ERROR! > java.util.concurrent.ExecutionException: > org.apache.tinkerpop.gremlin.driver.exception.ResponseException: > Authenticator is not ready to handle requests > at > org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest.shouldAuditLogWithKrb5Authenticator(GremlinServerAuditLogIntegrateTest.java:222) > Caused by: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: > Authenticator is not ready to handle requests > {code} > cc/ [~divijvaidya] [~HadoopMarc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2504) Intermittently failing server/driver integration tests
[ https://issues.apache.org/jira/browse/TINKERPOP-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1732#comment-1732 ] Marc de Lignie commented on TINKERPOP-2504: --- Confident is not the word, but there is no other way to test, see [https://github.com/apache/tinkerpop/pull/1417/checks?check_run_id=2370517254|[https://github.com/apache/tinkerpop/pull/1417/checks?check_run_id=2370517254].] The changes I made to the TestClientFactory make some tests more resilient against low available cpu resources. The old failures in gremlin-server integration tests from the git log on master mostly have to do with the teardown of the KdcFixture, so I removed some suspicious code lines. If errors about deleting principals keep popping up, you could simply catch these exceptions and only log them (I did not want to use that anti-pattern in the current stage). > Intermittently failing server/driver integration tests > -- > > Key: TINKERPOP-2504 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2504 > Project: TinkerPop > Issue Type: Improvement > Components: driver, server >Affects Versions: 3.4.9 >Reporter: Stephen Mallette >Priority: Minor > > I've noticed this test failing intermittently on Travis and more consistently > on the {{master}} branch with Docker. It fails with a > {{ConcurrentModificationException}} (haven't been able to easily get the > entire stack trace thanks to the docker issue and i've not caught it > happening in the last few days on Travis). Adding this line of code seems to > have made the test pass more consistently: > https://github.com/apache/tinkerpop/commit/4b099b3c84a350aae953cdf517aa11c7017eb2ae > which would indicate something perhaps fishy with how hosts are being marked > dead and iterated. Would be nice to get rid of that little hack. > Another common failure that is fairly consistent is with > {{GremlinServerAuditLogIntegrateTest.shouldAuditLogWithKrb5Authenticator}} > {code} > [ERROR] > shouldAuditLogWithKrb5Authenticator(org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest) > Time elapsed: 3.16 s <<< ERROR! > java.util.concurrent.ExecutionException: > org.apache.tinkerpop.gremlin.driver.exception.ResponseException: > Authenticator is not ready to handle requests > at > org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest.shouldAuditLogWithKrb5Authenticator(GremlinServerAuditLogIntegrateTest.java:222) > Caused by: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: > Authenticator is not ready to handle requests > {code} > cc/ [~divijvaidya] [~HadoopMarc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2504) Intermittently failing server/driver integration tests
[ https://issues.apache.org/jira/browse/TINKERPOP-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17326277#comment-17326277 ] Marc de Lignie commented on TINKERPOP-2504: --- Great, glad you found the inspiration! I suppose that means that your earlier suggestion to cherrypick [https://github.com/apache/tinkerpop/commit/56e4f3f84eca9993089225e2f6f7cd0f01c30a9a] into [https://github.com/apache/tinkerpop/pull/1417] is stale now. > Intermittently failing server/driver integration tests > -- > > Key: TINKERPOP-2504 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2504 > Project: TinkerPop > Issue Type: Improvement > Components: driver, server >Affects Versions: 3.4.9 >Reporter: Stephen Mallette >Priority: Minor > > I've noticed this test failing intermittently on Travis and more consistently > on the {{master}} branch with Docker. It fails with a > {{ConcurrentModificationException}} (haven't been able to easily get the > entire stack trace thanks to the docker issue and i've not caught it > happening in the last few days on Travis). Adding this line of code seems to > have made the test pass more consistently: > https://github.com/apache/tinkerpop/commit/4b099b3c84a350aae953cdf517aa11c7017eb2ae > which would indicate something perhaps fishy with how hosts are being marked > dead and iterated. Would be nice to get rid of that little hack. > Another common failure that is fairly consistent is with > {{GremlinServerAuditLogIntegrateTest.shouldAuditLogWithKrb5Authenticator}} > {code} > [ERROR] > shouldAuditLogWithKrb5Authenticator(org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest) > Time elapsed: 3.16 s <<< ERROR! > java.util.concurrent.ExecutionException: > org.apache.tinkerpop.gremlin.driver.exception.ResponseException: > Authenticator is not ready to handle requests > at > org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest.shouldAuditLogWithKrb5Authenticator(GremlinServerAuditLogIntegrateTest.java:222) > Caused by: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: > Authenticator is not ready to handle requests > {code} > cc/ [~divijvaidya] [~HadoopMarc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2504) Intermittently failing server/driver integration tests
[ https://issues.apache.org/jira/browse/TINKERPOP-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17326849#comment-17326849 ] Marc de Lignie commented on TINKERPOP-2504: --- OK CI is running on the squashed commits. I guess you did not mean "git rebase -i TINKERPOP-2550" because our branches differ by 800+ commits, but I am sure you will find your way. > Intermittently failing server/driver integration tests > -- > > Key: TINKERPOP-2504 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2504 > Project: TinkerPop > Issue Type: Improvement > Components: driver, server >Affects Versions: 3.4.9 >Reporter: Stephen Mallette >Priority: Minor > > I've noticed this test failing intermittently on Travis and more consistently > on the {{master}} branch with Docker. It fails with a > {{ConcurrentModificationException}} (haven't been able to easily get the > entire stack trace thanks to the docker issue and i've not caught it > happening in the last few days on Travis). Adding this line of code seems to > have made the test pass more consistently: > https://github.com/apache/tinkerpop/commit/4b099b3c84a350aae953cdf517aa11c7017eb2ae > which would indicate something perhaps fishy with how hosts are being marked > dead and iterated. Would be nice to get rid of that little hack. > Another common failure that is fairly consistent is with > {{GremlinServerAuditLogIntegrateTest.shouldAuditLogWithKrb5Authenticator}} > {code} > [ERROR] > shouldAuditLogWithKrb5Authenticator(org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest) > Time elapsed: 3.16 s <<< ERROR! > java.util.concurrent.ExecutionException: > org.apache.tinkerpop.gremlin.driver.exception.ResponseException: > Authenticator is not ready to handle requests > at > org.apache.tinkerpop.gremlin.server.GremlinServerAuditLogIntegrateTest.shouldAuditLogWithKrb5Authenticator(GremlinServerAuditLogIntegrateTest.java:222) > Caused by: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: > Authenticator is not ready to handle requests > {code} > cc/ [~divijvaidya] [~HadoopMarc] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (TINKERPOP-2680) Create call() step to allow for calling procedures
[ https://issues.apache.org/jira/browse/TINKERPOP-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471104#comment-17471104 ] Marc de Lignie edited comment on TINKERPOP-2680 at 1/8/22, 11:27 AM: - Please clean up the description for this issue (I have no rights to modify it myself). E.g.: The Gremlin query language does not include a step to call procedures that are unique to an implementation. Adding such a step allows implementations to leverage Gremlin and offer a single interface for users to interact with the systems comprising the implementation, such as storage and indexing backends. With this feature we will create a step that allows implementations to pass on requests to the implementation's inner services. was (Author: hadoopmarc): Please clean up the description for this (I have no rights to modify it myself). E.g.: The Gremlin query language does not include a step to call procedures that are unique to an implementation. Adding such a step allows implementations to leverage Gremlin and offer a single interface for users to interact with the systems comprising the implementation, such as storage and indexing backends. With this feature we will create a step that allows implementations to pass on requests to the implementation's inner services. > Create call() step to allow for calling procedures > -- > > Key: TINKERPOP-2680 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2680 > Project: TinkerPop > Issue Type: Improvement > Components: language, process >Affects Versions: 3.5.1 >Reporter: Dave Bechberger >Priority: Major > > One of the gaps in the Gremlin query language is a step that enables > implemented to call procedures that are unique to that specific database. > Adding this ability allows implementers to leverage Gremlin by enabling them > to leverage it as a single interface for users to interact with the database > without having to separate interfaces to handle database-specific operations. > > With this feature we will create a step that allows implementors to provide > database specific functionality. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (TINKERPOP-2680) Create call() step to allow for calling procedures
[ https://issues.apache.org/jira/browse/TINKERPOP-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471104#comment-17471104 ] Marc de Lignie commented on TINKERPOP-2680: --- Please clean up the description for this (I have no rights to modify it myself). E.g.: The Gremlin query language does not include a step to call procedures that are unique to an implementation. Adding such a step allows implementations to leverage Gremlin and offer a single interface for users to interact with the systems comprising the implementation, such as storage and indexing backends. With this feature we will create a step that allows implementations to pass on requests to the implementation's inner services. > Create call() step to allow for calling procedures > -- > > Key: TINKERPOP-2680 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2680 > Project: TinkerPop > Issue Type: Improvement > Components: language, process >Affects Versions: 3.5.1 >Reporter: Dave Bechberger >Priority: Major > > One of the gaps in the Gremlin query language is a step that enables > implemented to call procedures that are unique to that specific database. > Adding this ability allows implementers to leverage Gremlin by enabling them > to leverage it as a single interface for users to interact with the database > without having to separate interfaces to handle database-specific operations. > > With this feature we will create a step that allows implementors to provide > database specific functionality. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (TINKERPOP-2743) Gremlin-python ref docs use with_* and without_*
Marc de Lignie created TINKERPOP-2743: - Summary: Gremlin-python ref docs use with_* and without_* Key: TINKERPOP-2743 URL: https://issues.apache.org/jira/browse/TINKERPOP-2743 Project: TinkerPop Issue Type: Bug Affects Versions: 3.6.0 Reporter: Marc de Lignie Fix For: 3.6.1 In the reference docs gremlin-python steps with* and without* are represented as with_* and without_*, e.g. with_remote should be withRemote, without_strategies should be withoutStrategies. It feels like some auto formatter that ran wild, so possibly more harm was done. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (TINKERPOP-1852) Recipe fails for highly meshed connected components
Marc de Lignie created TINKERPOP-1852: - Summary: Recipe fails for highly meshed connected components Key: TINKERPOP-1852 URL: https://issues.apache.org/jira/browse/TINKERPOP-1852 Project: TinkerPop Issue Type: Bug Components: documentation Affects Versions: 3.2.6, 3.3.0 Environment: gremlin-console Reporter: Marc de Lignie Priority: Minor On the JanusGraph user list the connected-components recipe was shown to fail for a completely meshed component with a size of 35. The successive thread[https://groups.google.com/forum/#!topic/janusgraph-users/NmwZyag1w2M] shows a better, though mixed groovy-gremlin recipe. A pure gremlin solution along the lines provided probably exists. In particaluar I did not try group('m').by().by() yet to store components with (k,v) = (vertex, list of vertices per component). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (TINKERPOP-1852) Recipe fails for highly meshed connected components
[ https://issues.apache.org/jira/browse/TINKERPOP-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc de Lignie updated TINKERPOP-1852: -- Description: On the JanusGraph user list the connected-components recipe was shown to fail for a completely meshed component with a size of 35. The successive thread [https://groups.google.com/forum/#!topic/janusgraph-users/NmwZyag1w2M] shows a better, though mixed groovy-gremlin recipe. A pure gremlin solution along the lines provided probably exists. In particaluar I did not try group('m').by().by() yet to store components with (k,v) = (vertex, list of vertices per component). (was: On the JanusGraph user list the connected-components recipe was shown to fail for a completely meshed component with a size of 35. The successive thread[https://groups.google.com/forum/#!topic/janusgraph-users/NmwZyag1w2M] shows a better, though mixed groovy-gremlin recipe. A pure gremlin solution along the lines provided probably exists. In particaluar I did not try group('m').by().by() yet to store components with (k,v) = (vertex, list of vertices per component).) > Recipe fails for highly meshed connected components > --- > > Key: TINKERPOP-1852 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1852 > Project: TinkerPop > Issue Type: Bug > Components: documentation >Affects Versions: 3.3.0, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Priority: Minor > > On the JanusGraph user list the connected-components recipe was shown to fail > for a completely meshed component with a size of 35. The successive thread > [https://groups.google.com/forum/#!topic/janusgraph-users/NmwZyag1w2M] shows > a better, though mixed groovy-gremlin recipe. A pure gremlin solution along > the lines provided probably exists. In particaluar I did not try > group('m').by().by() yet to store components with (k,v) = (vertex, list of > vertices per component). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (TINKERPOP-1852) Recipe fails for highly meshed connected components
[ https://issues.apache.org/jira/browse/TINKERPOP-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc de Lignie updated TINKERPOP-1852: -- Description: On the JanusGraph user list the connected-components recipe was shown to fail for a completely meshed component with a size of 35. The successive thread [https://groups.google.com/forum/#!topic/janusgraph-users/NmwZyag1w2M] shows a better solution, both as a mixed groovy-gremlin and a pure gremlin query (with the latter having slightly less well scaling behavior because of the required path computations). (was: On the JanusGraph user list the connected-components recipe was shown to fail for a completely meshed component with a size of 35. The successive thread [https://groups.google.com/forum/#!topic/janusgraph-users/NmwZyag1w2M] shows a better, though mixed groovy-gremlin recipe. A pure gremlin solution along the lines provided probably exists. In particaluar I did not try group('m').by().by() yet to store components with (k,v) = (vertex, list of vertices per component).) > Recipe fails for highly meshed connected components > --- > > Key: TINKERPOP-1852 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1852 > Project: TinkerPop > Issue Type: Bug > Components: documentation >Affects Versions: 3.3.0, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Priority: Minor > > On the JanusGraph user list the connected-components recipe was shown to fail > for a completely meshed component with a size of 35. The successive thread > [https://groups.google.com/forum/#!topic/janusgraph-users/NmwZyag1w2M] shows > a better solution, both as a mixed groovy-gremlin and a pure gremlin query > (with the latter having slightly less well scaling behavior because of the > required path computations). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1852) Recipe fails for highly meshed connected components
[ https://issues.apache.org/jira/browse/TINKERPOP-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285838#comment-16285838 ] Marc de Lignie commented on TINKERPOP-1852: --- One question I have: can the current recipe be updated without providing a solution for OLAP? I do not feel comfortable in just suggesting that it might work for SparkGraphComputer without extensive testing that it has any value over the OLTP case. Also I feel that a proper distributed algo should provide performance benefits (py parallellization) over an OLTP case for a graph that still fits in memory. Further, it should be shown that an OLAP algo for connected components has the same order of magnitude performance as the one in Apache Spark Graphx. That being said I would prefer updating the OLTP case and leaving the OLAP case as work in progress (while removing the old OLAP example). > Recipe fails for highly meshed connected components > --- > > Key: TINKERPOP-1852 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1852 > Project: TinkerPop > Issue Type: Bug > Components: documentation >Affects Versions: 3.3.0, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Priority: Minor > > On the JanusGraph user list the connected-components recipe was shown to fail > for a completely meshed component with a size of 35. The successive thread > [https://groups.google.com/forum/#!topic/janusgraph-users/NmwZyag1w2M] shows > a better solution, both as a mixed groovy-gremlin and a pure gremlin query > (with the latter having slightly less well scaling behavior because of the > required path computations). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288230#comment-16288230 ] Marc de Lignie commented on TINKERPOP-1786: --- Hi Marko, I saw your note, I will take a look, Marc > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Assignee: Marko A. Rodriguez >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288234#comment-16288234 ] Marc de Lignie commented on TINKERPOP-1786: --- I mean Daniel, not Marko... > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Assignee: Marko A. Rodriguez >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289731#comment-16289731 ] Marc de Lignie commented on TINKERPOP-1786: --- A nasty one, sorry about the inconvenience. Marc > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Assignee: Marko A. Rodriguez >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1641) Authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355054#comment-16355054 ] Marc de Lignie commented on TINKERPOP-1641: --- Hi Stephen, to be honest, my current interest shifted to https://issues.apache.org/jira/browse/TINKERPOP-1852 . Apache Directory-Kerby had two releases in the mean time (see issue above), so anyone could pick up this work now. I might pick it up somewhere later. > Authentication for gremlin-python > - > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Follow-up on > [TINKERPOP-1566|https://issues.apache.org/jira/browse/TINKERPOP-1566]: > * make gremlin-python authenticate against SimpleAuthenticator and > Krb5Authenticator > * add audit logging for bytecode requests (in addition to existing audit > logging of scrip requests and of the authenticated user by TINKERPOP-1566) > * tests and documentation for this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1757) GiraphGraphComputer does not start Zookeeper in local mode
[ https://issues.apache.org/jira/browse/TINKERPOP-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426019#comment-16426019 ] Marc de Lignie commented on TINKERPOP-1757: --- I am afraid that not the fix is trivial but rather the issue itself, in that it is very unlikely to hit any real usage of TinkerPop. So, I prefer the issue to be closed in Unresolved state with the upcoming deprecation + the availability of a workaround as the motivation. > GiraphGraphComputer does not start Zookeeper in local mode > -- > > Key: TINKERPOP-1757 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1757 > Project: TinkerPop > Issue Type: Bug > Components: hadoop >Affects Versions: 3.3.0, 3.2.6 > Environment: gremlin-console in Ubuntu with Oracle java 8 >Reporter: Marc de Lignie >Priority: Trivial > > As discussed in > [https://groups.google.com/forum/#!topic/gremlin-users/Ztbr_uJhoxY] running > GiraphGraphComputer in local mode (without an external or pseudo hadoop > cluster) requires the following configured properties (add section local to > the hadoop properties file): > {code:java} > gremlin.hadoop.inputLocation=data/tinkerpop-modern.kryo > giraph.minWorkers=1 > giraph.maxWorkers=1 > giraph.SplitMasterWorker=false > mapreduce.job.maps=4 > mapreduce.job.reduces=2 > {code} > The real problem is that GiraphGraphComputer does not manage to start its > Zookeeper service because it gets relative classpath items from somewhere > (gremlin-console provides the acual shell command used to attempt starting > Zookeeper in its logging). > Current workaround is to start gremlin-console as follows: > {code:java} > export CLASSPATH=$PWD/lib/* > bin/gremlin.sh > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482474#comment-16482474 ] Marc de Lignie commented on TINKERPOP-1967: --- I pushed my work on [#TINKERPOP-1852] to [https://github.com/vtslab/incubator-tinkerpop/tree/components|[https://github.com/vtslab/incubator-tinkerpop/tree/components].] It includes the WeakComponentsVertexProgram + tests as well as the improved recipe for connected components with ideas from the gremlin user list and the new vertex program. I got stuck on also wanting to include an algo for storage-backed graphs like JanusGraph, but this new Jira issue prompted me to no longer pursue this. Largest issue right now to finish the work is to have the vertex program tested on the friendster graph (with known published outcome). Small issues are: 1) not passing the revapi check and 2) not having updated yet the vertexprogram with [#TINKERPOP-1862]. @Stephen, are you, as assignee, also working on this? > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482474#comment-16482474 ] Marc de Lignie edited comment on TINKERPOP-1967 at 5/21/18 1:10 PM: I pushed my work on [https://issues.apache.org/jira/browse/TINKERPOP-1852|TINKERPOP-1852] to [https://github.com/vtslab/incubator-tinkerpop/tree/components|[https://github.com/vtslab/incubator-tinkerpop/tree/components].] It includes the WeakComponentsVertexProgram + tests as well as the improved recipe for connected components with ideas from the gremlin user list and the new vertex program. I got stuck on also wanting to include an algo for storage-backed graphs like JanusGraph, but this new Jira issue prompted me to no longer pursue this. Largest issue right now to finish the work is to have the vertex program tested on the friendster graph (with known published outcome). Small issues are: 1) not passing the revapi check and 2) not having updated yet the vertexprogram with [https://issues.apache.org/jira/browse/TINKERPOP-1862|TINKERPOP-1862]. @Stephen, are you, as assignee, also working on this? was (Author: hadoopmarc): I pushed my work on [#TINKERPOP-1852] to [https://github.com/vtslab/incubator-tinkerpop/tree/components|[https://github.com/vtslab/incubator-tinkerpop/tree/components].] It includes the WeakComponentsVertexProgram + tests as well as the improved recipe for connected components with ideas from the gremlin user list and the new vertex program. I got stuck on also wanting to include an algo for storage-backed graphs like JanusGraph, but this new Jira issue prompted me to no longer pursue this. Largest issue right now to finish the work is to have the vertex program tested on the friendster graph (with known published outcome). Small issues are: 1) not passing the revapi check and 2) not having updated yet the vertexprogram with [#TINKERPOP-1862]. @Stephen, are you, as assignee, also working on this? > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482474#comment-16482474 ] Marc de Lignie edited comment on TINKERPOP-1967 at 5/21/18 1:11 PM: I pushed my work on [https://issues.apache.org/jira/browse/TINKERPOP-1852|TINKERPOP-1852] to [https://github.com/vtslab/incubator-tinkerpop/tree/components|https://github.com/vtslab/incubator-tinkerpop/tree/components]. It includes the WeakComponentsVertexProgram + tests as well as the improved recipe for connected components with ideas from the gremlin user list and the new vertex program. I got stuck on also wanting to include an algo for storage-backed graphs like JanusGraph, but this new Jira issue prompted me to no longer pursue this. Largest issue right now to finish the work is to have the vertex program tested on the friendster graph (with known published outcome). Small issues are: 1) not passing the revapi check and 2) not having updated yet the vertexprogram with [https://issues.apache.org/jira/browse/TINKERPOP-1862|TINKERPOP-1862]. @Stephen, are you, as assignee, also working on this? was (Author: hadoopmarc): I pushed my work on [https://issues.apache.org/jira/browse/TINKERPOP-1852|TINKERPOP-1852] to [https://github.com/vtslab/incubator-tinkerpop/tree/components|[https://github.com/vtslab/incubator-tinkerpop/tree/components].] It includes the WeakComponentsVertexProgram + tests as well as the improved recipe for connected components with ideas from the gremlin user list and the new vertex program. I got stuck on also wanting to include an algo for storage-backed graphs like JanusGraph, but this new Jira issue prompted me to no longer pursue this. Largest issue right now to finish the work is to have the vertex program tested on the friendster graph (with known published outcome). Small issues are: 1) not passing the revapi check and 2) not having updated yet the vertexprogram with [https://issues.apache.org/jira/browse/TINKERPOP-1862|TINKERPOP-1862]. @Stephen, are you, as assignee, also working on this? > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482925#comment-16482925 ] Marc de Lignie commented on TINKERPOP-1967: --- I gave your feature branch a quick scan: * your {{ConnectedComponentVertexProgram}} is clearly much more developed with all the traversal steps present * I am not so sure we can simply dismiss the OLTP traversal as educational only after the improvements Robert Dale and I made (see the code in the recipe of my feature branch). I would like to run some performance tests on in-memory TinkerGraphs to be sure. If there is still use for the OLTP query, my take on the recipe would provide useful text (apart from the external references and the explanation about weakly connected components) * Also for your code it would be worthwhile to run a one-off test with the friendster graph (I plan to do this, but no problem if you chime in on this). * It is a good idea indeed to mention that the vertexprogram will be present since 3.4.0, because users of older versions may yet land on the latest ref docs. > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482925#comment-16482925 ] Marc de Lignie edited comment on TINKERPOP-1967 at 5/21/18 7:15 PM: I gave your feature branch a quick scan: * your {{ConnectedComponentVertexProgram}} is clearly much more developed with all the traversal steps present * I am not so sure we can simply dismiss the OLTP traversal as educational only after the improvements Robert Dale and I made (see the code in the recipe of my feature branch). I would like to run some performance tests on in-memory TinkerGraphs to be sure. If there is still use for the OLTP query, my take on the recipe would provide useful text (in addition to its external references and the explanation about weakly connected components) * Also for your code it would be worthwhile to run a one-off test with the friendster graph (I plan to do this, but no problem if you chime in on this). * It is a good idea indeed to mention that the vertexprogram will be present since 3.4.0, because users of older versions may yet land on the latest ref docs. was (Author: hadoopmarc): I gave your feature branch a quick scan: * your {{ConnectedComponentVertexProgram}} is clearly much more developed with all the traversal steps present * I am not so sure we can simply dismiss the OLTP traversal as educational only after the improvements Robert Dale and I made (see the code in the recipe of my feature branch). I would like to run some performance tests on in-memory TinkerGraphs to be sure. If there is still use for the OLTP query, my take on the recipe would provide useful text (apart from the external references and the explanation about weakly connected components) * Also for your code it would be worthwhile to run a one-off test with the friendster graph (I plan to do this, but no problem if you chime in on this). * It is a good idea indeed to mention that the vertexprogram will be present since 3.4.0, because users of older versions may yet land on the latest ref docs. > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16502391#comment-16502391 ] Marc de Lignie commented on TINKERPOP-1967: --- Quite busy at the moment, with other things... I put in a day or so with the Friendster graph but was not lucky, A single 48Gb machine really could not dig it (endless spilling for lack of memory). Hortonworks (the software on the cluster I can access) enriched their hadoop-2.7 with a hadoop-2.9 feature that made it incompatible with TinkerPop's vanilla hadoop and an attempt to build TinkerPop with the HDP artifacts proved unsuccesful. I quit the Friendster graph for now and will do the hopefully easier comparison between the OLTP query and the connectedComponent() step on a few atificial graphs of increasing size. If the OLTP query still seems useful, it will not be much work to contibrute the documentation I prepared. If not, my contribution has not much added value. I will try to find some time. > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503837#comment-16503837 ] Marc de Lignie commented on TINKERPOP-1967: --- The comparison between the OLTP query and the connectedComponent step was more productive. Below you see figures for graphs with random components of fixed size with a edge/vertex ratio of 6, for which the two algorithms have the same order of magnitude runtime. This means the OLTP query could still be useful for networks with long strings of vertices. So, I will do some more work to produce some comparison graphs, add them to my work and rework them into a PR against TTINKERPOP-1967. OLTP: Duration: 0.064 Component sizes: [100, 100, 100, 100, 100, 100, 100, 100, 100, 100] Step: Duration: 0.052 Component sizes: [100, 100, 100, 100, 100, 100, 100, 100, 100, 100] OLTP: Duration: 0.134 Component sizes: [200, 200, 200, 200, 200, 200, 200, 200, 200, 200] Step: Duration: 0.096 Component sizes: [200, 200, 200, 200, 200, 200, 200, 200, 200, 200] OLTP: Duration: 0.269 Component sizes: [400, 400, 400, 400, 400, 400, 400, 400, 400, 400] Step: Duration: 0.129 Component sizes: [400, 400, 400, 400, 400, 400, 400, 400, 400, 400] OLTP: Duration: 0.585 Component sizes: [800, 800, 800, 800, 800, 800, 800, 800, 800, 800] Step: Duration: 0.252 Component sizes: [800, 800, 800, 800, 800, 800, 800, 800, 800, 800] OLTP: Duration: 1.183 Component sizes: [1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600] Step: Duration: 0.559 Component sizes: [1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600] OLTP: Duration: 2.375 Component sizes: [3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200] Step: Duration: 1.125 Component sizes: [3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200] OLTP: Duration: 4.986 Component sizes: [6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400] Step: Duration: 2.326 Component sizes: [6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400] OLTP: Duration: 10.507 Component sizes: [12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800] Step: Duration: 6.308 Component sizes: [12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800] OLTP: Duration: 22.265 Component sizes: [25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600] Step: Duration: 16.212 Component sizes: [25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600] > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503837#comment-16503837 ] Marc de Lignie edited comment on TINKERPOP-1967 at 6/6/18 7:57 PM: --- The comparison between the OLTP query and the connectedComponent step was more productive. Below you see figures for graphs with random components of fixed size with an edge/vertex ratio of 6, for which the two algorithms have the same order of magnitude runtime. This means the OLTP query could still be useful for networks with long strings of vertices. So, I will do some more work to produce some comparison graphs, add them to my work and rework them into a PR against TINKERPOP-1967. OLTP: Duration: 0.064 Component sizes: [100, 100, 100, 100, 100, 100, 100, 100, 100, 100] Step: Duration: 0.052 Component sizes: [100, 100, 100, 100, 100, 100, 100, 100, 100, 100] OLTP: Duration: 0.134 Component sizes: [200, 200, 200, 200, 200, 200, 200, 200, 200, 200] Step: Duration: 0.096 Component sizes: [200, 200, 200, 200, 200, 200, 200, 200, 200, 200] OLTP: Duration: 0.269 Component sizes: [400, 400, 400, 400, 400, 400, 400, 400, 400, 400] Step: Duration: 0.129 Component sizes: [400, 400, 400, 400, 400, 400, 400, 400, 400, 400] OLTP: Duration: 0.585 Component sizes: [800, 800, 800, 800, 800, 800, 800, 800, 800, 800] Step: Duration: 0.252 Component sizes: [800, 800, 800, 800, 800, 800, 800, 800, 800, 800] OLTP: Duration: 1.183 Component sizes: [1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600] Step: Duration: 0.559 Component sizes: [1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600] OLTP: Duration: 2.375 Component sizes: [3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200] Step: Duration: 1.125 Component sizes: [3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200] OLTP: Duration: 4.986 Component sizes: [6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400] Step: Duration: 2.326 Component sizes: [6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400] OLTP: Duration: 10.507 Component sizes: [12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800] Step: Duration: 6.308 Component sizes: [12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800] OLTP: Duration: 22.265 Component sizes: [25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600] Step: Duration: 16.212 Component sizes: [25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600] was (Author: hadoopmarc): The comparison between the OLTP query and the connectedComponent step was more productive. Below you see figures for graphs with random components of fixed size with a edge/vertex ratio of 6, for which the two algorithms have the same order of magnitude runtime. This means the OLTP query could still be useful for networks with long strings of vertices. So, I will do some more work to produce some comparison graphs, add them to my work and rework them into a PR against TTINKERPOP-1967. OLTP: Duration: 0.064 Component sizes: [100, 100, 100, 100, 100, 100, 100, 100, 100, 100] Step: Duration: 0.052 Component sizes: [100, 100, 100, 100, 100, 100, 100, 100, 100, 100] OLTP: Duration: 0.134 Component sizes: [200, 200, 200, 200, 200, 200, 200, 200, 200, 200] Step: Duration: 0.096 Component sizes: [200, 200, 200, 200, 200, 200, 200, 200, 200, 200] OLTP: Duration: 0.269 Component sizes: [400, 400, 400, 400, 400, 400, 400, 400, 400, 400] Step: Duration: 0.129 Component sizes: [400, 400, 400, 400, 400, 400, 400, 400, 400, 400] OLTP: Duration: 0.585 Component sizes: [800, 800, 800, 800, 800, 800, 800, 800, 800, 800] Step: Duration: 0.252 Component sizes: [800, 800, 800, 800, 800, 800, 800, 800, 800, 800] OLTP: Duration: 1.183 Component sizes: [1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600] Step: Duration: 0.559 Component sizes: [1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600, 1600] OLTP: Duration: 2.375 Component sizes: [3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200] Step: Duration: 1.125 Component sizes: [3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200, 3200] OLTP: Duration: 4.986 Component sizes: [6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400] Step: Duration: 2.326 Component sizes: [6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400, 6400] OLTP: Duration: 10.507 Component sizes: [12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800] Step: Duration: 6.308 Component sizes: [12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800, 12800] OLTP: Duration: 22.265 Component sizes: [25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600] Step: Duration: 16.212 Component sizes: [25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600, 25600] > Add a connectedComponent() step > --- > > Key
[jira] [Commented] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505181#comment-16505181 ] Marc de Lignie commented on TINKERPOP-1967: --- [~spmallette] The code for the VertexProgramStrategy suggest that you can configure the number of workers that is used by the TinkerGraphComputer when calling one of the xyzVertexProgramSteps. Are the graph properties for this documented anywhere? Also, I saw that the ClusterCountMapReduce and ClusterPopulationMapReduce classes are not usable for the ConnectedComponentVertexProgram. Was that a true choice (given that these classes predate the VertexProgram steps). With the ComputerProgram builder it was easy to configure the number of workers. > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505181#comment-16505181 ] Marc de Lignie edited comment on TINKERPOP-1967 at 6/7/18 8:01 PM: --- [~spmallette] The code for the VertexProgramStrategy suggest that you can configure the number of workers that is used by the TinkerGraphComputer when calling one of the xyzVertexProgramSteps. Are the graph properties for this documented anywhere? Also, I saw that the ClusterCountMapReduce and ClusterPopulationMapReduce classes are not usable for the ConnectedComponentVertexProgram. Was that a true choice (given that these classes predate the VertexProgram steps). With the GraphComputer builder it was easy to configure the number of workers. was (Author: hadoopmarc): [~spmallette] The code for the VertexProgramStrategy suggest that you can configure the number of workers that is used by the TinkerGraphComputer when calling one of the xyzVertexProgramSteps. Are the graph properties for this documented anywhere? Also, I saw that the ClusterCountMapReduce and ClusterPopulationMapReduce classes are not usable for the ConnectedComponentVertexProgram. Was that a true choice (given that these classes predate the VertexProgram steps). With the ComputerProgram builder it was easy to configure the number of workers. > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508618#comment-16508618 ] Marc de Lignie commented on TINKERPOP-1967: --- [~spmallette] I gave it a look. When I pulled your branch the connectedComponent commit was dated at May 17 with hash f91d3d9d21f1e921a071200668b0fd0e33b321a8, see: [https://github.com/vtslab/incubator-tinkerpop/commits/TINKERPOP-1967-vtslab] When I look at the current TINKERPOP-1967 branch, the connectedComponent commit is dated around May 6 with hash fc0b2dc5c2f0ecce9a1690df36dc9b061dad5d1b, see: [https://github.com/apache/tinkerpop/commits/TINKERPOP-1967] Did you change history? Anyway, I guess you want the PR to be based on the actual history :) > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508618#comment-16508618 ] Marc de Lignie edited comment on TINKERPOP-1967 at 6/11/18 7:37 PM: [~spmallette] I gave it a look. When I pulled your branch the connectedComponent commit was dated at May 17 with hash f91d3d9d21f1e921a071200668b0fd0e33b321a8, see: [https://github.com/vtslab/incubator-tinkerpop/commits/TINKERPOP-1967-vtslab] When I look at the current TINKERPOP-1967 branch, the connectedComponent commit is dated around May 6 with hash fc0b2dc5c2f0ecce9a1690df36dc9b061dad5d1b, see: [https://github.com/apache/tinkerpop/commits/TINKERPOP-1967] Did you change history? Anyway, I guess you want the PR to be based on the current history :) was (Author: hadoopmarc): [~spmallette] I gave it a look. When I pulled your branch the connectedComponent commit was dated at May 17 with hash f91d3d9d21f1e921a071200668b0fd0e33b321a8, see: [https://github.com/vtslab/incubator-tinkerpop/commits/TINKERPOP-1967-vtslab] When I look at the current TINKERPOP-1967 branch, the connectedComponent commit is dated around May 6 with hash fc0b2dc5c2f0ecce9a1690df36dc9b061dad5d1b, see: [https://github.com/apache/tinkerpop/commits/TINKERPOP-1967] Did you change history? Anyway, I guess you want the PR to be based on the actual history :) > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508618#comment-16508618 ] Marc de Lignie edited comment on TINKERPOP-1967 at 6/11/18 7:51 PM: [~spmallette] I gave it a look. When I pulled your branch the connectedComponent commit was dated at May 17 with hash f91d3d9d21f1e921a071200668b0fd0e33b321a8, see: [https://github.com/vtslab/incubator-tinkerpop/commits/TINKERPOP-1967-vtslab] When I look at the current TINKERPOP-1967 branch, the connectedComponent commit is dated at May 17 with hash fc0b2dc5c2f0ecce9a1690df36dc9b061dad5d1b, see: [https://github.com/apache/tinkerpop/commits/TINKERPOP-1967] Did you change history? Anyway, I guess you want the PR to be based on the current history :) was (Author: hadoopmarc): [~spmallette] I gave it a look. When I pulled your branch the connectedComponent commit was dated at May 17 with hash f91d3d9d21f1e921a071200668b0fd0e33b321a8, see: [https://github.com/vtslab/incubator-tinkerpop/commits/TINKERPOP-1967-vtslab] When I look at the current TINKERPOP-1967 branch, the connectedComponent commit is dated around May 6 with hash fc0b2dc5c2f0ecce9a1690df36dc9b061dad5d1b, see: [https://github.com/apache/tinkerpop/commits/TINKERPOP-1967] Did you change history? Anyway, I guess you want the PR to be based on the current history :) > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508703#comment-16508703 ] Marc de Lignie commented on TINKERPOP-1967: --- [~spmallette] Merging my commits on current 1967 is painless, however, docs generation from console now fails on the ref/traversal doc in the connectedComponent part (which I did not change). Have to look into this some other time before I create the updated PR. > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1967) Add a connectedComponent() step
[ https://issues.apache.org/jira/browse/TINKERPOP-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510014#comment-16510014 ] Marc de Lignie commented on TINKERPOP-1967: --- [~spmallette] The recent with() step commit really has the building of the traversal.asciidoc fail, both in master and in TINKERPOP-1967. Tag 3.3.3 builds fine with bin/process-docs.sh in my environment. So, I will wait for resolution before I rebase my commits for TINKERPOP-1967. I also tried "{{docker/build.sh -d", but that failed on gremlin-python irrespective of the TinkerPop version:}} {{[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.8:run (setup-py-env) on project gremlin-python: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "virtualenv" (in directory "/usr/src/tinkerpop/gremlin-python/target/python2"): error=2, No such file or directory}} {{[ERROR] around Ant part .. @ 13:107 in /usr/src/tinkerpop/gremlin-python/target/antrun/build-main.xml}} > Add a connectedComponent() step > --- > > Key: TINKERPOP-1967 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1967 > Project: TinkerPop > Issue Type: Improvement > Components: process >Affects Versions: 3.3.3 >Reporter: stephen mallette >Assignee: stephen mallette >Priority: Minor > Fix For: 3.4.0 > > > Given TINKERPOP-1852 we should probably just simplify and improve performance > of connected component identification. Implementing this will involved the > creation of {{ConnectedComponentVertexProgram}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-1495) Global list deduplication doesn't work in OLAP
[ https://issues.apache.org/jira/browse/TINKERPOP-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565366#comment-15565366 ] Marc de Lignie commented on TINKERPOP-1495: --- This one looks a lot simpler: gremlin> graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties') ==>hadoopgraph[gryoinputformat->gryooutputformat] gremlin> g = graph.traversal(computer(SparkGraphComputer)) ==>graphtraversalsource[hadoopgraph[gryoinputformat->gryooutputformat], sparkgraphcomputer] gremlin> g.V().groupCount().select(values).unfold().dedup() ==>1 ==>1 ==>1 ==>1 ==>1 ==>1 In TinkerGraph this gives a single "1". OLAP via remote connection also fails. > Global list deduplication doesn't work in OLAP > -- > > Key: TINKERPOP-1495 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1495 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.2 >Reporter: Daniel Kuppitz >Assignee: Marko A. Rodriguez > > {noformat} > gremlin> g = TinkerFactory.createModern().traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> a = TinkerFactory.createModern().traversal().withComputer() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], graphcomputer] > gremlin> > gremlin> > g.V().as("a").repeat(both()).times(3).emit().as("b").group().by(select("a")).by(select("b").dedup().order().by(id).fold()).select(values).unfold().dedup() > ==>[v[1],v[2],v[3],v[4],v[5],v[6]] > gremlin> > a.V().as("a").repeat(both()).times(3).emit().as("b").group().by(select("a")).by(select("b").dedup().order().by(id).fold()).select(values).unfold().dedup() > ==>[v[1],v[2],v[3],v[4],v[5],v[6]] > ==>[v[1],v[2],v[3],v[4],v[5],v[6]] > ==>[v[1],v[2],v[3],v[4],v[5],v[6]] > ==>[v[1],v[3],v[4],v[5],v[6]] > ==>[v[1],v[2],v[3],v[4],v[6]] > ==>[v[1],v[2],v[3],v[4],v[5]] > gremlin> > {noformat} > _Not tested in {{tp31}}._ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1495) Global list deduplication doesn't work in OLAP
[ https://issues.apache.org/jira/browse/TINKERPOP-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565442#comment-15565442 ] Marc de Lignie commented on TINKERPOP-1495: --- Maybe the following is handy for comparison. This works ok: gremlin> :> g.V().map{it -> 1}.dedup() ==>1 > Global list deduplication doesn't work in OLAP > -- > > Key: TINKERPOP-1495 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1495 > Project: TinkerPop > Issue Type: Bug > Components: process >Affects Versions: 3.2.2 >Reporter: Daniel Kuppitz >Assignee: Marko A. Rodriguez > > {noformat} > gremlin> g = TinkerFactory.createModern().traversal() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard] > gremlin> a = TinkerFactory.createModern().traversal().withComputer() > ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], graphcomputer] > gremlin> > gremlin> > g.V().as("a").repeat(both()).times(3).emit().as("b").group().by(select("a")).by(select("b").dedup().order().by(id).fold()).select(values).unfold().dedup() > ==>[v[1],v[2],v[3],v[4],v[5],v[6]] > gremlin> > a.V().as("a").repeat(both()).times(3).emit().as("b").group().by(select("a")).by(select("b").dedup().order().by(id).fold()).select(values).unfold().dedup() > ==>[v[1],v[2],v[3],v[4],v[5],v[6]] > ==>[v[1],v[2],v[3],v[4],v[5],v[6]] > ==>[v[1],v[2],v[3],v[4],v[5],v[6]] > ==>[v[1],v[3],v[4],v[5],v[6]] > ==>[v[1],v[2],v[3],v[4],v[6]] > ==>[v[1],v[2],v[3],v[4],v[5]] > gremlin> > {noformat} > _Not tested in {{tp31}}._ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TINKERPOP-1566) Kerberos authentication for gremlin-server
Marc de Lignie created TINKERPOP-1566: - Summary: Kerberos authentication for gremlin-server Key: TINKERPOP-1566 URL: https://issues.apache.org/jira/browse/TINKERPOP-1566 Project: TinkerPop Issue Type: Improvement Components: server Reporter: Marc de Lignie Priority: Minor Gremlin server would benefit from an explicit Kerberos authentication plugin, because preparing and maintaining such a plugin is nontrivial. Also, many other Apache project provide kerberized services. In gremlin-console the standard Krb5LoginModule can be configured. Gremlin-server already includes the pluggable Sasl framework that can host the proposed Kerberos authentication plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server
[ https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15691288#comment-15691288 ] Marc de Lignie commented on TINKERPOP-1566: --- This issue is a follow-up on a short discussion on the dev list: https://lists.apache.org/thread.html/dde14d6c7fd660f94de326761d72c482fa4f08b40ebfed1772d8ea14@%3Cdev.tinkerpop.apache.org%3E I will add a link to my git repo later on, so that the work on this issue can be followed. > Kerberos authentication for gremlin-server > -- > > Key: TINKERPOP-1566 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1566 > Project: TinkerPop > Issue Type: Improvement > Components: server >Affects Versions: 3.2.3 >Reporter: Marc de Lignie >Priority: Minor > Labels: security > > Gremlin server would benefit from an explicit Kerberos authentication plugin, > because preparing and maintaining such a plugin is nontrivial. Also, many > other Apache project provide kerberized services. > In gremlin-console the standard Krb5LoginModule can be configured. > Gremlin-server already includes the pluggable Sasl framework that can host > the proposed Kerberos authentication plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TINKERPOP-1566) Kerberos authentication for gremlin-server
[ https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc de Lignie updated TINKERPOP-1566: -- Affects Version/s: (was: 3.2.3) Fix Version/s: 3.3.0 > Kerberos authentication for gremlin-server > -- > > Key: TINKERPOP-1566 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1566 > Project: TinkerPop > Issue Type: Improvement > Components: server >Reporter: Marc de Lignie >Priority: Minor > Labels: security > Fix For: 3.3.0 > > > Gremlin server would benefit from an explicit Kerberos authentication plugin, > because preparing and maintaining such a plugin is nontrivial. Also, many > other Apache project provide kerberized services. > In gremlin-console the standard Krb5LoginModule can be configured. > Gremlin-server already includes the pluggable Sasl framework that can host > the proposed Kerberos authentication plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server
[ https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695818#comment-15695818 ] Marc de Lignie commented on TINKERPOP-1566: --- A first test demonstrating Kerberos authentication between gremlin-driver and gremlin-server is available at: https://github.com/vtslab/incubator-tinkerpop/tree/TINKERPOP-1566/gremlin-server Relevant tests are: SaslAppTest (a copied smoketest for the kdc text fixture) AuthKrb5IntegrateTest (which tests the new Krb5Authenticator class) Still lots of work to do regarding configuration, clean-up, additional tests and documentation. > Kerberos authentication for gremlin-server > -- > > Key: TINKERPOP-1566 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1566 > Project: TinkerPop > Issue Type: Improvement > Components: server >Reporter: Marc de Lignie >Priority: Minor > Labels: security > Fix For: 3.3.0 > > > Gremlin server would benefit from an explicit Kerberos authentication plugin, > because preparing and maintaining such a plugin is nontrivial. Also, many > other Apache project provide kerberized services. > In gremlin-console the standard Krb5LoginModule can be configured. > Gremlin-server already includes the pluggable Sasl framework that can host > the proposed Kerberos authentication plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server
[ https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15699683#comment-15699683 ] Marc de Lignie commented on TINKERPOP-1566: --- Gremlin-console can only be configured for Kerberos authentication when a "remote objects" connection is set up to gremlin server. With the "serializeResultsToString" option, the Kerberos authentication fails with a "java.lang.String cannot be cast to a byte[]" in the driver Handler, line 108 (actually, I only tested this in 3.1.1, but the offending evaluateChallenge call is the same). I suppose this is a bug, since the presentation of query results in the console is not supposed to interfere with the authentication mechanism configured. @[~spmallette] Do you agree? How do you prefer this to be handled, as a separate issue (because it also effects 3.1.x and 3.2.x) or as part of the current TINKERPOP-1566 issue? Cheers,Marc > Kerberos authentication for gremlin-server > -- > > Key: TINKERPOP-1566 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1566 > Project: TinkerPop > Issue Type: Improvement > Components: server >Reporter: Marc de Lignie >Priority: Minor > Labels: security > Fix For: 3.3.0 > > > Gremlin server would benefit from an explicit Kerberos authentication plugin, > because preparing and maintaining such a plugin is nontrivial. Also, many > other Apache project provide kerberized services. > In gremlin-console the standard Krb5LoginModule can be configured. > Gremlin-server already includes the pluggable Sasl framework that can host > the proposed Kerberos authentication plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server
[ https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15701853#comment-15701853 ] Marc de Lignie commented on TINKERPOP-1566: --- I added a failing test on branch TINKERPOP-1566 (almost equal to master) regarding the serialization/casting issue for Kerberos challenges, reported here yesterday: https://github.com/vtslab/incubator-tinkerpop/commit/da53479ade8bd586ab48878577eefa74b1d446ac Just uncomment shouldAuthenticateWithSerializeResultToString and run. Cheers,Marc > Kerberos authentication for gremlin-server > -- > > Key: TINKERPOP-1566 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1566 > Project: TinkerPop > Issue Type: Improvement > Components: server >Reporter: Marc de Lignie >Priority: Minor > Labels: security > Fix For: 3.3.0 > > > Gremlin server would benefit from an explicit Kerberos authentication plugin, > because preparing and maintaining such a plugin is nontrivial. Also, many > other Apache project provide kerberized services. > In gremlin-console the standard Krb5LoginModule can be configured. > Gremlin-server already includes the pluggable Sasl framework that can host > the proposed Kerberos authentication plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server
[ https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702232#comment-15702232 ] Marc de Lignie commented on TINKERPOP-1566: --- Gremlin-server instances in a Kerberos realm will typically need audit logging. I can think of three approaches for this: 1. Add debug messages relevant to audit logging 2. Add info messages relevant to audit logging 3. Conditionally add info messages relevant to audit logging Approach 1. is a no go (debug messages are not for production). Approach 2. is a no go too (logging personal information, also for those who not need it). Therefore, I propose using approach 3, with two conditional logger.info() statements added, one in SaslAuthenticationHandler that binds the username to its remote socket address, and one in AbstractEvalOpProcessor that binds each gremlin request to the channel's remote socket address. Here, conditional means something like the presence of a system property in the JAVA_OPTIONS: -Dgremlin.server.security.audit=true Agree? Cheers, Marc > Kerberos authentication for gremlin-server > -- > > Key: TINKERPOP-1566 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1566 > Project: TinkerPop > Issue Type: Improvement > Components: server >Reporter: Marc de Lignie >Priority: Minor > Labels: security > Fix For: 3.3.0 > > > Gremlin server would benefit from an explicit Kerberos authentication plugin, > because preparing and maintaining such a plugin is nontrivial. Also, many > other Apache project provide kerberized services. > In gremlin-console the standard Krb5LoginModule can be configured. > Gremlin-server already includes the pluggable Sasl framework that can host > the proposed Kerberos authentication plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server
[ https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15746237#comment-15746237 ] Marc de Lignie commented on TINKERPOP-1566: --- @[~spmallette] thanks for the feedback. My branch at https://github.com/vtslab/incubator-tinkerpop/tree/TINKERPOP-1566 is a number of commits further now and loose ends are becoming fewer. I initiated the process for getting permission from my organization regarding the copyright transfer, so I am working towards a PR. The gremlin-python maintainers might take a look at https://github.com/vtslab/incubator-tinkerpop/commit/1087746998531738f5212d76a70f950b37dbc704 and see whether they agree with the way they are mentioned. Even better would be to take up the challlenge and integrate python's https://pypi.python.org/pypi/pure-sasl/ client into gremlin-python. Pure-sasl supports GSSAPI (Kerberos) out-of-the-box, it says. > Kerberos authentication for gremlin-server > -- > > Key: TINKERPOP-1566 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1566 > Project: TinkerPop > Issue Type: Improvement > Components: server >Reporter: Marc de Lignie >Priority: Minor > Labels: security > Fix For: 3.3.0 > > > Gremlin server would benefit from an explicit Kerberos authentication plugin, > because preparing and maintaining such a plugin is nontrivial. Also, many > other Apache project provide kerberized services. > In gremlin-console the standard Krb5LoginModule can be configured. > Gremlin-server already includes the pluggable Sasl framework that can host > the proposed Kerberos authentication plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server
[ https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15762143#comment-15762143 ] Marc de Lignie commented on TINKERPOP-1566: --- You are right, consider the org.assertj test dependency removed. > Kerberos authentication for gremlin-server > -- > > Key: TINKERPOP-1566 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1566 > Project: TinkerPop > Issue Type: Improvement > Components: server >Reporter: Marc de Lignie >Priority: Minor > Labels: security > Fix For: 3.3.0 > > > Gremlin server would benefit from an explicit Kerberos authentication plugin, > because preparing and maintaining such a plugin is nontrivial. Also, many > other Apache project provide kerberized services. > In gremlin-console the standard Krb5LoginModule can be configured. > Gremlin-server already includes the pluggable Sasl framework that can host > the proposed Kerberos authentication plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server
[ https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15824460#comment-15824460 ] Marc de Lignie commented on TINKERPOP-1566: --- I am in the rebase and final testing phase. Hope it won't take too long anymore. > Kerberos authentication for gremlin-server > -- > > Key: TINKERPOP-1566 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1566 > Project: TinkerPop > Issue Type: Improvement > Components: server >Reporter: Marc de Lignie >Priority: Minor > Labels: security > Fix For: 3.3.0 > > > Gremlin server would benefit from an explicit Kerberos authentication plugin, > because preparing and maintaining such a plugin is nontrivial. Also, many > other Apache project provide kerberized services. > In gremlin-console the standard Krb5LoginModule can be configured. > Gremlin-server already includes the pluggable Sasl framework that can host > the proposed Kerberos authentication plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TINKERPOP-1641) Authentication for gremlin-python
Marc de Lignie created TINKERPOP-1641: - Summary: Authentication for gremlin-python Key: TINKERPOP-1641 URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 Project: TinkerPop Issue Type: Improvement Components: driver, server Affects Versions: 3.3.0 Reporter: Marc de Lignie Priority: Minor Follow-up on [TINKERPOP-1566|https://issues.apache.org/jira/browse/TINKERPOP-1566]: * make gremlin-python authenticate against SimpleAuthenticator and Krb5Authenticator * add audit logging for bytecode requests (in addition to existing audit logging of scrip requests and of the authenticated user by TINKERPOP-1566) * tests and documentation for this -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TINKERPOP-1641) Authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886063#comment-15886063 ] Marc de Lignie commented on TINKERPOP-1641: --- To get debug info from gremlin server during gremlin-python authentication development: * run standalone bin/gremlin-server.sh 3.3.0-SNAPSHOT (branch master) with: ** new conf/gremlin-server-testpython.yaml (port 45942, SimpleAuthenticator or Krb5Authenticator) ** DEBUG in conf/log4j.properties file * adapt conftest.py to use port 45942 in PR554 * mvn clean install -DskipTests * mvn install -pl gremlin-python -Pglv-python > Authentication for gremlin-python > - > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: driver, server >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Follow-up on > [TINKERPOP-1566|https://issues.apache.org/jira/browse/TINKERPOP-1566]: > * make gremlin-python authenticate against SimpleAuthenticator and > Krb5Authenticator > * add audit logging for bytecode requests (in addition to existing audit > logging of scrip requests and of the authenticated user by TINKERPOP-1566) > * tests and documentation for this -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TINKERPOP-1641) Authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892014#comment-15892014 ] Marc de Lignie commented on TINKERPOP-1641: --- @[~davebshow] Got SimpleAuthenticator working in gremlin-python tests, see: [https://github.com/vtslab/incubator-tinkerpop/commit/7e7490b6ad19d915872a3a1b46e269d76c69fd6c] Will move on to kerberos authentication for gremlin-python. Cheers, Marc > Authentication for gremlin-python > - > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: driver, server >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Follow-up on > [TINKERPOP-1566|https://issues.apache.org/jira/browse/TINKERPOP-1566]: > * make gremlin-python authenticate against SimpleAuthenticator and > Krb5Authenticator > * add audit logging for bytecode requests (in addition to existing audit > logging of scrip requests and of the authenticated user by TINKERPOP-1566) > * tests and documentation for this -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TINKERPOP-1641) Authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016308#comment-16016308 ] Marc de Lignie commented on TINKERPOP-1641: --- Not much new at this moment. Had problems using the MIT Kerberos client (wrapped by python) with the Apache Directory-Kerby kdc as test fixture. This resulted in the following issue: https://issues.apache.org/jira/browse/DIRKRB-631 Until this gets fixed I plan to do some work in getting gremlin-python authenticated using FreeIPA as a kdc. > Authentication for gremlin-python > - > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: driver, server >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Follow-up on > [TINKERPOP-1566|https://issues.apache.org/jira/browse/TINKERPOP-1566]: > * make gremlin-python authenticate against SimpleAuthenticator and > Krb5Authenticator > * add audit logging for bytecode requests (in addition to existing audit > logging of scrip requests and of the authenticated user by TINKERPOP-1566) > * tests and documentation for this -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TINKERPOP-1641) Authentication for gremlin-python
[ https://issues.apache.org/jira/browse/TINKERPOP-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016326#comment-16016326 ] Marc de Lignie commented on TINKERPOP-1641: --- Thanks for the warning, I do not mean to use freeIPA as a dependency, only as a temporary solution until Apache Directory-Kerby gets fixed for use with recent MIT-Kerberos distributions. > Authentication for gremlin-python > - > > Key: TINKERPOP-1641 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1641 > Project: TinkerPop > Issue Type: Improvement > Components: driver, server >Affects Versions: 3.3.0 >Reporter: Marc de Lignie >Priority: Minor > > Follow-up on > [TINKERPOP-1566|https://issues.apache.org/jira/browse/TINKERPOP-1566]: > * make gremlin-python authenticate against SimpleAuthenticator and > Krb5Authenticator > * add audit logging for bytecode requests (in addition to existing audit > logging of scrip requests and of the authenticated user by TINKERPOP-1566) > * tests and documentation for this -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TINKERPOP-1757) GiraphGraphComputer does not start Zookeeper in local mode
Marc de Lignie created TINKERPOP-1757: - Summary: GiraphGraphComputer does not start Zookeeper in local mode Key: TINKERPOP-1757 URL: https://issues.apache.org/jira/browse/TINKERPOP-1757 Project: TinkerPop Issue Type: Bug Components: hadoop Affects Versions: 3.2.6, 3.3.0 Environment: gremlin-console in Ubuntu with Oracle java 8 Reporter: Marc de Lignie Priority: Trivial Fix For: 3.2.7, 3.3.1 As discussed in [https://groups.google.com/forum/#!topic/gremlin-users/Ztbr_uJhoxY] running GiraphGraphComputer in local mode (without an external or pseudo hadoop cluster) requires the following configured properties (add section local to the hadoop properties file): {code:java} gremlin.hadoop.inputLocation=data/tinkerpop-modern.kryo giraph.minWorkers=1 giraph.maxWorkers=1 giraph.SplitMasterWorker=false mapreduce.job.maps=4 mapreduce.job.reduces=2 {code} The real problem is that GiraphGraphComputer does not manage to start its Zookeeper service because it gets relative classpath items from somewhere (gremlin-console provides the acual shell command used to attempt starting Zookeeper in its logging). Current workaround is to start gremlin-console as follows: {code:bash} export CLASSPATH=$PWD/lib/* bin/gremlin.sh {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (TINKERPOP-1757) GiraphGraphComputer does not start Zookeeper in local mode
[ https://issues.apache.org/jira/browse/TINKERPOP-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc de Lignie updated TINKERPOP-1757: -- Description: As discussed in [https://groups.google.com/forum/#!topic/gremlin-users/Ztbr_uJhoxY] running GiraphGraphComputer in local mode (without an external or pseudo hadoop cluster) requires the following configured properties (add section local to the hadoop properties file): {code:java} gremlin.hadoop.inputLocation=data/tinkerpop-modern.kryo giraph.minWorkers=1 giraph.maxWorkers=1 giraph.SplitMasterWorker=false mapreduce.job.maps=4 mapreduce.job.reduces=2 {code} The real problem is that GiraphGraphComputer does not manage to start its Zookeeper service because it gets relative classpath items from somewhere (gremlin-console provides the acual shell command used to attempt starting Zookeeper in its logging). Current workaround is to start gremlin-console as follows: {code:java} export CLASSPATH=$PWD/lib/* bin/gremlin.sh {code} was: As discussed in [https://groups.google.com/forum/#!topic/gremlin-users/Ztbr_uJhoxY] running GiraphGraphComputer in local mode (without an external or pseudo hadoop cluster) requires the following configured properties (add section local to the hadoop properties file): {code:java} gremlin.hadoop.inputLocation=data/tinkerpop-modern.kryo giraph.minWorkers=1 giraph.maxWorkers=1 giraph.SplitMasterWorker=false mapreduce.job.maps=4 mapreduce.job.reduces=2 {code} The real problem is that GiraphGraphComputer does not manage to start its Zookeeper service because it gets relative classpath items from somewhere (gremlin-console provides the acual shell command used to attempt starting Zookeeper in its logging). Current workaround is to start gremlin-console as follows: {code:bash} export CLASSPATH=$PWD/lib/* bin/gremlin.sh {code} > GiraphGraphComputer does not start Zookeeper in local mode > -- > > Key: TINKERPOP-1757 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1757 > Project: TinkerPop > Issue Type: Bug > Components: hadoop >Affects Versions: 3.3.0, 3.2.6 > Environment: gremlin-console in Ubuntu with Oracle java 8 >Reporter: Marc de Lignie >Priority: Trivial > Fix For: 3.2.7, 3.3.1 > > > As discussed in > [https://groups.google.com/forum/#!topic/gremlin-users/Ztbr_uJhoxY] running > GiraphGraphComputer in local mode (without an external or pseudo hadoop > cluster) requires the following configured properties (add section local to > the hadoop properties file): > {code:java} > gremlin.hadoop.inputLocation=data/tinkerpop-modern.kryo > giraph.minWorkers=1 > giraph.maxWorkers=1 > giraph.SplitMasterWorker=false > mapreduce.job.maps=4 > mapreduce.job.reduces=2 > {code} > The real problem is that GiraphGraphComputer does not manage to start its > Zookeeper service because it gets relative classpath items from somewhere > (gremlin-console provides the acual shell command used to attempt starting > Zookeeper in its logging). > Current workaround is to start gremlin-console as follows: > {code:java} > export CLASSPATH=$PWD/lib/* > bin/gremlin.sh > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
Marc de Lignie created TINKERPOP-1786: - Summary: Recipe and missing manifest items for Spark on Yarn Key: TINKERPOP-1786 URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 Project: TinkerPop Issue Type: Improvement Components: hadoop Affects Versions: 3.2.6, 3.3.0 Environment: gremlin-console Reporter: Marc de Lignie Priority: Minor Fix For: 3.2.7, 3.3.1 Thorough documentation for running OLAP queries on Spark on Yarn has been missing, keeping some users from getting the benefits of this nice feature of the Tinkerpop stack and resulting in a significant number of questions on the gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc de Lignie updated TINKERPOP-1786: -- Affects Version/s: 3.1.8 > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169628#comment-16169628 ] Marc de Lignie commented on TINKERPOP-1786: --- This issue was discussed on the [dev list|http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201707.mbox/browser] (July 6) and the approach suggested by Stephen Malette turns out to be perfectly possible. A future PR is in pretty good shape and can be previewed [here|https://github.com/vtslab/incubator-tinkerpop/commit/d2a7d9e85b76b2ccfda1c15d0c18def331fb75ea]. What I plan to do next: * merge this commit into a branch based on master (this will break things because of the differences between Spark 1.6 and Spark 2.0) * final test of the TP32 and TP33 versions on a real cluster * issue a PR with TP32 and TP33 feature branches and receive review comments Of course, any suggestions on the approach are also welcome at this stage. > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169628#comment-16169628 ] Marc de Lignie edited comment on TINKERPOP-1786 at 9/18/17 5:49 AM: This issue was discussed on the [dev list|http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201707.mbox/browser] (July 6) and the approach suggested by Stephen Mallette turns out to be perfectly possible. A future PR is in pretty good shape and can be previewed [here|https://github.com/vtslab/incubator-tinkerpop/commit/d2a7d9e85b76b2ccfda1c15d0c18def331fb75ea]. What I plan to do next: * merge this commit into a branch based on master (this will break things because of the differences between Spark 1.6 and Spark 2.0) * final test of the TP32 and TP33 versions on a real cluster * issue a PR with TP32 and TP33 feature branches and receive review comments Of course, any suggestions on the approach are also welcome at this stage. was (Author: hadoopmarc): This issue was discussed on the [dev list|http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201707.mbox/browser] (July 6) and the approach suggested by Stephen Malette turns out to be perfectly possible. A future PR is in pretty good shape and can be previewed [here|https://github.com/vtslab/incubator-tinkerpop/commit/d2a7d9e85b76b2ccfda1c15d0c18def331fb75ea]. What I plan to do next: * merge this commit into a branch based on master (this will break things because of the differences between Spark 1.6 and Spark 2.0) * final test of the TP32 and TP33 versions on a real cluster * issue a PR with TP32 and TP33 feature branches and receive review comments Of course, any suggestions on the approach are also welcome at this stage. > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)