[jira] [Updated] (TINKERPOP-2540) Unexpected count() step result

[Guobao Chen (Jira)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guobao Chen updated TINKERPOP-2540:
---
Description: 
After installing latest (tinkerpop version 3.4.10), it's found that running a 
specific gremlin script with 'count()' step gives unexpected result.

*Steps to reproduce*

1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
client

2 create a modern graph with TinkerFactory
{code:java}
gremlin> graph = TinkerFactory.createModern()
 ==>tinkergraph[vertices:6 edges:6]
gremlin> g=graph.traversal()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> g
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]{code}
3 run following script:
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
{code}
Note that usser 'testuser1' doesn't exist in the created 'modern' graph.

*Expected result*

a numeric number showing the count (or 0 if there is no vertices from previous 
steps before count() step)

*Actual result*
 See a list of vertices:
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
 ==>v[1]
 ==>v[2]
 ==>v[3]
 ==>v[4]
 ==>v[5]
 ==>v[6]{code}

  was:
After installing latest (tinkerpop version 3.4.10), it's found that running a 
specific gremlin script with 'count()' step gives unexpected result.

*Steps to reproduce*

1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
client

2 create a modern graph with TinkerFactory
{code:java}
gremlin> graph = TinkerFactory.createModern()
 ==>tinkergraph[vertices:6 edges:6]
gremlin> g=graph.traversal()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> g
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]{code}
3 query with following script in the result
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
{code}
Note that usser with name 'testuser1' doesn't exist in the created 'modern' 
graph.

*Expected result*

a numeric number showing the count (or 0 if there is no vertices from previous 
steps before count() step)

*Actual result*
 See a list of vertices:
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
 ==>v[1]
 ==>v[2]
 ==>v[3]
 ==>v[4]
 ==>v[5]
 ==>v[6]{code}


> Unexpected count() step result
> --
>
> Key: TINKERPOP-2540
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2540
> Project: TinkerPop
>  Issue Type: Bug
>  Components: process
>Affects Versions: 3.4.10
> Environment: OS: Ubuntu 20.04.2 LTS
> Java: Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
>Reporter: Guobao Chen
>Priority: Blocker
>
> After installing latest (tinkerpop version 3.4.10), it's found that running a 
> specific gremlin script with 'count()' step gives unexpected result.
> *Steps to reproduce*
> 1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
> client
> 2 create a modern graph with TinkerFactory
> {code:java}
> gremlin> graph = TinkerFactory.createModern()
>  ==>tinkergraph[vertices:6 edges:6]
> gremlin> g=graph.traversal()
> ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
> gremlin> g
> ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]{code}
> 3 run following script:
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
> {code}
> Note that usser 'testuser1' doesn't exist in the created 'modern' graph.
> *Expected result*
> a numeric number showing the count (or 0 if there is no vertices from 
> previous steps before count() step)
> *Actual result*
>  See a list of vertices:
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
>  ==>v[1]
>  ==>v[2]
>  ==>v[3]
>  ==>v[4]
>  ==>v[5]
>  ==>v[6]{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (TINKERPOP-2540) Unexpected count() step result

[Guobao Chen (Jira)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guobao Chen updated TINKERPOP-2540:
---
Description: 
After installing latest (tinkerpop version 3.4.10), it's found that running a 
specific gremlin script with 'count()' step gives unexpected result.

*Steps to reproduce*

1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
client

2 create a modern graph with TinkerFactory
{code:java}
gremlin> graph = TinkerFactory.createModern()
 ==>tinkergraph[vertices:6 edges:6]
gremlin> g=graph.traversal()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> g
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]{code}
3 query with following script in the result
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
{code}
Note that usser with name 'testuser1' doesn't exist in the created 'modern' 
graph.

*Expected result*

a numeric number showing the count (or 0 if there is no vertices from previous 
steps before count() step)

*Actual result*
 See a list of vertices:
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
 ==>v[1]
 ==>v[2]
 ==>v[3]
 ==>v[4]
 ==>v[5]
 ==>v[6]{code}

  was:
After installing latest (tinkerpop version 3.4.10), it's found that running a 
specific gremlin script with 'count()' step gives unexpected result.

*Steps to reproduce*

1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
client

2 create a modern graph with TinkerFactory
{code:java}
gremlin> graph = TinkerFactory.createModern()
 ==>tinkergraph[vertices:6 edges:6]{code}
3 query with following script in the result
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
{code}
Note that usser with name 'testuser1' doesn't exist in the created 'modern' 
graph.

*Expected result*

a numeric number showing the count (or 0 if there is no vertices from previous 
steps before count() step)

*Actual result*
 See a list of vertices:
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
 ==>v[1]
 ==>v[2]
 ==>v[3]
 ==>v[4]
 ==>v[5]
 ==>v[6]{code}


> Unexpected count() step result
> --
>
> Key: TINKERPOP-2540
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2540
> Project: TinkerPop
>  Issue Type: Bug
>  Components: process
>Affects Versions: 3.4.10
> Environment: OS: Ubuntu 20.04.2 LTS
> Java: Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
>Reporter: Guobao Chen
>Priority: Blocker
>
> After installing latest (tinkerpop version 3.4.10), it's found that running a 
> specific gremlin script with 'count()' step gives unexpected result.
> *Steps to reproduce*
> 1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
> client
> 2 create a modern graph with TinkerFactory
> {code:java}
> gremlin> graph = TinkerFactory.createModern()
>  ==>tinkergraph[vertices:6 edges:6]
> gremlin> g=graph.traversal()
> ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
> gremlin> g
> ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]{code}
> 3 query with following script in the result
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
> {code}
> Note that usser with name 'testuser1' doesn't exist in the created 'modern' 
> graph.
> *Expected result*
> a numeric number showing the count (or 0 if there is no vertices from 
> previous steps before count() step)
> *Actual result*
>  See a list of vertices:
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
>  ==>v[1]
>  ==>v[2]
>  ==>v[3]
>  ==>v[4]
>  ==>v[5]
>  ==>v[6]{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (TINKERPOP-2540) Unexpected count() step result

[Guobao Chen (Jira)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guobao Chen updated TINKERPOP-2540:
---
Priority: Blocker  (was: Major)

> Unexpected count() step result
> --
>
> Key: TINKERPOP-2540
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2540
> Project: TinkerPop
>  Issue Type: Bug
>  Components: process
>Affects Versions: 3.4.10
> Environment: OS: Ubuntu 20.04.2 LTS
> Java: Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
>Reporter: Guobao Chen
>Priority: Blocker
>
> After installing latest (tinkerpop version 3.4.10), it's found that running a 
> specific gremlin script with 'count()' step gives unexpected result.
> *Steps to reproduce*
> 1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
> client
> 2 create a modern graph with TinkerFactory
> {code:java}
> gremlin> graph = TinkerFactory.createModern()
>  ==>tinkergraph[vertices:6 edges:6]{code}
> 3 query with following script in the result
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
> {code}
> Note that usser with name 'testuser1' doesn't exist in the created 'modern' 
> graph.
> *Expected result*
> a numeric number showing the count (or 0 if there is no vertices from 
> previous steps before count() step)
> *Actual result*
>  See a list of vertices:
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
>  ==>v[1]
>  ==>v[2]
>  ==>v[3]
>  ==>v[4]
>  ==>v[5]
>  ==>v[6]{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (TINKERPOP-2540) Unexpected count() step result

[Guobao Chen (Jira)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306794#comment-17306794
 ] 

Guobao Chen commented on TINKERPOP-2540:


Seems this issue is related to or step and count step.

Following script returns expected result (without or):
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').has('name','testuser1').values('name','age').dedup().count()
==>0{code}
Also following script returns expected result (with or step but without count 
step):
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup()
gremlin> {code}

> Unexpected count() step result
> --
>
> Key: TINKERPOP-2540
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2540
> Project: TinkerPop
>  Issue Type: Bug
>  Components: process
>Affects Versions: 3.4.10
> Environment: OS: Ubuntu 20.04.2 LTS
> Java: Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
>Reporter: Guobao Chen
>Priority: Major
>
> After installing latest (tinkerpop version 3.4.10), it's found that running a 
> specific gremlin script with 'count()' step gives unexpected result.
> *Steps to reproduce*
> 1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
> client
> 2 create a modern graph with TinkerFactory
> {code:java}
> gremlin> graph = TinkerFactory.createModern()
>  ==>tinkergraph[vertices:6 edges:6]{code}
> 3 query with following script in the result
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
> {code}
> Note that usser with name 'testuser1' doesn't exist in the created 'modern' 
> graph.
> *Expected result*
> a numeric number showing the count (or 0 if there is no vertices from 
> previous steps before count() step)
> *Actual result*
>  See a list of vertices:
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
>  ==>v[1]
>  ==>v[2]
>  ==>v[3]
>  ==>v[4]
>  ==>v[5]
>  ==>v[6]{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (TINKERPOP-2540) Unexpected count() step result

[Guobao Chen (Jira)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guobao Chen updated TINKERPOP-2540:
---
Description: 
After installing latest (tinkerpop version 3.4.10), it's found that running a 
specific gremlin script with 'count()' step gives unexpected result.

*Steps to reproduce*

1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
client

2 create a modern graph with TinkerFactory
{code:java}
gremlin> graph = TinkerFactory.createModern()
 ==>tinkergraph[vertices:6 edges:6]{code}
3 query with following script in the result
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
{code}
Note that usser with name 'testuser1' doesn't exist in the created 'modern' 
graph.

*Expected result*

a numeric number showing the count (or 0 if there is no vertices from previous 
steps before count() step)

*Actual result*
 See a list of vertices:
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
 ==>v[1]
 ==>v[2]
 ==>v[3]
 ==>v[4]
 ==>v[5]
 ==>v[6]{code}

  was:
After installing latest (tinkerpop version 3.4.10), it's found that running a 
specific gremlin script with 'count()' step gives unexpected result.

*Steps to reproduce*

1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
client

2 create a modern graph with TinkerFactory
{code:java}
gremlin> graph = TinkerFactory.createModern()
 ==>tinkergraph[vertices:6 edges:6]{code}
3 query with following script in the result
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
{code}
*Expected result*

a numeric number showing the count (or 0 if there is no vertices from previous 
steps before count() step)

*Actual result*
 See a list of vertices:
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
 ==>v[1]
 ==>v[2]
 ==>v[3]
 ==>v[4]
 ==>v[5]
 ==>v[6]{code}


> Unexpected count() step result
> --
>
> Key: TINKERPOP-2540
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2540
> Project: TinkerPop
>  Issue Type: Bug
>  Components: process
>Affects Versions: 3.4.10
> Environment: OS: Ubuntu 20.04.2 LTS
> Java: Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
>Reporter: Guobao Chen
>Priority: Major
>
> After installing latest (tinkerpop version 3.4.10), it's found that running a 
> specific gremlin script with 'count()' step gives unexpected result.
> *Steps to reproduce*
> 1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
> client
> 2 create a modern graph with TinkerFactory
> {code:java}
> gremlin> graph = TinkerFactory.createModern()
>  ==>tinkergraph[vertices:6 edges:6]{code}
> 3 query with following script in the result
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
> {code}
> Note that usser with name 'testuser1' doesn't exist in the created 'modern' 
> graph.
> *Expected result*
> a numeric number showing the count (or 0 if there is no vertices from 
> previous steps before count() step)
> *Actual result*
>  See a list of vertices:
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
>  ==>v[1]
>  ==>v[2]
>  ==>v[3]
>  ==>v[4]
>  ==>v[5]
>  ==>v[6]{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (TINKERPOP-2540) Unexpected count() step result

[Guobao Chen (Jira)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guobao Chen updated TINKERPOP-2540:
---
Environment: 
OS: Ubuntu 20.04.2 LTS
Java: Java(TM) SE Runtime Environment (build 1.8.0_241-b07)

> Unexpected count() step result
> --
>
> Key: TINKERPOP-2540
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2540
> Project: TinkerPop
>  Issue Type: Bug
>  Components: process
>Affects Versions: 3.4.10
> Environment: OS: Ubuntu 20.04.2 LTS
> Java: Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
>Reporter: Guobao Chen
>Priority: Major
>
> After installing latest (tinkerpop version 3.4.10), it's found that running a 
> specific gremlin script with 'count()' step gives unexpected result.
> *Steps to reproduce*
> 1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
> client
> 2 create a modern graph with TinkerFactory
> {code:java}
> gremlin> graph = TinkerFactory.createModern()
>  ==>tinkergraph[vertices:6 edges:6]{code}
> 3 query with following script in the result
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
> {code}
> *Expected result*
> a numeric number showing the count (or 0 if there is no vertices from 
> previous steps before count() step)
> *Actual result*
>  See a list of vertices:
> {code:java}
> gremlin> 
> g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
>  ==>v[1]
>  ==>v[2]
>  ==>v[3]
>  ==>v[4]
>  ==>v[5]
>  ==>v[6]{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (TINKERPOP-2540) Unexpected count() step result

[Guobao Chen (Jira)]

Guobao Chen created TINKERPOP-2540:
--

 Summary: Unexpected count() step result
 Key: TINKERPOP-2540
 URL: https://issues.apache.org/jira/browse/TINKERPOP-2540
 Project: TinkerPop
  Issue Type: Bug
  Components: process
Affects Versions: 3.4.10
Reporter: Guobao Chen


After installing latest (tinkerpop version 3.4.10), it's found that running a 
specific gremlin script with 'count()' step gives unexpected result.

*Steps to reproduce*

1 Install tinkerpop 3.4.10, and connect to this server with gremlin console 
client

2 create a modern graph with TinkerFactory
{code:java}
gremlin> graph = TinkerFactory.createModern()
 ==>tinkergraph[vertices:6 edges:6]{code}
3 query with following script in the result
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
{code}
*Expected result*

a numeric number showing the count (or 0 if there is no vertices from previous 
steps before count() step)

*Actual result*
 See a list of vertices:
{code:java}
gremlin> 
g.V().hasLabel('person').has('name','testuser1').or().has('name','testuser1').values('name','age').dedup().count()
 ==>v[1]
 ==>v[2]
 ==>v[3]
 ==>v[4]
 ==>v[5]
 ==>v[6]{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Converging the worker pool was: [DISCUSS] Remote Transactions

[Stephen Mallette]

I've started to play with some ideas for pushing all requests
scripts/bytecode sessions/sessionless down one unified pipeline with one
shared thread pool. The most interesting thing I've seen so far is how much
the code could potentially simplify under this approach. The
GremlinExecutor was the wrong abstraction in a sense and it forced way too
many callbacks as arguments and made error handling kinda sloppy which made
the server code harder to reuse, extend and read. Of course, all that was
built before the first remote graph was built and we were just wildly
guessing at how DS Graph was going to work with all this. Now, I think the
remote model is much more clear, so maybe all this can be made "right" now.

I think that if I can see any success at all with this in the next few
days, we could include a new UnfiiedChannelizer that would replace the
existing ones. I think we'd keep the old one in place by default and keep
my copy/paste work for SessionOpProcessor that added bytecode there, with
the idea that the UnifiedChannelizer will become the future default as we
get it tested further along the 3.5.x line.

I imagine I'll have more details on this task tomorrow and will post back
here when I do.


On Thu, Mar 18, 2021 at 12:37 PM Divij Vaidya 
wrote:

> >
> > I suppose the
> > question is how would we ensure that each request for a session ends up
> > being executed on the same thread from the previous request if jobs are
> > randomly submitted to a worker pool?
>
>
> I haven't thought through the details, but on top of my head, we would have
> to maintain some request<->thread mapping on the server. This mapping is
> also a building block for a request cancellation feature in future where a
> client would be able to send a cancellation request to the server, the
> server will map the request to a thread executing that request and then set
> an interrupt on that thread to signify cancellation.
>
> Divij Vaidya
>
>
>
> On Thu, Mar 18, 2021 at 5:00 PM Stephen Mallette 
> wrote:
>
> > I started a fresh thread for this topic Divij brought up, with more
> > context:
> >
> > > In a scenario where we have both
> > > session-less and sesion-ful requests being made to the server, the
> > resource
> > > allocation will not be fair and may lead to starvation for some
> requests.
> > > This problem exists even today, hence not totally correlated to what
> you
> > > are proposing but I am afraid a wider adoption of explicit
> > > transactions will bring this problem to the spotlight. Hence, we should
> > fix
> > > this problem first. A solution would entail converging the worker pool
> > for
> > > both session vs session-less requests.
> >
> > I'm not sure we can get that done in 3.5.0, but maybe? I suppose the
> > question is how would we ensure that each request for a session ends up
> > being executed on the same thread from the previous request if jobs are
> > randomly submitted to a worker pool?
> >
> >
> > On Thu, Mar 18, 2021 at 11:52 AM Divij Vaidya 
> > wrote:
> >
> > > Great idea Stephen. I agree with standardizing the explicit transaction
> > > semantics in cases of remote server (implicit aka sessionless is
> already
> > > self explanatory) and unifying the syntax with embedded graph usage.
> > >
> > > Couple of notes:
> > >
> > > 1. I would favor the begin() instead of create() as it's closer to
> > > universally prominent SQL transaction syntax. This would reduce the
> > > learning curve for users of TP.
> > >
> > > 2. From an implementation standpoint, sessionless requests are queued
> on
> > > the server side and are serviced by the worker thread pool. But a
> > > session-ful aka explicit transaction aka managed transaction starts a
> new
> > > single worked thread pool every time. In a scenario where we have both
> > > session-less and sesion-ful requests being made to the server, the
> > resource
> > > allocation will not be fair and may lead to starvation for some
> requests.
> > > This problem exists even today, hence not totally correlated to what
> you
> > > are proposing but I am afraid a wider adoption of explicit
> > > transactions will bring this problem to the spotlight. Hence, we should
> > fix
> > > this problem first. A solution would entail converging the worker pool
> > for
> > > both session vs session-less requests.
> > >
> > > 3. You are proposing the idea of having a transaction scoped traversal
> > > object. I agree with it but we need more clarification in behavior wrt
> > the
> > > following scenarios:
> > >
> > > Q. What happens when g.tx().commit() is called on a transaction scoped
> > > traversal object? Does the traversal get closed?
> > > Q. Currently, the same traversal object could be used to execute
> multiple
> > > session-less requests simultaneously in a thread safe manner. Does the
> > same
> > > behaviour apply to transaction scoped traversal? If not, then how do I
> > send
> > > multiple requests in parallel from the client all scoped to the same
> > > transaction

Re: [DISCUSS] ANTLR and gremlin-script

[Joshua Shinavier]

Btw. to your other questions:

I think Stephen's attitude toward TP4 is that there should be a smooth
transition, so ideally this would be for TP3, migrating to TP4 when the
time comes.

Native Java traversals are not accessible to a parser, so I do not see how
they would be impacted by gremlin-grammar or gremlin-language.

Josh



On Mon, Mar 22, 2021 at 2:00 PM Joshua Shinavier  wrote:

> I am not yet entirely sure what it means either, but I am thinking that it
> would be nice to be able to
> a) validate Gremlin expressions written in languages other than Groovy or
> Java, and
> b) parse Gremlin expressions in Gremlin language variants, producing
> JVM-based traversals, or even
> c) parse Gremlin expressions in Gremlin language variants, producing
> native traversals in the host language
> These options range from easy (a) to hard (c).
>
> Another way to look at this is: let's use Stephen's grammar as a template
> for a more generic grammar which is more flexible w.r.t. the input language.
>
> Josh
>
>
>
> On Mon, Mar 22, 2021 at 11:37 AM pieter gmail 
> wrote:
>
>> Hi,
>>
>> Exciting as this is I am not quite sure what it means.
>>
>> Naively  perhaps it the idea,
>> Arbitary gremlin string -> antlr parser -> some AST walker -> gremlin
>> byte code -> java in memory steps ... -> voila
>>
>> Is the grammar going to be the primary and only
>> interface/specification, or will the native java implementation bypass
>> the grammar going straight to the steps instead?
>>
>> Is this aimed at the gremlin 3 or 4?
>>
>> Cheers
>> Pieter
>>
>> On Tue, 2021-03-16 at 15:47 -0400, Stephen Mallette wrote:
>> > Here is the PR: https://github.com/apache/tinkerpop/pull/1408
>> >
>> > On Tue, Mar 16, 2021 at 6:14 AM Stephen Mallette
>> > 
>> > wrote:
>> >
>> > > No branch yet, but I think I will be sending the PR today.
>> > >
>> > > On Mon, Mar 15, 2021 at 9:33 PM Joshua Shinavier
>> > > 
>> > > wrote:
>> > >
>> > > > Is there a branch we can take a look at before the PR is ready?
>> > > >
>> > > > Josh
>> > > >
>> > > > On Fri, Mar 12, 2021 at 5:42 AM Stephen Mallette
>> > > > 
>> > > > wrote:
>> > > >
>> > > > > I've been working on forming a pull request for this task. I
>> > > > > don't
>> > > > think IP
>> > > > > Clearance is necessary as I originally did because the
>> > > > > contribution is
>> > > > > really just an ANTLR4 grammar file with some tests to validate
>> > > > > things.
>> > > > > Therefore, it's not a big body of independent code as I'd
>> > > > > perhaps
>> > > > initially
>> > > > > envisioned. Compared to gremlint, this addition is pretty
>> > > > > simple and
>> > > > > straightforward. I've created this issue in JIRA with some
>> > > > > additional
>> > > > notes
>> > > > > on what to expect in this initial body of work:
>> > > > >
>> > > > > https://issues.apache.org/jira/browse/TINKERPOP-2533
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Mon, Feb 8, 2021 at 10:06 AM Stephen Mallette
>> > > > > 
>> > > > > wrote:
>> > > > >
>> > > > > > Just wanted to leave an update on this thread. It was nice to
>> > > > > > see some
>> > > > > > support for it. I've not had time to focus on the task itself
>> > > > > > so sorry
>> > > > > > there hasn't been much movement, but I hope to see it on
>> > > > > > track soon. I
>> > > > > > thought to update the thread after I came across yet another
>> > > > > > nice
>> > > > usage
>> > > > > for
>> > > > > > it. I've long wanted to unify our test framework (i.e.
>> > > > > > deprecate the
>> > > > JVM
>> > > > > > process suite in favor of the GLV test suite). I was
>> > > > > > experimenting
>> > > > with
>> > > > > > what that might look like on Friday and hit a circular
>> > > > > > dependency
>> > > > which
>> > > > > > constantly trips things up where gremlin-test wants to depend
>> > > > > > on
>> > > > > > gremlin-groovy (for ScriptEngine support) but gremlin-groovy
>> > > > > > depends
>> > > > on
>> > > > > > gremlin-test and tinkergraph with  scope already. I
>> > > > > > think the
>> > > > > > introduction of gremlin-script would let gremlin-test build
>> > > > > > the
>> > > > Traversal
>> > > > > > object from a Gremlin string and thus avoid that circular
>> > > > relationship.
>> > > > > >
>> > > > > > On Fri, Jan 8, 2021 at 2:43 AM pieter gmail
>> > > > > > 
>> > > > > > wrote:
>> > > > > >
>> > > > > > > +1
>> > > > > > >
>> > > > > > > I have often thought the language specification should be a
>> > > > > > > project
>> > > > > > > separate from the implementations, and done in a formal but
>> > > > > > > plain
>> > > > > > > English format similar to OMG or IETF specifications.
>> > > > > > >
>> > > > > > > I suspect Sqlg's code base would have been fastly different
>> > > > > > > if it had
>> > > > > > > evolved from a grammer instead of an api.
>> > > > > > >
>> > > > > > > Cheers
>> > > > > > > Pieter
>> > > > > > >
>> > > > > > > On Thu, 2020-12-24 at 14:41 -0500, Stephen Mallette wrote:
>> > > > > > > > As a project, over the years, we'

Re: [DISCUSS] ANTLR and gremlin-script

[Joshua Shinavier]

I am not yet entirely sure what it means either, but I am thinking that it
would be nice to be able to
a) validate Gremlin expressions written in languages other than Groovy or
Java, and
b) parse Gremlin expressions in Gremlin language variants, producing
JVM-based traversals, or even
c) parse Gremlin expressions in Gremlin language variants, producing native
traversals in the host language
These options range from easy (a) to hard (c).

Another way to look at this is: let's use Stephen's grammar as a template
for a more generic grammar which is more flexible w.r.t. the input language.

Josh



On Mon, Mar 22, 2021 at 11:37 AM pieter gmail 
wrote:

> Hi,
>
> Exciting as this is I am not quite sure what it means.
>
> Naively  perhaps it the idea,
> Arbitary gremlin string -> antlr parser -> some AST walker -> gremlin
> byte code -> java in memory steps ... -> voila
>
> Is the grammar going to be the primary and only
> interface/specification, or will the native java implementation bypass
> the grammar going straight to the steps instead?
>
> Is this aimed at the gremlin 3 or 4?
>
> Cheers
> Pieter
>
> On Tue, 2021-03-16 at 15:47 -0400, Stephen Mallette wrote:
> > Here is the PR: https://github.com/apache/tinkerpop/pull/1408
> >
> > On Tue, Mar 16, 2021 at 6:14 AM Stephen Mallette
> > 
> > wrote:
> >
> > > No branch yet, but I think I will be sending the PR today.
> > >
> > > On Mon, Mar 15, 2021 at 9:33 PM Joshua Shinavier
> > > 
> > > wrote:
> > >
> > > > Is there a branch we can take a look at before the PR is ready?
> > > >
> > > > Josh
> > > >
> > > > On Fri, Mar 12, 2021 at 5:42 AM Stephen Mallette
> > > > 
> > > > wrote:
> > > >
> > > > > I've been working on forming a pull request for this task. I
> > > > > don't
> > > > think IP
> > > > > Clearance is necessary as I originally did because the
> > > > > contribution is
> > > > > really just an ANTLR4 grammar file with some tests to validate
> > > > > things.
> > > > > Therefore, it's not a big body of independent code as I'd
> > > > > perhaps
> > > > initially
> > > > > envisioned. Compared to gremlint, this addition is pretty
> > > > > simple and
> > > > > straightforward. I've created this issue in JIRA with some
> > > > > additional
> > > > notes
> > > > > on what to expect in this initial body of work:
> > > > >
> > > > > https://issues.apache.org/jira/browse/TINKERPOP-2533
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Feb 8, 2021 at 10:06 AM Stephen Mallette
> > > > > 
> > > > > wrote:
> > > > >
> > > > > > Just wanted to leave an update on this thread. It was nice to
> > > > > > see some
> > > > > > support for it. I've not had time to focus on the task itself
> > > > > > so sorry
> > > > > > there hasn't been much movement, but I hope to see it on
> > > > > > track soon. I
> > > > > > thought to update the thread after I came across yet another
> > > > > > nice
> > > > usage
> > > > > for
> > > > > > it. I've long wanted to unify our test framework (i.e.
> > > > > > deprecate the
> > > > JVM
> > > > > > process suite in favor of the GLV test suite). I was
> > > > > > experimenting
> > > > with
> > > > > > what that might look like on Friday and hit a circular
> > > > > > dependency
> > > > which
> > > > > > constantly trips things up where gremlin-test wants to depend
> > > > > > on
> > > > > > gremlin-groovy (for ScriptEngine support) but gremlin-groovy
> > > > > > depends
> > > > on
> > > > > > gremlin-test and tinkergraph with  scope already. I
> > > > > > think the
> > > > > > introduction of gremlin-script would let gremlin-test build
> > > > > > the
> > > > Traversal
> > > > > > object from a Gremlin string and thus avoid that circular
> > > > relationship.
> > > > > >
> > > > > > On Fri, Jan 8, 2021 at 2:43 AM pieter gmail
> > > > > > 
> > > > > > wrote:
> > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > I have often thought the language specification should be a
> > > > > > > project
> > > > > > > separate from the implementations, and done in a formal but
> > > > > > > plain
> > > > > > > English format similar to OMG or IETF specifications.
> > > > > > >
> > > > > > > I suspect Sqlg's code base would have been fastly different
> > > > > > > if it had
> > > > > > > evolved from a grammer instead of an api.
> > > > > > >
> > > > > > > Cheers
> > > > > > > Pieter
> > > > > > >
> > > > > > > On Thu, 2020-12-24 at 14:41 -0500, Stephen Mallette wrote:
> > > > > > > > As a project, over the years, we've often been asked the
> > > > > > > > question
> > > > as
> > > > > > > > to why
> > > > > > > > Gremlin doesn't have an ANTLR style grammar. There have
> > > > > > > > been
> > > > varying
> > > > > > > > answers over the years to explain the reasoning but in
> > > > > > > > recent years
> > > > > > > > I've
> > > > > > > > started to see where our dependence on Java for driving
> > > > > > > > Gremlin
> > > > > > > > design has
> > > > > > > > not translated well as we have expanded Gremlin into
> > > > > > > > other
> > > > > > > >

Re: [DISCUSS] ANTLR and gremlin-script

[pieter gmail]

Hi,

Exciting as this is I am not quite sure what it means.

Naively  perhaps it the idea,
Arbitary gremlin string -> antlr parser -> some AST walker -> gremlin
byte code -> java in memory steps ... -> voila

Is the grammar going to be the primary and only
interface/specification, or will the native java implementation bypass
the grammar going straight to the steps instead?

Is this aimed at the gremlin 3 or 4?

Cheers
Pieter

On Tue, 2021-03-16 at 15:47 -0400, Stephen Mallette wrote:
> Here is the PR: https://github.com/apache/tinkerpop/pull/1408
> 
> On Tue, Mar 16, 2021 at 6:14 AM Stephen Mallette
> 
> wrote:
> 
> > No branch yet, but I think I will be sending the PR today.
> > 
> > On Mon, Mar 15, 2021 at 9:33 PM Joshua Shinavier
> > 
> > wrote:
> > 
> > > Is there a branch we can take a look at before the PR is ready?
> > > 
> > > Josh
> > > 
> > > On Fri, Mar 12, 2021 at 5:42 AM Stephen Mallette
> > > 
> > > wrote:
> > > 
> > > > I've been working on forming a pull request for this task. I
> > > > don't
> > > think IP
> > > > Clearance is necessary as I originally did because the
> > > > contribution is
> > > > really just an ANTLR4 grammar file with some tests to validate
> > > > things.
> > > > Therefore, it's not a big body of independent code as I'd
> > > > perhaps
> > > initially
> > > > envisioned. Compared to gremlint, this addition is pretty
> > > > simple and
> > > > straightforward. I've created this issue in JIRA with some
> > > > additional
> > > notes
> > > > on what to expect in this initial body of work:
> > > > 
> > > > https://issues.apache.org/jira/browse/TINKERPOP-2533
> > > > 
> > > > 
> > > > 
> > > > On Mon, Feb 8, 2021 at 10:06 AM Stephen Mallette
> > > > 
> > > > wrote:
> > > > 
> > > > > Just wanted to leave an update on this thread. It was nice to
> > > > > see some
> > > > > support for it. I've not had time to focus on the task itself
> > > > > so sorry
> > > > > there hasn't been much movement, but I hope to see it on
> > > > > track soon. I
> > > > > thought to update the thread after I came across yet another
> > > > > nice
> > > usage
> > > > for
> > > > > it. I've long wanted to unify our test framework (i.e.
> > > > > deprecate the
> > > JVM
> > > > > process suite in favor of the GLV test suite). I was
> > > > > experimenting
> > > with
> > > > > what that might look like on Friday and hit a circular
> > > > > dependency
> > > which
> > > > > constantly trips things up where gremlin-test wants to depend
> > > > > on
> > > > > gremlin-groovy (for ScriptEngine support) but gremlin-groovy
> > > > > depends
> > > on
> > > > > gremlin-test and tinkergraph with  scope already. I
> > > > > think the
> > > > > introduction of gremlin-script would let gremlin-test build
> > > > > the
> > > Traversal
> > > > > object from a Gremlin string and thus avoid that circular
> > > relationship.
> > > > > 
> > > > > On Fri, Jan 8, 2021 at 2:43 AM pieter gmail
> > > > > 
> > > > > wrote:
> > > > > 
> > > > > > +1
> > > > > > 
> > > > > > I have often thought the language specification should be a
> > > > > > project
> > > > > > separate from the implementations, and done in a formal but
> > > > > > plain
> > > > > > English format similar to OMG or IETF specifications.
> > > > > > 
> > > > > > I suspect Sqlg's code base would have been fastly different
> > > > > > if it had
> > > > > > evolved from a grammer instead of an api.
> > > > > > 
> > > > > > Cheers
> > > > > > Pieter
> > > > > > 
> > > > > > On Thu, 2020-12-24 at 14:41 -0500, Stephen Mallette wrote:
> > > > > > > As a project, over the years, we've often been asked the
> > > > > > > question
> > > as
> > > > > > > to why
> > > > > > > Gremlin doesn't have an ANTLR style grammar. There have
> > > > > > > been
> > > varying
> > > > > > > answers over the years to explain the reasoning but in
> > > > > > > recent years
> > > > > > > I've
> > > > > > > started to see where our dependence on Java for driving
> > > > > > > Gremlin
> > > > > > > design has
> > > > > > > not translated well as we have expanded Gremlin into
> > > > > > > other
> > > > > > > programming
> > > > > > > ecosystems. Using Java has often allowed idioms of that
> > > > > > > language to
> > > > > > > leak
> > > > > > > into Gremlin itself which introduces friction when
> > > > > > > implemented
> > > > > > > outside of
> > > > > > > the JVM. I think that there is some advantage to
> > > > > > > designing Gremlin
> > > > > > > more
> > > > > > > with just graphs/usage in mind and then determining how
> > > > > > > that design
> > > > > > > choice
> > > > > > > looks in each programming language.
> > > > > > > 
> > > > > > > I think that using an ANTLR grammar to drive that design
> > > > > > > work for
> > > > > > > Gremlin
> > > > > > > makes a lot of sense in this context. We would
> > > > > > > effectively have
> > > > > > > something
> > > > > > > like a gremlin-script which would become the new language
> > > archetype.
> > > > > > > New
> > > > > > > st

Re: [DISCUSS] ANTLR and gremlin-script

[Joshua Shinavier]

Awesome. This definitely fills a major gap, and as Florian said, it is
great that it is already improving the documentation.

High level-question: do you have any thoughts about Gremlin syntax
validation *in general*, i.e. for language variants other than Groovy? It
would be interesting to explore the following:

   1. Specify an abstract Gremlin grammar in a neutral language like YAML
   2. Write some helper code for generating ANTLR grammars from the YAML
   3. For each Gremlin language variant, write a smaller amount of code
   based on (2) to generate a language-specific ANTLR grammar

Based on my experience with Dragon, I have pretty good handle on what the
YAML (1) would need to look like, and how to write language-neutral helper
code (2). What would take a little more investigation is to what extent we
could do (3) on the basis of an understanding of the target language alone.
E.g. if a new step or signature is added to Gremlin, will it be enough to
add a specification of the step to the abstract grammar, or would we need
to special-case the step for each language variant? I suspect we wouldn't
have to do *too* much special-casing, but that's to be determined.

I might suggest calling the module "gremlin-language" if we were to
undertake the above. That would also allow other schemas to be provided
which would help us generate the structure API into different
implementation languages in a consistent way, as we discussed last year
.

Btw. I will be giving a Category Theory and Applications presentation
 next week
which will illustrate how something like the above might work.

Josh



On Tue, Mar 16, 2021 at 12:48 PM Stephen Mallette 
wrote:

> Here is the PR: https://github.com/apache/tinkerpop/pull/1408
>
> On Tue, Mar 16, 2021 at 6:14 AM Stephen Mallette 
> wrote:
>
> > No branch yet, but I think I will be sending the PR today.
> >
> > On Mon, Mar 15, 2021 at 9:33 PM Joshua Shinavier 
> > wrote:
> >
> >> Is there a branch we can take a look at before the PR is ready?
> >>
> >> Josh
> >>
> >> On Fri, Mar 12, 2021 at 5:42 AM Stephen Mallette 
> >> wrote:
> >>
> >> > I've been working on forming a pull request for this task. I don't
> >> think IP
> >> > Clearance is necessary as I originally did because the contribution is
> >> > really just an ANTLR4 grammar file with some tests to validate things.
> >> > Therefore, it's not a big body of independent code as I'd perhaps
> >> initially
> >> > envisioned. Compared to gremlint, this addition is pretty simple and
> >> > straightforward. I've created this issue in JIRA with some additional
> >> notes
> >> > on what to expect in this initial body of work:
> >> >
> >> > https://issues.apache.org/jira/browse/TINKERPOP-2533
> >> >
> >> >
> >> >
> >> > On Mon, Feb 8, 2021 at 10:06 AM Stephen Mallette <
> spmalle...@gmail.com>
> >> > wrote:
> >> >
> >> > > Just wanted to leave an update on this thread. It was nice to see
> some
> >> > > support for it. I've not had time to focus on the task itself so
> sorry
> >> > > there hasn't been much movement, but I hope to see it on track
> soon. I
> >> > > thought to update the thread after I came across yet another nice
> >> usage
> >> > for
> >> > > it. I've long wanted to unify our test framework (i.e. deprecate the
> >> JVM
> >> > > process suite in favor of the GLV test suite). I was experimenting
> >> with
> >> > > what that might look like on Friday and hit a circular dependency
> >> which
> >> > > constantly trips things up where gremlin-test wants to depend on
> >> > > gremlin-groovy (for ScriptEngine support) but gremlin-groovy depends
> >> on
> >> > > gremlin-test and tinkergraph with  scope already. I think the
> >> > > introduction of gremlin-script would let gremlin-test build the
> >> Traversal
> >> > > object from a Gremlin string and thus avoid that circular
> >> relationship.
> >> > >
> >> > > On Fri, Jan 8, 2021 at 2:43 AM pieter gmail <
> pieter.mar...@gmail.com>
> >> > > wrote:
> >> > >
> >> > >> +1
> >> > >>
> >> > >> I have often thought the language specification should be a project
> >> > >> separate from the implementations, and done in a formal but plain
> >> > >> English format similar to OMG or IETF specifications.
> >> > >>
> >> > >> I suspect Sqlg's code base would have been fastly different if it
> had
> >> > >> evolved from a grammer instead of an api.
> >> > >>
> >> > >> Cheers
> >> > >> Pieter
> >> > >>
> >> > >> On Thu, 2020-12-24 at 14:41 -0500, Stephen Mallette wrote:
> >> > >> > As a project, over the years, we've often been asked the question
> >> as
> >> > >> > to why
> >> > >> > Gremlin doesn't have an ANTLR style grammar. There have been
> >> varying
> >> > >> > answers over the years to explain the reasoning but in recent
> years
> >> > >> > I've
> >> > >> > started to see where our dependence on Java for driving Gremlin
> >> > >> > design has
> >> > >> > not transla

[jira] [Commented] (TINKERPOP-2538) traversal is not always closed in Tinkerpop test suites

[Stephen Mallette (Jira)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306126#comment-17306126
 ] 

Stephen Mallette commented on TINKERPOP-2538:
-

I had thought that we had handled this problem a long time ago, but I can't 
find evidence for it. This was definitely an issue for orientdb at one point 
and I think an issue with Titan/JanusGraph. I can't help but wonder if I'm 
mixing this problem of closing {{Traversal}} with calling {{close()}} on the 
{{GraphProvider}} between suite runs as I do see some hooks in there for that, 
but I guess that doesn't really help in this case, especially in the situation 
you linked to in {{IoCustomTest}}. 

While I was writing ideas for how to fix this, it occurred to me that Divij has 
long ago added this:

https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/util/iterator/StoreIteratorCounter.java

it was meant to find iterator leaks and is used in TinkerGraph as an example 
which has helped find many such problems in the test suite. Here is the 
TinkerGraph implementation of {{Iterator}} that gets used to do this:

https://github.com/apache/tinkerpop/blob/master/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphIterator.java

I suppose it is worth noting that the test suite doesn't call {{close()}] 
explicitly much because iterating to the end of a {{Traversal}} should have the 
effect of releasing resources - i.e. {{close()}} gets called. You can see such 
things happen in {{next()}} for example:

https://github.com/apache/tinkerpop/blob/7f7d3a485c7f100f98047b71672a0c2c9ab855b4/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/util/DefaultTraversal.java#L230

Why did the iterator leak thing divij built not catch this leak? I think it's 
because {{TinkerGraphIterator}} has a {{tryComputeNext()}} to prefetch the next 
value and therefore a call to {{next()}} could trigger the end of a traversal. 

That said, I'm not sure what the fix should be here. Going through the test 
suite to explicitly add {{close()}} everywhere is a lot of busy work and is 
then no guarantee that new ones won't slip in later. We could build registries 
for traversals in {{GraphProvider}} or elsewhere to track all the traversals 
created so that they could be closed, but that's probably an equal amount of 
refactoring or worse. Perhaps graph providers should look at 
{{TinkerGraphIterator}} to solve the problem? Maybe for test suite purposes a 
pre-fetch could be enabled to allow the release of resources?

> traversal is not always closed in Tinkerpop test suites
> ---
>
> Key: TINKERPOP-2538
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2538
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: test-suite
>Affects Versions: 3.4.10
>Reporter: Norio Akagi
>Priority: Minor
>
> Tinkerpop offers test suites so that GraphProviders can utilize and test 
> their own Graph database implementation.
> [http://tinkerpop.apache.org/docs/3.4.10-SNAPSHOT/dev/provider/#validating-with-gremlin-test]
> The concept is that a graph provider can pass GraphProvider and Graph class 
> to inject their own implementation. GraphProvider needs to implement *clear* 
> method to clean up resources after each test case is complete.
> However, depending on the design GraphProvider's implementation requires 
> GraphTraversal to be closed after a query is complete. Right now there is no 
> way for Graph instance to clean up running traversals.
> This leads to the issue that in unit test case some resource won't be 
> released.
> One example can be seen here:
> [https://github.com/apache/tinkerpop/blob/0d266da3e5c274afa9306367263e5c9098bedd93/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/structure/io/IoCustomTest.java#L151]
> In the code above, we run one traversal to get a Vertex but the traversal 
> `g1.traversal()` is not closed.
> Ideally Tinkerpop should take this use-case into account and clean up 
> traversals in test-suites (or at least provide a way to do that for providers 
> without touching the test-suite code base).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DSISCUSS] Remote Transactions

[Stephen Mallette]

>  I assume there will also be an explicit rollback ?

yes, you could call rollback() directly (I think that's what you're asking):

g = traversal().withRemote(conn)
gtx = g.tx().create()
assert gtx.isOpen() == true
gtx.addV('person').iterate()
gtx.addV('software').iterate()
gtx.rollback() // alternatively you could explicitly commit()
assert gtx.isOpen() == false




On Sat, Mar 20, 2021 at 10:34 AM Kelvin Lawrence 
wrote:

> Just a quick comment to add to what others have said. This functionality
> has been a common ask by users who do not want to have to fall back to
> sending text queries just to get the semantics of a session. Thanks for
> taking this on. It will be a nice addition to the Gremlin client/server
> interaction model. The syntax you proposed looks good to me. I assume there
> will also be an explicit rollback ?
>
> Cheers,
> Kelvin
>
> Kelvin R. Lawrence
> http://www.kelvinlawrence.net
> https://www.linkedin.com/in/krlawrence/
> Twitter: @gfxman
>
> From: Stephen Mallette
> Sent: Thursday, March 18, 2021 7:16 PM
> To: dev@tinkerpop.apache.org
> Subject: Re: [DSISCUSS] Remote Transactions
>
> I've created this issue for tracking:
>
> https://issues.apache.org/jira/browse/TINKERPOP-2537
>
> and have pushed a branch for early review:
>
> https://github.com/apache/tinkerpop/tree/TINKERPOP-2537
>
> The server-side is a bit of a copy/paste mess at the moment, so try not to
> pay attention there too much and focus more on the driver and Transaction
> interfaces side of things. At this point I'm amazed at how easily this fell
> together. It feels like I'm almost overlooking something important as I'd
> not have guessed this to go in so straightforwardly. The hardest part was
> figuring out the interaction among the various interfaces to get the user
> API right but once that was untangled, stuff just started working.
>
> At this point, I think I need to pay some attention to the server-side
> to make this pull request ready. I basically cut/paste a giant grip of code
> from TraversalOpProcessor into SessionOpProcessor and tweaked a few lines
> here and there. Not terribly ideal, but it was the fastest way to be sure
> that the general premise of the client-side was going to actually work. Now
> the question is how to make the server part nice, but refactoring there
> isn't traditionally easy unfortunately.
>
>
>
>
> On Thu, Mar 18, 2021 at 12:53 PM Stephen Mallette 
> wrote:
>
> > First of all, I managed to fumble my last code sample - sorry. You can
> > either use the Transaction object explicitly like this:
> >
> > g = traversal().withRemote(conn)
> > tx = g.tx() // Transaction object
> > gtx = tx.begin()  // GraphTraversalSource
> > assert tx.isOpen() == true
> > gtx.addV('person').iterate()
> > gtx.addV('software').iterate()
> > tx.commit() // alternatively you could explicitly rollback()
> > assert tx.isOpen() == false
> >
> > Or you can look more like the old style:
> >
> > g = traversal().withRemote(conn)
> > gtx = g.tx().begin()  // GraphTraversalSource
> > assert gtx.tx().isOpen() == true
> > gtx.addV('person').iterate()
> > gtx.addV('software').iterate()
> > gtx.tx().commit() // alternatively you could explicitly rollback()
> > assert gtx.tx().isOpen() == false
> >
> > Both look fairly fluid I think and I'm especially pleased with the second
> > one. Need to do a bit more to prevent weird nested transactions and such
> > but pretty neat for a just a day's worth of hacking about.
> >
> > Answers to Divij's questions inline below:
> >
> > On Thu, Mar 18, 2021 at 11:52 AM Divij Vaidya 
> > wrote:
> >
> >> Great idea Stephen. I agree with standardizing the explicit transaction
> >> semantics in cases of remote server (implicit aka sessionless is already
> >> self explanatory) and unifying the syntax with embedded graph usage.
> >>
> >> Couple of notes:
> >>
> >> 1. I would favor the begin() instead of create() as it's closer to
> >> universally prominent SQL transaction syntax. This would reduce the
> >> learning curve for users of TP.
> >>
> >
> > I'm ok with begin().
> >
> >
> >> 2. From an implementation standpoint, sessionless requests are queued on
> >> the server side and are serviced by the worker thread pool. But a
> >> session-ful aka explicit transaction aka managed transaction starts a
> new
> >> single worked thread pool every time. In a scenario where we have both
> >> session-less and sesion-ful requests being made to the server, the
> >> resource
> >> allocation will not be fair and may lead to starvation for some
> requests.
> >> This problem exists even today, hence not totally correlated to what you
> >> are proposing but I am afraid a wider adoption of explicit
> >> transactions will bring this problem to the spotlight. Hence, we should
> >> fix
> >> this problem first. A solution would entail converging the worker pool
> for
> >> both session vs session-less requests.
> >>
> >> 3. You are proposing the idea of having a transaction scoped traversal
> >> object. I agree wi