Re: [DISCUSS] ANTLR and gremlin-script
+1 I have often thought the language specification should be a project separate from the implementations, and done in a formal but plain English format similar to OMG or IETF specifications. I suspect Sqlg's code base would have been fastly different if it had evolved from a grammer instead of an api. Cheers Pieter On Thu, 2020-12-24 at 14:41 -0500, Stephen Mallette wrote: > As a project, over the years, we've often been asked the question as > to why > Gremlin doesn't have an ANTLR style grammar. There have been varying > answers over the years to explain the reasoning but in recent years > I've > started to see where our dependence on Java for driving Gremlin > design has > not translated well as we have expanded Gremlin into other > programming > ecosystems. Using Java has often allowed idioms of that language to > leak > into Gremlin itself which introduces friction when implemented > outside of > the JVM. I think that there is some advantage to designing Gremlin > more > with just graphs/usage in mind and then determining how that design > choice > looks in each programming language. > > I think that using an ANTLR grammar to drive that design work for > Gremlin > makes a lot of sense in this context. We would effectively have > something > like a gremlin-script which would become the new language archetype. > New > steps, language changes, etc. would be discussed in its context and > then > implemented in the grammar and later in each programming language we > support in the style a developer would expect. An interesting upside > of > this approach is that we can implement gremlin-script in the > ScriptEngine > and replace GremlinGroovyScriptEngine which would help us strengthen > our > security story in Gremlin Server. Groovy processing would just be a > fallback to Gremlin scripts that could not be processed by the AST. > In fact > users who didn't need Groovy could simply not install it at all and > thus > boast a much more secure system. > > I think that inclusion of a grammar in our project is an exciting new > direction for us to take and will help in a variety of areas beyond > those > I've already related. > > If we like this direction, Amazon Neptune already maintains such a > grammar > and would be willing to contribute it to the project to live in open > source. The contribution would go through the same IP Clearance > process > gremlint is going through since it was developed outside of > TinkerPop. I'd > be happy to guide that process through if we draw to consensus here.
[jira] [Commented] (TINKERPOP-2481) IdentityRemovalStrategy not installed
[
https://issues.apache.org/jira/browse/TINKERPOP-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260900#comment-17260900
]
ASF GitHub Bot commented on TINKERPOP-2481:
---
bechbd commented on pull request #1378:
URL: https://github.com/apache/tinkerpop/pull/1378#issuecomment-756476302
+1
I took a look at this and I think that change makes a lot of sense from a
usability perspective. Honestly, I didn't know that was how `match` worked and
would have been confused
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> IdentityRemovalStrategy not installed
> -
>
> Key: TINKERPOP-2481
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2481
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.4.8
>Reporter: Stephen Mallette
>Priority: Minor
> Labels: breaking
>
> I'm not sure why but {{IdentityRemovalStrategy}} is not in play and almost
> looks like it never has been, yet I fairly certain we've gone about life
> thinking it was one of our standard strategies. Perhaps this issue title will
> shift once this is given more thought but I came to [notice this
> issue|https://groups.google.com/g/gremlin-users/c/HVtldzV0Xk8/m/saCBYsEQBgAJ]
> with this example:
> {code}
> gremlin> g.V().match(__.as("a").out("knows").as("b")).identity()
> ==>[]
> ==>[]
> gremlin>
> g.withStrategies(IdentityRemovalStrategy.instance()).V().match(__.as("a").out("knows").as("b")).identity()
> ==>[a:v[1],b:v[2]]
> ==>[a:v[1],b:v[4]]
> {code}
> So if the strategy had been in play, {{PathRetractionStrategy}} wouldn't have
> trimmed the history. Or, perhaps {{PathRetractionStrategy}} should have been
> smarter
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [DISCUSS] ANTLR and gremlin-script
First of all: I love this direction, for all of the reasons cited and the innovation I'm hoping that it opens up. Also, that Amazon Neptune has already done the work and would be willing to share it with the rest of the world is a remarkable gift. I'm just a user / admirer of the technology, but as an opinionated graph junkie, this is a really great thing. -Josh On Thu, Dec 24, 2020 at 1:41 PM Stephen Mallette wrote: > As a project, over the years, we've often been asked the question as to why > Gremlin doesn't have an ANTLR style grammar. There have been varying > answers over the years to explain the reasoning but in recent years I've > started to see where our dependence on Java for driving Gremlin design has > not translated well as we have expanded Gremlin into other programming > ecosystems. Using Java has often allowed idioms of that language to leak > into Gremlin itself which introduces friction when implemented outside of > the JVM. I think that there is some advantage to designing Gremlin more > with just graphs/usage in mind and then determining how that design choice > looks in each programming language. > > I think that using an ANTLR grammar to drive that design work for Gremlin > makes a lot of sense in this context. We would effectively have something > like a gremlin-script which would become the new language archetype. New > steps, language changes, etc. would be discussed in its context and then > implemented in the grammar and later in each programming language we > support in the style a developer would expect. An interesting upside of > this approach is that we can implement gremlin-script in the ScriptEngine > and replace GremlinGroovyScriptEngine which would help us strengthen our > security story in Gremlin Server. Groovy processing would just be a > fallback to Gremlin scripts that could not be processed by the AST. In fact > users who didn't need Groovy could simply not install it at all and thus > boast a much more secure system. > > I think that inclusion of a grammar in our project is an exciting new > direction for us to take and will help in a variety of areas beyond those > I've already related. > > If we like this direction, Amazon Neptune already maintains such a grammar > and would be willing to contribute it to the project to live in open > source. The contribution would go through the same IP Clearance process > gremlint is going through since it was developed outside of TinkerPop. I'd > be happy to guide that process through if we draw to consensus here. >
[jira] [Created] (TINKERPOP-2504) Failing ClientConnectionIntegrateTest
Stephen Mallette created TINKERPOP-2504:
---
Summary: Failing ClientConnectionIntegrateTest
Key: TINKERPOP-2504
URL: https://issues.apache.org/jira/browse/TINKERPOP-2504
Project: TinkerPop
Issue Type: Improvement
Components: driver, server
Affects Versions: 3.4.9
Reporter: Stephen Mallette
I've noticed this test failing intermittently on Travis and more consistently
on the {{master}} branch with Docker. It fails with a
{{ConcurrentModificationException}} (haven't been able to easily get the entire
stack trace thanks to the docker issue and i've not caught it happening in the
last few days on Travis). Adding this line of code seems to have made the test
pass more consistently:
https://github.com/apache/tinkerpop/commit/4b099b3c84a350aae953cdf517aa11c7017eb2ae
which would indicate something perhaps fishy with how hosts are being marked
dead and iterated. Would be nice to get rid of that little hack.
cc/ [~divijvaidya]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2481) IdentityRemovalStrategy not installed
[
https://issues.apache.org/jira/browse/TINKERPOP-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260730#comment-17260730
]
ASF GitHub Bot commented on TINKERPOP-2481:
---
spmallette opened a new pull request #1378:
URL: https://github.com/apache/tinkerpop/pull/1378
https://issues.apache.org/jira/browse/TINKERPOP-2481
https://issues.apache.org/jira/browse/TINKERPOP-2499
Before:
```text
gremlin> g.V().match(__.as("a").out("knows").as("b"))
==>[a:v[1],b:v[2]]
==>[a:v[1],b:v[4]]
gremlin> g.V().match(__.as("a").out("knows").as("b")).unfold()
gremlin> g.V().match(__.as("a").out("knows").as("b")).identity()
==>[]
==>[]
```
After:
```text
gremlin> g.V().match(__.as("a").out("knows").as("b"))
==>[a:v[1],b:v[2]]
==>[a:v[1],b:v[4]]
gremlin> g.V().match(__.as("a").out("knows").as("b")).identity()
==>[a:v[1],b:v[2]]
==>[a:v[1],b:v[4]]
gremlin> g.V().match(__.as("a").out("knows").as("b")).unfold()
==>a=v[1]
==>b=v[2]
==>a=v[1]
==>b=v[4]
```
This change goes for consistency of output over optimization avoiding a
situation where the addition of a step alters its behavior in some unexpected
way. This is effectively a breaking change as it modifies the output of
traversal output.
All tests pass with `docker/build.sh -t -n -i`
VOTE +1
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> IdentityRemovalStrategy not installed
> -
>
> Key: TINKERPOP-2481
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2481
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.4.8
>Reporter: Stephen Mallette
>Priority: Minor
>
> I'm not sure why but {{IdentityRemovalStrategy}} is not in play and almost
> looks like it never has been, yet I fairly certain we've gone about life
> thinking it was one of our standard strategies. Perhaps this issue title will
> shift once this is given more thought but I came to [notice this
> issue|https://groups.google.com/g/gremlin-users/c/HVtldzV0Xk8/m/saCBYsEQBgAJ]
> with this example:
> {code}
> gremlin> g.V().match(__.as("a").out("knows").as("b")).identity()
> ==>[]
> ==>[]
> gremlin>
> g.withStrategies(IdentityRemovalStrategy.instance()).V().match(__.as("a").out("knows").as("b")).identity()
> ==>[a:v[1],b:v[2]]
> ==>[a:v[1],b:v[4]]
> {code}
> So if the strategy had been in play, {{PathRetractionStrategy}} wouldn't have
> trimmed the history. Or, perhaps {{PathRetractionStrategy}} should have been
> smarter
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (TINKERPOP-2499) PathRetractionStrategy returns inconsistent results when match() is not detected as the final step
[
https://issues.apache.org/jira/browse/TINKERPOP-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Mallette updated TINKERPOP-2499:
Labels: breaking (was: )
> PathRetractionStrategy returns inconsistent results when match() is not
> detected as the final step
> --
>
> Key: TINKERPOP-2499
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2499
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.4.9
>Reporter: Stephen Mallette
>Priority: Major
> Labels: breaking
>
> This issue comes from TINKERPOP-2481 where it was originally noted that
> perhaps {{IdentityRemovalStrategy}} had something to do with the following:
> {code}
> gremlin> g.V().match(__.as("a").out("knows").as("b")).identity()
> ==>[]
> ==>[]
> gremlin>
> g.withStrategies(IdentityRemovalStrategy.instance()).V().match(__.as("a").out("knows").as("b")).identity()
> ==>[a:v[1],b:v[2]]
> ==>[a:v[1],b:v[4]]
> gremlin>
> g.withoutStrategies(PathRetractionStrategy).V().match(__.as("a").out("knows").as("b")).identity()
> ==>[a:v[1],b:v[2]]
> ==>[a:v[1],b:v[4]]
> gremlin> g.V().match(__.as("a").out("knows").as("b")).unfold()
> gremlin> g.V().match(__.as("a").out("knows").as("b")).unfold().path()
> ==>[v[1],v[1],v[2],[a:v[1],b:v[2]],a=v[1]]
> ==>[v[1],v[1],v[2],[a:v[1],b:v[2]],b=v[2]]
> ==>[v[1],v[1],v[4],[a:v[1],b:v[4]],a=v[1]]
> ==>[v[1],v[1],v[4],[a:v[1],b:v[4]],b=v[4]]
> {code}
> but it is not a good idea to fix the behavior of one strategy by way of
> another so {{PathRetractionStrategy}} needs to get a bit smarter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (TINKERPOP-2481) IdentityRemovalStrategy not installed
[
https://issues.apache.org/jira/browse/TINKERPOP-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Mallette updated TINKERPOP-2481:
Labels: breaking (was: )
> IdentityRemovalStrategy not installed
> -
>
> Key: TINKERPOP-2481
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2481
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.4.8
>Reporter: Stephen Mallette
>Priority: Minor
> Labels: breaking
>
> I'm not sure why but {{IdentityRemovalStrategy}} is not in play and almost
> looks like it never has been, yet I fairly certain we've gone about life
> thinking it was one of our standard strategies. Perhaps this issue title will
> shift once this is given more thought but I came to [notice this
> issue|https://groups.google.com/g/gremlin-users/c/HVtldzV0Xk8/m/saCBYsEQBgAJ]
> with this example:
> {code}
> gremlin> g.V().match(__.as("a").out("knows").as("b")).identity()
> ==>[]
> ==>[]
> gremlin>
> g.withStrategies(IdentityRemovalStrategy.instance()).V().match(__.as("a").out("knows").as("b")).identity()
> ==>[a:v[1],b:v[2]]
> ==>[a:v[1],b:v[4]]
> {code}
> So if the strategy had been in play, {{PathRetractionStrategy}} wouldn't have
> trimmed the history. Or, perhaps {{PathRetractionStrategy}} should have been
> smarter
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Make Gremlin Server Settings file more extensible
Could you be more specific about what you propose in your second idea? I guess i'd like to know how you would like to get the Constructor and how it would affect the API. Thanks On Wed, Jan 6, 2021 at 7:13 AM wrote: > Hi > > > We (JanusGraph) are already automatically settings overwriting defaults of > gremlin server settings, see here > https://github.com/JanusGraph/janusgraph/blob/master/janusgraph-server/src/main/java/org/janusgraph/graphdb/server/utils/JanusGraphSettingsUtils.java. > We are working on adding a GrpcServer to JanusGraph for management > interactions. Therefore, we would like to extend gremlin settings with our > own settings. The main problem is that gremlin using strict parsing of Yaml > which is great in the most place but prevent us from easying extending it. > > I have some ideas: > > * Provide a config flag to allow unknown properties (easiest solution) > * Provide a method which returns the snakeyaml Constructor > https://github.com/apache/tinkerpop/blob/master/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/Settings.java, > we could use it extend the constructor our way. > > Our current way to use just one config: > > 1. Parse our settings > 2. Parse gremlin-server.yaml as Map > 3. Remove our custom fields > 4. Reconstruct yaml from map > 5. Parse yaml with Settings.read > > Any idea's? > > Greetings, > > Jan >
[jira] [Commented] (TINKERPOP-2501) Websocket Connection handleError in javascript gremlin driver emits 'log' event instead of 'error'
[ https://issues.apache.org/jira/browse/TINKERPOP-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260547#comment-17260547 ] Stephen Mallette commented on TINKERPOP-2501: - I'm not an expert on the Javascript client but I think what you describe makes sense - please feel free to submit a PR for further review. > Websocket Connection handleError in javascript gremlin driver emits 'log' > event instead of 'error' > -- > > Key: TINKERPOP-2501 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2501 > Project: TinkerPop > Issue Type: Bug > Components: javascript >Reporter: Aditya Srivastava >Priority: Minor > > *Description* > Javascript gremlin module has one weird logging behavior during client > connection setup. > The webSocket error event emits a *handleError* function > [https://github.com/apache/tinkerpop/blob/64ec9fbc9e6d368f1a251b25cf7b120ace09298c/gremlin-javascript/src/main/javascript/gremlin-javascript/lib/driver/connection.js#L155] > and this function emits the error message under *log* event i.e > [https://github.com/apache/tinkerpop/blob/64ec9fbc9e6d368f1a251b25cf7b120ace09298c/gremlin-javascript/src/main/javascript/gremlin-javascript/lib/driver/connection.js#L235]. > It looks like this was broken recently by > [https://github.com/apache/tinkerpop/commit/ab43b03514cf82fa004653c3a0d082e8a744107d] > this commit. > I believe this should ideally be thrown under *error* event. Right now we > have to listen to these connection errors under *log* event. > I'll be happy to raise a pr for this once the bug gets approved by the core > team. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2502) Consistent start API for anonymous and regular traversals
[
https://issues.apache.org/jira/browse/TINKERPOP-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260515#comment-17260515
]
Stephen Mallette commented on TINKERPOP-2502:
-
I think I understand the problem you're describing, but I'm not quite picturing
the weight of the frustration. Could you share some code that demonstrates how
you librarify things?
> Consistent start API for anonymous and regular traversals
> -
>
> Key: TINKERPOP-2502
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2502
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.4.9
>Reporter: Christopher Smith
>Priority: Minor
>
> I am writing a graph-based application and am taking advantage of the
> method-based nature of Gremlin to librarify some of my common usage patterns.
> However, I'm encountering a frustrating problem where I frequently need to be
> able to attach the same traversal steps to either a real
> {{GraphTraversalSource}} or an anonymous traversal (usually for some nested
> reason like a coalesced conditional insert).
> The methods on {{__}} are mostly static, but I can call {{__.start()}} to
> obtain a live {{GraphTraversal}} object and then proceed from there (invoking
> {{GraphTraversal#addV}}, for example). {{GraphTraversalSource}}, however,
> requires me to invoke the method {{GraphTraversalSource#addV}} to get
> started, and there's no common base type. I think I could theoretically use
> something like {{inject()}}, but that seems particularly odd.
> It would be helpful to either have both {{GraphTraversal}} and
> {{GraphTraversalSource}} implement a common interface holding the "start
> opcodes" or to have a {{GraphTraversalSource#start()}} method that could be
> used to obtain a "blank" traversal.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (TINKERPOP-2503) Implement look-ahead on PathRetractionStrategy
[
https://issues.apache.org/jira/browse/TINKERPOP-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Mallette updated TINKERPOP-2503:
Description:
With TINKERPOP-2499 we made traversal results from {{match()}} more consistent
but it came with the cost of not being as aggressive at trimming unneeded
labels. It would be good to get back to supporting patterns we once had (which
were much too strictly defined and relied on the execution of other strategies
to work) where we could properly recognize:
{code}
match(__.as("a").out("knows").as("b")).select('a')
match(__.as("a").out("knows").as("b")).dedup('a')
{code}
and perhaps even various calls to {{path()}} which has always completely exited
{{PathRetractionStrategy}}. If this gets done for 3.5.0, then Upgrade Docs for
"match() Consistency" should be edited
was:
With TINKERPOP-2499 we made traversal results from {{match()}} more consistent
but it came with the cost of not being as aggressive at trimming unneeded
labels. It would be good to get back to supporting patterns we once had (which
were much too strictly defined and relied on the execution of other strategies
to work) where we could properly recognize:
{code}
match(__.as("a").out("knows").as("b")).select('a')
match(__.as("a").out("knows").as("b")).dedup('a')
{code}
and perhaps even various calls to {{path()}} which has always completely exited
{{PathRetractionStrategy}}
> Implement look-ahead on PathRetractionStrategy
> --
>
> Key: TINKERPOP-2503
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2503
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.4.9
>Reporter: Stephen Mallette
>Priority: Major
>
> With TINKERPOP-2499 we made traversal results from {{match()}} more
> consistent but it came with the cost of not being as aggressive at trimming
> unneeded labels. It would be good to get back to supporting patterns we once
> had (which were much too strictly defined and relied on the execution of
> other strategies to work) where we could properly recognize:
> {code}
> match(__.as("a").out("knows").as("b")).select('a')
> match(__.as("a").out("knows").as("b")).dedup('a')
> {code}
> and perhaps even various calls to {{path()}} which has always completely
> exited {{PathRetractionStrategy}}. If this gets done for 3.5.0, then Upgrade
> Docs for "match() Consistency" should be edited
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (TINKERPOP-2503) Implement look-ahead on PathRetractionStrategy
Stephen Mallette created TINKERPOP-2503:
---
Summary: Implement look-ahead on PathRetractionStrategy
Key: TINKERPOP-2503
URL: https://issues.apache.org/jira/browse/TINKERPOP-2503
Project: TinkerPop
Issue Type: Improvement
Components: process
Affects Versions: 3.4.9
Reporter: Stephen Mallette
With TINKERPOP-2499 we made traversal results from {{match()}} more consistent
but it came with the cost of not being as aggressive at trimming unneeded
labels. It would be good to get back to supporting patterns we once had (which
were much too strictly defined and relied on the execution of other strategies
to work) where we could properly recognize:
{code}
match(__.as("a").out("knows").as("b")).select('a')
match(__.as("a").out("knows").as("b")).dedup('a')
{code}
and perhaps even various calls to {{path()}} which has always completely exited
{{PathRetractionStrategy}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (TINKERPOP-1925) Don't apply PathRetractionStrategy recursively
[
https://issues.apache.org/jira/browse/TINKERPOP-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Mallette reassigned TINKERPOP-1925:
---
Assignee: Stephen Mallette (was: Ted Wilmes)
> Don't apply PathRetractionStrategy recursively
> --
>
> Key: TINKERPOP-1925
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1925
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.2.7
>Reporter: Daniel Kuppitz
>Assignee: Stephen Mallette
>Priority: Major
>
> {{PathRetractionStrategy}} should not be applied recursively, if possible.
> This is considered an anti-pattern and just recently it caused a lot of
> trouble. This commit fixed the recent issue:
> https://github.com/apache/tinkerpop/commit/b8eb8a02f1675a1b22b86ccc54d22453c65f406f
> {{PathRetractionStrategy}} collects requirements of child traversals, before
> other strategies have a chance to mutate them. When other strategies get to
> the child traversals, mutate them and thus add or remove requirements, a few
> things can go crazy, especially because traverser requirements get cached
> when they're first requested.
> Since I do not fully understand the {{PathRetractionStrategy}} code, my fix
> was actually a workaround. As you can see in the commit linked above, I
> implemented the same anti-pattern in {{SubgraphStrategy}} and
> {{IncidentToAdjacentStrategy}}; these are the 2 strategies that can
> potentially change the traverser requirements.
> It would be nice to revert these changes and instead make
> {{PathRetractionStrategy}} work like all others. However, I'm not sure if
> that's even possible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
