Some thoughts inline: On Mon, Mar 22, 2021 at 2:37 PM pieter gmail <[email protected]> wrote:
> Hi, > > Exciting as this is I am not quite sure what it means. > > Naively perhaps it the idea, > Arbitary gremlin string -> antlr parser -> some AST walker -> gremlin > byte code -> java in memory steps ... -> voila > > Is the grammar going to be the primary and only > interface/specification, or will the native java implementation bypass > the grammar going straight to the steps instead? > Perhaps the most practical point of the grammar is that I don't have much faith in Groovy these days. It's been a security problem for all the major graph providers, there are many issues with performance (some of which we've managed to handle, some we've handled a bit less well and some perhaps not at all), and nowadays we aren't getting much response from their community on major issues that impact us: https://issues.apache.org/jira/browse/TINKERPOP-2526 So, the grammar seems like a good way to find a bit of independence there as we can process Gremlin strings (most of them anyway at this point) without Groovy. I used to think that Gremlin bytecode would allow us to get rid of Gremlin strings, but that seems like an impossibility. There are just too many users with that model. The same could be said of bytecode at this point. So I sense, neither is going away any time soon, so we may as well accept it and try to make both environments work as well as possible. Perhaps in TP4 someday we could make the decision of grammar or bytecode, but for now that decision doesn't seem like a realistic one to make. As I alluded to earlier in this thread, I do think that the grammar becomes "more primary" as the interface/specification over the native java implementation. I think the language variants suffered a bit because Gremlin was so heavily driven by Java. Thinking about Gremlin as a language first and then determining how it fits in Java, Groovy, Python, etc. I think will help us. > > Is this aimed at the gremlin 3 or 4? > The grammar will land in 3.5.0, but it won't do very much initially. It's there for us to start thinking around and to start to put it to work in the replacement of the GremlinGroovyScriptEngine - I think we would target that for 3.5.1. I don't think about TP4 that much tbh. I tend to think that there is a bit more to learn about Gremlin as a grammar and how to better support non-JVM languages before then. I also think there are a number of gaps to fill with some missing pieces to the Gremln language itself. All of that can happen in TP3 in my mind...It remains a decent place to experiment and learn. > > Cheers > Pieter > > On Tue, 2021-03-16 at 15:47 -0400, Stephen Mallette wrote: > > Here is the PR: https://github.com/apache/tinkerpop/pull/1408 > > > > On Tue, Mar 16, 2021 at 6:14 AM Stephen Mallette > > <[email protected]> > > wrote: > > > > > No branch yet, but I think I will be sending the PR today. > > > > > > On Mon, Mar 15, 2021 at 9:33 PM Joshua Shinavier > > > <[email protected]> > > > wrote: > > > > > > > Is there a branch we can take a look at before the PR is ready? > > > > > > > > Josh > > > > > > > > On Fri, Mar 12, 2021 at 5:42 AM Stephen Mallette > > > > <[email protected]> > > > > wrote: > > > > > > > > > I've been working on forming a pull request for this task. I > > > > > don't > > > > think IP > > > > > Clearance is necessary as I originally did because the > > > > > contribution is > > > > > really just an ANTLR4 grammar file with some tests to validate > > > > > things. > > > > > Therefore, it's not a big body of independent code as I'd > > > > > perhaps > > > > initially > > > > > envisioned. Compared to gremlint, this addition is pretty > > > > > simple and > > > > > straightforward. I've created this issue in JIRA with some > > > > > additional > > > > notes > > > > > on what to expect in this initial body of work: > > > > > > > > > > https://issues.apache.org/jira/browse/TINKERPOP-2533 > > > > > > > > > > > > > > > > > > > > On Mon, Feb 8, 2021 at 10:06 AM Stephen Mallette > > > > > <[email protected]> > > > > > wrote: > > > > > > > > > > > Just wanted to leave an update on this thread. It was nice to > > > > > > see some > > > > > > support for it. I've not had time to focus on the task itself > > > > > > so sorry > > > > > > there hasn't been much movement, but I hope to see it on > > > > > > track soon. I > > > > > > thought to update the thread after I came across yet another > > > > > > nice > > > > usage > > > > > for > > > > > > it. I've long wanted to unify our test framework (i.e. > > > > > > deprecate the > > > > JVM > > > > > > process suite in favor of the GLV test suite). I was > > > > > > experimenting > > > > with > > > > > > what that might look like on Friday and hit a circular > > > > > > dependency > > > > which > > > > > > constantly trips things up where gremlin-test wants to depend > > > > > > on > > > > > > gremlin-groovy (for ScriptEngine support) but gremlin-groovy > > > > > > depends > > > > on > > > > > > gremlin-test and tinkergraph with <test> scope already. I > > > > > > think the > > > > > > introduction of gremlin-script would let gremlin-test build > > > > > > the > > > > Traversal > > > > > > object from a Gremlin string and thus avoid that circular > > > > relationship. > > > > > > > > > > > > On Fri, Jan 8, 2021 at 2:43 AM pieter gmail > > > > > > <[email protected]> > > > > > > wrote: > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > I have often thought the language specification should be a > > > > > > > project > > > > > > > separate from the implementations, and done in a formal but > > > > > > > plain > > > > > > > English format similar to OMG or IETF specifications. > > > > > > > > > > > > > > I suspect Sqlg's code base would have been fastly different > > > > > > > if it had > > > > > > > evolved from a grammer instead of an api. > > > > > > > > > > > > > > Cheers > > > > > > > Pieter > > > > > > > > > > > > > > On Thu, 2020-12-24 at 14:41 -0500, Stephen Mallette wrote: > > > > > > > > As a project, over the years, we've often been asked the > > > > > > > > question > > > > as > > > > > > > > to why > > > > > > > > Gremlin doesn't have an ANTLR style grammar. There have > > > > > > > > been > > > > varying > > > > > > > > answers over the years to explain the reasoning but in > > > > > > > > recent years > > > > > > > > I've > > > > > > > > started to see where our dependence on Java for driving > > > > > > > > Gremlin > > > > > > > > design has > > > > > > > > not translated well as we have expanded Gremlin into > > > > > > > > other > > > > > > > > programming > > > > > > > > ecosystems. Using Java has often allowed idioms of that > > > > > > > > language to > > > > > > > > leak > > > > > > > > into Gremlin itself which introduces friction when > > > > > > > > implemented > > > > > > > > outside of > > > > > > > > the JVM. I think that there is some advantage to > > > > > > > > designing Gremlin > > > > > > > > more > > > > > > > > with just graphs/usage in mind and then determining how > > > > > > > > that design > > > > > > > > choice > > > > > > > > looks in each programming language. > > > > > > > > > > > > > > > > I think that using an ANTLR grammar to drive that design > > > > > > > > work for > > > > > > > > Gremlin > > > > > > > > makes a lot of sense in this context. We would > > > > > > > > effectively have > > > > > > > > something > > > > > > > > like a gremlin-script which would become the new language > > > > archetype. > > > > > > > > New > > > > > > > > steps, language changes, etc. would be discussed in its > > > > > > > > context and > > > > > > > > then > > > > > > > > implemented in the grammar and later in each programming > > > > > > > > language > > > > we > > > > > > > > support in the style a developer would expect. An > > > > > > > > interesting > > > > upside > > > > > > > > of > > > > > > > > this approach is that we can implement gremlin-script in > > > > > > > > the > > > > > > > > ScriptEngine > > > > > > > > and replace GremlinGroovyScriptEngine which would help us > > > > strengthen > > > > > > > > our > > > > > > > > security story in Gremlin Server. Groovy processing would > > > > > > > > just be a > > > > > > > > fallback to Gremlin scripts that could not be processed > > > > > > > > by the AST. > > > > > > > > In fact > > > > > > > > users who didn't need Groovy could simply not install it > > > > > > > > at all and > > > > > > > > thus > > > > > > > > boast a much more secure system. > > > > > > > > > > > > > > > > I think that inclusion of a grammar in our project is an > > > > > > > > exciting > > > > new > > > > > > > > direction for us to take and will help in a variety of > > > > > > > > areas beyond > > > > > > > > those > > > > > > > > I've already related. > > > > > > > > > > > > > > > > If we like this direction, Amazon Neptune already > > > > > > > > maintains such a > > > > > > > > grammar > > > > > > > > and would be willing to contribute it to the project to > > > > > > > > live in > > > > open > > > > > > > > source. The contribution would go through the same IP > > > > > > > > Clearance > > > > > > > > process > > > > > > > > gremlint is going through since it was developed outside > > > > > > > > of > > > > > > > > TinkerPop. I'd > > > > > > > > be happy to guide that process through if we draw to > > > > > > > > consensus > > > > here. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
