Awesome. This definitely fills a major gap, and as Florian said, it is great that it is already improving the documentation.
High level-question: do you have any thoughts about Gremlin syntax validation *in general*, i.e. for language variants other than Groovy? It would be interesting to explore the following: 1. Specify an abstract Gremlin grammar in a neutral language like YAML 2. Write some helper code for generating ANTLR grammars from the YAML 3. For each Gremlin language variant, write a smaller amount of code based on (2) to generate a language-specific ANTLR grammar Based on my experience with Dragon, I have pretty good handle on what the YAML (1) would need to look like, and how to write language-neutral helper code (2). What would take a little more investigation is to what extent we could do (3) on the basis of an understanding of the target language alone. E.g. if a new step or signature is added to Gremlin, will it be enough to add a specification of the step to the abstract grammar, or would we need to special-case the step for each language variant? I suspect we wouldn't have to do *too* much special-casing, but that's to be determined. I might suggest calling the module "gremlin-language" if we were to undertake the above. That would also allow other schemas to be provided which would help us generate the structure API into different implementation languages in a consistent way, as we discussed last year <https://www.slideshare.net/joshsh/tinkerpop-2020>. Btw. I will be giving a Category Theory and Applications presentation <https://www.meetup.com/Category-Theory/events/nnrhgsyccfbhc/> next week which will illustrate how something like the above might work. Josh On Tue, Mar 16, 2021 at 12:48 PM Stephen Mallette <[email protected]> wrote: > Here is the PR: https://github.com/apache/tinkerpop/pull/1408 > > On Tue, Mar 16, 2021 at 6:14 AM Stephen Mallette <[email protected]> > wrote: > > > No branch yet, but I think I will be sending the PR today. > > > > On Mon, Mar 15, 2021 at 9:33 PM Joshua Shinavier <[email protected]> > > wrote: > > > >> Is there a branch we can take a look at before the PR is ready? > >> > >> Josh > >> > >> On Fri, Mar 12, 2021 at 5:42 AM Stephen Mallette <[email protected]> > >> wrote: > >> > >> > I've been working on forming a pull request for this task. I don't > >> think IP > >> > Clearance is necessary as I originally did because the contribution is > >> > really just an ANTLR4 grammar file with some tests to validate things. > >> > Therefore, it's not a big body of independent code as I'd perhaps > >> initially > >> > envisioned. Compared to gremlint, this addition is pretty simple and > >> > straightforward. I've created this issue in JIRA with some additional > >> notes > >> > on what to expect in this initial body of work: > >> > > >> > https://issues.apache.org/jira/browse/TINKERPOP-2533 > >> > > >> > > >> > > >> > On Mon, Feb 8, 2021 at 10:06 AM Stephen Mallette < > [email protected]> > >> > wrote: > >> > > >> > > Just wanted to leave an update on this thread. It was nice to see > some > >> > > support for it. I've not had time to focus on the task itself so > sorry > >> > > there hasn't been much movement, but I hope to see it on track > soon. I > >> > > thought to update the thread after I came across yet another nice > >> usage > >> > for > >> > > it. I've long wanted to unify our test framework (i.e. deprecate the > >> JVM > >> > > process suite in favor of the GLV test suite). I was experimenting > >> with > >> > > what that might look like on Friday and hit a circular dependency > >> which > >> > > constantly trips things up where gremlin-test wants to depend on > >> > > gremlin-groovy (for ScriptEngine support) but gremlin-groovy depends > >> on > >> > > gremlin-test and tinkergraph with <test> scope already. I think the > >> > > introduction of gremlin-script would let gremlin-test build the > >> Traversal > >> > > object from a Gremlin string and thus avoid that circular > >> relationship. > >> > > > >> > > On Fri, Jan 8, 2021 at 2:43 AM pieter gmail < > [email protected]> > >> > > wrote: > >> > > > >> > >> +1 > >> > >> > >> > >> I have often thought the language specification should be a project > >> > >> separate from the implementations, and done in a formal but plain > >> > >> English format similar to OMG or IETF specifications. > >> > >> > >> > >> I suspect Sqlg's code base would have been fastly different if it > had > >> > >> evolved from a grammer instead of an api. > >> > >> > >> > >> Cheers > >> > >> Pieter > >> > >> > >> > >> On Thu, 2020-12-24 at 14:41 -0500, Stephen Mallette wrote: > >> > >> > As a project, over the years, we've often been asked the question > >> as > >> > >> > to why > >> > >> > Gremlin doesn't have an ANTLR style grammar. There have been > >> varying > >> > >> > answers over the years to explain the reasoning but in recent > years > >> > >> > I've > >> > >> > started to see where our dependence on Java for driving Gremlin > >> > >> > design has > >> > >> > not translated well as we have expanded Gremlin into other > >> > >> > programming > >> > >> > ecosystems. Using Java has often allowed idioms of that language > to > >> > >> > leak > >> > >> > into Gremlin itself which introduces friction when implemented > >> > >> > outside of > >> > >> > the JVM. I think that there is some advantage to designing > Gremlin > >> > >> > more > >> > >> > with just graphs/usage in mind and then determining how that > design > >> > >> > choice > >> > >> > looks in each programming language. > >> > >> > > >> > >> > I think that using an ANTLR grammar to drive that design work for > >> > >> > Gremlin > >> > >> > makes a lot of sense in this context. We would effectively have > >> > >> > something > >> > >> > like a gremlin-script which would become the new language > >> archetype. > >> > >> > New > >> > >> > steps, language changes, etc. would be discussed in its context > and > >> > >> > then > >> > >> > implemented in the grammar and later in each programming language > >> we > >> > >> > support in the style a developer would expect. An interesting > >> upside > >> > >> > of > >> > >> > this approach is that we can implement gremlin-script in the > >> > >> > ScriptEngine > >> > >> > and replace GremlinGroovyScriptEngine which would help us > >> strengthen > >> > >> > our > >> > >> > security story in Gremlin Server. Groovy processing would just > be a > >> > >> > fallback to Gremlin scripts that could not be processed by the > AST. > >> > >> > In fact > >> > >> > users who didn't need Groovy could simply not install it at all > and > >> > >> > thus > >> > >> > boast a much more secure system. > >> > >> > > >> > >> > I think that inclusion of a grammar in our project is an exciting > >> new > >> > >> > direction for us to take and will help in a variety of areas > beyond > >> > >> > those > >> > >> > I've already related. > >> > >> > > >> > >> > If we like this direction, Amazon Neptune already maintains such > a > >> > >> > grammar > >> > >> > and would be willing to contribute it to the project to live in > >> open > >> > >> > source. The contribution would go through the same IP Clearance > >> > >> > process > >> > >> > gremlint is going through since it was developed outside of > >> > >> > TinkerPop. I'd > >> > >> > be happy to guide that process through if we draw to consensus > >> here. > >> > >> > >> > >> > >> > >> > >> > > >> > > >
