Please see below: On Mon, Mar 22, 2021 at 10:25 AM Joshua Shinavier <[email protected]> wrote:
> Awesome. This definitely fills a major gap, and as Florian said, it is > great that it is already improving the documentation. > > High level-question: do you have any thoughts about Gremlin syntax > validation *in general*, i.e. for language variants other than Groovy? It > would be interesting to explore the following: > The grammar as it is tends to be roughly formed around what can be expressed in Groovy as it is meant to be a safe way to parse Gremlin scripts which up until now have been Groovy-based. We won't talk about gremlin-jython :) So, tactically speaking, there's like this immediate need to "fix groovy" but then there is this bigger picture in deciding what direction the grammar goes from here. In thinking this contribution through, I'd thought that it made sense that Groovy and the grammar stayed fairly close to one another as a nice point of Groovy is that you can make it do neat tricks when building DSLs but I don't think it necessarily needs to. > 1. Specify an abstract Gremlin grammar in a neutral language like YAML > 2. Write some helper code for generating ANTLR grammars from the YAML > 3. For each Gremlin language variant, write a smaller amount of code > based on (2) to generate a language-specific ANTLR grammar > I like how you present this idea because one of our big problems and big assets are language variants and generating our way to ANTLR support in each makes sense to me. I'd be curious what a YAML representation and the related work might look like and how you think we might structure it. > > Based on my experience with Dragon, I have pretty good handle on what the > YAML (1) would need to look like, and how to write language-neutral helper > code (2). What would take a little more investigation is to what extent we > could do (3) on the basis of an understanding of the target language alone. > E.g. if a new step or signature is added to Gremlin, will it be enough to > add a specification of the step to the abstract grammar, or would we need > to special-case the step for each language variant? I suspect we wouldn't > have to do *too* much special-casing, but that's to be determined. > > I might suggest calling the module "gremlin-language" if we were to > undertake the above. That would also allow other schemas to be provided > which would help us generate the structure API into different > implementation languages in a consistent way, as we discussed last year > <https://www.slideshare.net/joshsh/tinkerpop-2020>. > Are you suggesting I change gremlin-grammar to gremlin-language with that work in mind? I don't think it's a problem to do so. I guess gremlin-language would eventually house (1), (2), and (3)? Right now, though, gremlin-grammar also generates a Java parser from the one ANTLR file I have there. Would the idea be that we'd organize this module to generate parsers for each language we supported as well? or would that live somewhere else? > > Btw. I will be giving a Category Theory and Applications presentation > <https://www.meetup.com/Category-Theory/events/nnrhgsyccfbhc/> next week > which will illustrate how something like the above might work. > > Josh > > > > On Tue, Mar 16, 2021 at 12:48 PM Stephen Mallette <[email protected]> > wrote: > > > Here is the PR: https://github.com/apache/tinkerpop/pull/1408 > > > > On Tue, Mar 16, 2021 at 6:14 AM Stephen Mallette <[email protected]> > > wrote: > > > > > No branch yet, but I think I will be sending the PR today. > > > > > > On Mon, Mar 15, 2021 at 9:33 PM Joshua Shinavier <[email protected]> > > > wrote: > > > > > >> Is there a branch we can take a look at before the PR is ready? > > >> > > >> Josh > > >> > > >> On Fri, Mar 12, 2021 at 5:42 AM Stephen Mallette < > [email protected]> > > >> wrote: > > >> > > >> > I've been working on forming a pull request for this task. I don't > > >> think IP > > >> > Clearance is necessary as I originally did because the contribution > is > > >> > really just an ANTLR4 grammar file with some tests to validate > things. > > >> > Therefore, it's not a big body of independent code as I'd perhaps > > >> initially > > >> > envisioned. Compared to gremlint, this addition is pretty simple and > > >> > straightforward. I've created this issue in JIRA with some > additional > > >> notes > > >> > on what to expect in this initial body of work: > > >> > > > >> > https://issues.apache.org/jira/browse/TINKERPOP-2533 > > >> > > > >> > > > >> > > > >> > On Mon, Feb 8, 2021 at 10:06 AM Stephen Mallette < > > [email protected]> > > >> > wrote: > > >> > > > >> > > Just wanted to leave an update on this thread. It was nice to see > > some > > >> > > support for it. I've not had time to focus on the task itself so > > sorry > > >> > > there hasn't been much movement, but I hope to see it on track > > soon. I > > >> > > thought to update the thread after I came across yet another nice > > >> usage > > >> > for > > >> > > it. I've long wanted to unify our test framework (i.e. deprecate > the > > >> JVM > > >> > > process suite in favor of the GLV test suite). I was experimenting > > >> with > > >> > > what that might look like on Friday and hit a circular dependency > > >> which > > >> > > constantly trips things up where gremlin-test wants to depend on > > >> > > gremlin-groovy (for ScriptEngine support) but gremlin-groovy > depends > > >> on > > >> > > gremlin-test and tinkergraph with <test> scope already. I think > the > > >> > > introduction of gremlin-script would let gremlin-test build the > > >> Traversal > > >> > > object from a Gremlin string and thus avoid that circular > > >> relationship. > > >> > > > > >> > > On Fri, Jan 8, 2021 at 2:43 AM pieter gmail < > > [email protected]> > > >> > > wrote: > > >> > > > > >> > >> +1 > > >> > >> > > >> > >> I have often thought the language specification should be a > project > > >> > >> separate from the implementations, and done in a formal but plain > > >> > >> English format similar to OMG or IETF specifications. > > >> > >> > > >> > >> I suspect Sqlg's code base would have been fastly different if it > > had > > >> > >> evolved from a grammer instead of an api. > > >> > >> > > >> > >> Cheers > > >> > >> Pieter > > >> > >> > > >> > >> On Thu, 2020-12-24 at 14:41 -0500, Stephen Mallette wrote: > > >> > >> > As a project, over the years, we've often been asked the > question > > >> as > > >> > >> > to why > > >> > >> > Gremlin doesn't have an ANTLR style grammar. There have been > > >> varying > > >> > >> > answers over the years to explain the reasoning but in recent > > years > > >> > >> > I've > > >> > >> > started to see where our dependence on Java for driving Gremlin > > >> > >> > design has > > >> > >> > not translated well as we have expanded Gremlin into other > > >> > >> > programming > > >> > >> > ecosystems. Using Java has often allowed idioms of that > language > > to > > >> > >> > leak > > >> > >> > into Gremlin itself which introduces friction when implemented > > >> > >> > outside of > > >> > >> > the JVM. I think that there is some advantage to designing > > Gremlin > > >> > >> > more > > >> > >> > with just graphs/usage in mind and then determining how that > > design > > >> > >> > choice > > >> > >> > looks in each programming language. > > >> > >> > > > >> > >> > I think that using an ANTLR grammar to drive that design work > for > > >> > >> > Gremlin > > >> > >> > makes a lot of sense in this context. We would effectively have > > >> > >> > something > > >> > >> > like a gremlin-script which would become the new language > > >> archetype. > > >> > >> > New > > >> > >> > steps, language changes, etc. would be discussed in its context > > and > > >> > >> > then > > >> > >> > implemented in the grammar and later in each programming > language > > >> we > > >> > >> > support in the style a developer would expect. An interesting > > >> upside > > >> > >> > of > > >> > >> > this approach is that we can implement gremlin-script in the > > >> > >> > ScriptEngine > > >> > >> > and replace GremlinGroovyScriptEngine which would help us > > >> strengthen > > >> > >> > our > > >> > >> > security story in Gremlin Server. Groovy processing would just > > be a > > >> > >> > fallback to Gremlin scripts that could not be processed by the > > AST. > > >> > >> > In fact > > >> > >> > users who didn't need Groovy could simply not install it at all > > and > > >> > >> > thus > > >> > >> > boast a much more secure system. > > >> > >> > > > >> > >> > I think that inclusion of a grammar in our project is an > exciting > > >> new > > >> > >> > direction for us to take and will help in a variety of areas > > beyond > > >> > >> > those > > >> > >> > I've already related. > > >> > >> > > > >> > >> > If we like this direction, Amazon Neptune already maintains > such > > a > > >> > >> > grammar > > >> > >> > and would be willing to contribute it to the project to live in > > >> open > > >> > >> > source. The contribution would go through the same IP Clearance > > >> > >> > process > > >> > >> > gremlint is going through since it was developed outside of > > >> > >> > TinkerPop. I'd > > >> > >> > be happy to guide that process through if we draw to consensus > > >> here. > > >> > >> > > >> > >> > > >> > >> > > >> > > > >> > > > > > >
