I've renamed it to gremlin-language - I plan to merge this PR today. Thanks!
On Thu, Mar 25, 2021 at 11:27 AM Joshua Shinavier <[email protected]> wrote: > Hi Stephen, > > Replies inline. > > On Wed, Mar 24, 2021 at 5:18 AM Stephen Mallette <[email protected]> > wrote: > > > [...] > > I'd thought that it made sense that Groovy and the grammar stayed > > fairly close to one another as a nice point of Groovy is that you can > make > > it do neat tricks when building DSLs but I don't think it necessarily > needs > > to. > > > > I think those neat tricks ought to be possible with the generalized grammar > as well, but it's all a little hand-wavy right now. If this is of interest, > which it seems to be, I'll see if I can do a small proof of concept some > time soon. > > > > > 1. Specify an abstract Gremlin grammar in a neutral language like YAML > > > 2. Write some helper code for generating ANTLR grammars from the > YAML > > > 3. For each Gremlin language variant, write a smaller amount of code > > > based on (2) to generate a language-specific ANTLR grammar > > > > > > > I like how you present this idea because one of our big problems and big > > assets are language variants and generating our way to ANTLR support in > > each makes sense to me. I'd be curious what a YAML representation and the > > related work might look like and how you think we might structure it. > > > > > I'm curious too, but it would probably look a lot like the other kinds of > transformations we perform on schemas and data with YAML or JSON as the > source. E.g. instead of an expression like this: > > traversalMethod_out: > 'out' LPAREN stringLiteralList RPAREN > ; > > you'd have something more like this: > > - name: out > category: traversalMethod > parameters: > - type: > list: label > > and we would transform the latter to the former in the case of > Gremlin-Groovy, and to something slightly different in the case of other > variants. > > > > > > Are you suggesting I change gremlin-grammar to gremlin-language with that > > work in mind? > > > > That's my suggestion, because gremlin-language sounds a little more > inclusive of other abstract Gremlin language specifications in addition to > the grammar. Not to get too far ahead of ourselves, but I think it should > be possible to use the YAML-based specification I'm hinting at above for > other things like serializing/deserializing Gremlin traversals, and > traversal results in non-text formats like JSON, Thrift, Protobuf, Avro, > YAML of course, etc. A cleaner solution to the "format zoo" was another > pain area we discussed last year, where I think mappings can help. > > > > > I don't think it's a problem to do so. I guess > > gremlin-language would eventually house (1), (2), and (3)? Right now, > > though, gremlin-grammar also generates a Java parser from the one ANTLR > > file I have there. Would the idea be that we'd organize this module to > > generate parsers for each language we supported as well? or would that > live > > somewhere else? > > > > Yeah, I think once we can generate the parser for Gremlin-Groovy (which > should look almost identical to what you already have), it ought to be > straightforward to generate parser for other languages. Btw. if I give this > a go, and the solution looks promising, maybe I will make it the topic of a > future Category Theory and Applications session. The first session on > Dragon is tonight at 6pm PDT. > > Josh > > > > > > > > > > > > > > Btw. I will be giving a Category Theory and Applications presentation > > > <https://www.meetup.com/Category-Theory/events/nnrhgsyccfbhc/> next > week > > > which will illustrate how something like the above might work. > > > > > > Josh > > > > > > > > > > > > On Tue, Mar 16, 2021 at 12:48 PM Stephen Mallette < > [email protected]> > > > wrote: > > > > > > > Here is the PR: https://github.com/apache/tinkerpop/pull/1408 > > > > > > > > On Tue, Mar 16, 2021 at 6:14 AM Stephen Mallette < > [email protected] > > > > > > > wrote: > > > > > > > > > No branch yet, but I think I will be sending the PR today. > > > > > > > > > > On Mon, Mar 15, 2021 at 9:33 PM Joshua Shinavier < > [email protected]> > > > > > wrote: > > > > > > > > > >> Is there a branch we can take a look at before the PR is ready? > > > > >> > > > > >> Josh > > > > >> > > > > >> On Fri, Mar 12, 2021 at 5:42 AM Stephen Mallette < > > > [email protected]> > > > > >> wrote: > > > > >> > > > > >> > I've been working on forming a pull request for this task. I > don't > > > > >> think IP > > > > >> > Clearance is necessary as I originally did because the > > contribution > > > is > > > > >> > really just an ANTLR4 grammar file with some tests to validate > > > things. > > > > >> > Therefore, it's not a big body of independent code as I'd > perhaps > > > > >> initially > > > > >> > envisioned. Compared to gremlint, this addition is pretty simple > > and > > > > >> > straightforward. I've created this issue in JIRA with some > > > additional > > > > >> notes > > > > >> > on what to expect in this initial body of work: > > > > >> > > > > > >> > https://issues.apache.org/jira/browse/TINKERPOP-2533 > > > > >> > > > > > >> > > > > > >> > > > > > >> > On Mon, Feb 8, 2021 at 10:06 AM Stephen Mallette < > > > > [email protected]> > > > > >> > wrote: > > > > >> > > > > > >> > > Just wanted to leave an update on this thread. It was nice to > > see > > > > some > > > > >> > > support for it. I've not had time to focus on the task itself > so > > > > sorry > > > > >> > > there hasn't been much movement, but I hope to see it on track > > > > soon. I > > > > >> > > thought to update the thread after I came across yet another > > nice > > > > >> usage > > > > >> > for > > > > >> > > it. I've long wanted to unify our test framework (i.e. > deprecate > > > the > > > > >> JVM > > > > >> > > process suite in favor of the GLV test suite). I was > > experimenting > > > > >> with > > > > >> > > what that might look like on Friday and hit a circular > > dependency > > > > >> which > > > > >> > > constantly trips things up where gremlin-test wants to depend > on > > > > >> > > gremlin-groovy (for ScriptEngine support) but gremlin-groovy > > > depends > > > > >> on > > > > >> > > gremlin-test and tinkergraph with <test> scope already. I > think > > > the > > > > >> > > introduction of gremlin-script would let gremlin-test build > the > > > > >> Traversal > > > > >> > > object from a Gremlin string and thus avoid that circular > > > > >> relationship. > > > > >> > > > > > > >> > > On Fri, Jan 8, 2021 at 2:43 AM pieter gmail < > > > > [email protected]> > > > > >> > > wrote: > > > > >> > > > > > > >> > >> +1 > > > > >> > >> > > > > >> > >> I have often thought the language specification should be a > > > project > > > > >> > >> separate from the implementations, and done in a formal but > > plain > > > > >> > >> English format similar to OMG or IETF specifications. > > > > >> > >> > > > > >> > >> I suspect Sqlg's code base would have been fastly different > if > > it > > > > had > > > > >> > >> evolved from a grammer instead of an api. > > > > >> > >> > > > > >> > >> Cheers > > > > >> > >> Pieter > > > > >> > >> > > > > >> > >> On Thu, 2020-12-24 at 14:41 -0500, Stephen Mallette wrote: > > > > >> > >> > As a project, over the years, we've often been asked the > > > question > > > > >> as > > > > >> > >> > to why > > > > >> > >> > Gremlin doesn't have an ANTLR style grammar. There have > been > > > > >> varying > > > > >> > >> > answers over the years to explain the reasoning but in > recent > > > > years > > > > >> > >> > I've > > > > >> > >> > started to see where our dependence on Java for driving > > Gremlin > > > > >> > >> > design has > > > > >> > >> > not translated well as we have expanded Gremlin into other > > > > >> > >> > programming > > > > >> > >> > ecosystems. Using Java has often allowed idioms of that > > > language > > > > to > > > > >> > >> > leak > > > > >> > >> > into Gremlin itself which introduces friction when > > implemented > > > > >> > >> > outside of > > > > >> > >> > the JVM. I think that there is some advantage to designing > > > > Gremlin > > > > >> > >> > more > > > > >> > >> > with just graphs/usage in mind and then determining how > that > > > > design > > > > >> > >> > choice > > > > >> > >> > looks in each programming language. > > > > >> > >> > > > > > >> > >> > I think that using an ANTLR grammar to drive that design > work > > > for > > > > >> > >> > Gremlin > > > > >> > >> > makes a lot of sense in this context. We would effectively > > have > > > > >> > >> > something > > > > >> > >> > like a gremlin-script which would become the new language > > > > >> archetype. > > > > >> > >> > New > > > > >> > >> > steps, language changes, etc. would be discussed in its > > context > > > > and > > > > >> > >> > then > > > > >> > >> > implemented in the grammar and later in each programming > > > language > > > > >> we > > > > >> > >> > support in the style a developer would expect. An > interesting > > > > >> upside > > > > >> > >> > of > > > > >> > >> > this approach is that we can implement gremlin-script in > the > > > > >> > >> > ScriptEngine > > > > >> > >> > and replace GremlinGroovyScriptEngine which would help us > > > > >> strengthen > > > > >> > >> > our > > > > >> > >> > security story in Gremlin Server. Groovy processing would > > just > > > > be a > > > > >> > >> > fallback to Gremlin scripts that could not be processed by > > the > > > > AST. > > > > >> > >> > In fact > > > > >> > >> > users who didn't need Groovy could simply not install it at > > all > > > > and > > > > >> > >> > thus > > > > >> > >> > boast a much more secure system. > > > > >> > >> > > > > > >> > >> > I think that inclusion of a grammar in our project is an > > > exciting > > > > >> new > > > > >> > >> > direction for us to take and will help in a variety of > areas > > > > beyond > > > > >> > >> > those > > > > >> > >> > I've already related. > > > > >> > >> > > > > > >> > >> > If we like this direction, Amazon Neptune already maintains > > > such > > > > a > > > > >> > >> > grammar > > > > >> > >> > and would be willing to contribute it to the project to > live > > in > > > > >> open > > > > >> > >> > source. The contribution would go through the same IP > > Clearance > > > > >> > >> > process > > > > >> > >> > gremlint is going through since it was developed outside of > > > > >> > >> > TinkerPop. I'd > > > > >> > >> > be happy to guide that process through if we draw to > > consensus > > > > >> here. > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > > > > > >
