Hi Stephen, Replies inline.
On Wed, Mar 24, 2021 at 5:18 AM Stephen Mallette <[email protected]> wrote: > [...] > I'd thought that it made sense that Groovy and the grammar stayed > fairly close to one another as a nice point of Groovy is that you can make > it do neat tricks when building DSLs but I don't think it necessarily needs > to. > I think those neat tricks ought to be possible with the generalized grammar as well, but it's all a little hand-wavy right now. If this is of interest, which it seems to be, I'll see if I can do a small proof of concept some time soon. > 1. Specify an abstract Gremlin grammar in a neutral language like YAML > > 2. Write some helper code for generating ANTLR grammars from the YAML > > 3. For each Gremlin language variant, write a smaller amount of code > > based on (2) to generate a language-specific ANTLR grammar > > > > I like how you present this idea because one of our big problems and big > assets are language variants and generating our way to ANTLR support in > each makes sense to me. I'd be curious what a YAML representation and the > related work might look like and how you think we might structure it. > I'm curious too, but it would probably look a lot like the other kinds of transformations we perform on schemas and data with YAML or JSON as the source. E.g. instead of an expression like this: traversalMethod_out: 'out' LPAREN stringLiteralList RPAREN ; you'd have something more like this: - name: out category: traversalMethod parameters: - type: list: label and we would transform the latter to the former in the case of Gremlin-Groovy, and to something slightly different in the case of other variants. > Are you suggesting I change gremlin-grammar to gremlin-language with that > work in mind? That's my suggestion, because gremlin-language sounds a little more inclusive of other abstract Gremlin language specifications in addition to the grammar. Not to get too far ahead of ourselves, but I think it should be possible to use the YAML-based specification I'm hinting at above for other things like serializing/deserializing Gremlin traversals, and traversal results in non-text formats like JSON, Thrift, Protobuf, Avro, YAML of course, etc. A cleaner solution to the "format zoo" was another pain area we discussed last year, where I think mappings can help. > I don't think it's a problem to do so. I guess > gremlin-language would eventually house (1), (2), and (3)? Right now, > though, gremlin-grammar also generates a Java parser from the one ANTLR > file I have there. Would the idea be that we'd organize this module to > generate parsers for each language we supported as well? or would that live > somewhere else? > Yeah, I think once we can generate the parser for Gremlin-Groovy (which should look almost identical to what you already have), it ought to be straightforward to generate parser for other languages. Btw. if I give this a go, and the solution looks promising, maybe I will make it the topic of a future Category Theory and Applications session. The first session on Dragon is tonight at 6pm PDT. Josh > > > > > > Btw. I will be giving a Category Theory and Applications presentation > > <https://www.meetup.com/Category-Theory/events/nnrhgsyccfbhc/> next week > > which will illustrate how something like the above might work. > > > > Josh > > > > > > > > On Tue, Mar 16, 2021 at 12:48 PM Stephen Mallette <[email protected]> > > wrote: > > > > > Here is the PR: https://github.com/apache/tinkerpop/pull/1408 > > > > > > On Tue, Mar 16, 2021 at 6:14 AM Stephen Mallette <[email protected] > > > > > wrote: > > > > > > > No branch yet, but I think I will be sending the PR today. > > > > > > > > On Mon, Mar 15, 2021 at 9:33 PM Joshua Shinavier <[email protected]> > > > > wrote: > > > > > > > >> Is there a branch we can take a look at before the PR is ready? > > > >> > > > >> Josh > > > >> > > > >> On Fri, Mar 12, 2021 at 5:42 AM Stephen Mallette < > > [email protected]> > > > >> wrote: > > > >> > > > >> > I've been working on forming a pull request for this task. I don't > > > >> think IP > > > >> > Clearance is necessary as I originally did because the > contribution > > is > > > >> > really just an ANTLR4 grammar file with some tests to validate > > things. > > > >> > Therefore, it's not a big body of independent code as I'd perhaps > > > >> initially > > > >> > envisioned. Compared to gremlint, this addition is pretty simple > and > > > >> > straightforward. I've created this issue in JIRA with some > > additional > > > >> notes > > > >> > on what to expect in this initial body of work: > > > >> > > > > >> > https://issues.apache.org/jira/browse/TINKERPOP-2533 > > > >> > > > > >> > > > > >> > > > > >> > On Mon, Feb 8, 2021 at 10:06 AM Stephen Mallette < > > > [email protected]> > > > >> > wrote: > > > >> > > > > >> > > Just wanted to leave an update on this thread. It was nice to > see > > > some > > > >> > > support for it. I've not had time to focus on the task itself so > > > sorry > > > >> > > there hasn't been much movement, but I hope to see it on track > > > soon. I > > > >> > > thought to update the thread after I came across yet another > nice > > > >> usage > > > >> > for > > > >> > > it. I've long wanted to unify our test framework (i.e. deprecate > > the > > > >> JVM > > > >> > > process suite in favor of the GLV test suite). I was > experimenting > > > >> with > > > >> > > what that might look like on Friday and hit a circular > dependency > > > >> which > > > >> > > constantly trips things up where gremlin-test wants to depend on > > > >> > > gremlin-groovy (for ScriptEngine support) but gremlin-groovy > > depends > > > >> on > > > >> > > gremlin-test and tinkergraph with <test> scope already. I think > > the > > > >> > > introduction of gremlin-script would let gremlin-test build the > > > >> Traversal > > > >> > > object from a Gremlin string and thus avoid that circular > > > >> relationship. > > > >> > > > > > >> > > On Fri, Jan 8, 2021 at 2:43 AM pieter gmail < > > > [email protected]> > > > >> > > wrote: > > > >> > > > > > >> > >> +1 > > > >> > >> > > > >> > >> I have often thought the language specification should be a > > project > > > >> > >> separate from the implementations, and done in a formal but > plain > > > >> > >> English format similar to OMG or IETF specifications. > > > >> > >> > > > >> > >> I suspect Sqlg's code base would have been fastly different if > it > > > had > > > >> > >> evolved from a grammer instead of an api. > > > >> > >> > > > >> > >> Cheers > > > >> > >> Pieter > > > >> > >> > > > >> > >> On Thu, 2020-12-24 at 14:41 -0500, Stephen Mallette wrote: > > > >> > >> > As a project, over the years, we've often been asked the > > question > > > >> as > > > >> > >> > to why > > > >> > >> > Gremlin doesn't have an ANTLR style grammar. There have been > > > >> varying > > > >> > >> > answers over the years to explain the reasoning but in recent > > > years > > > >> > >> > I've > > > >> > >> > started to see where our dependence on Java for driving > Gremlin > > > >> > >> > design has > > > >> > >> > not translated well as we have expanded Gremlin into other > > > >> > >> > programming > > > >> > >> > ecosystems. Using Java has often allowed idioms of that > > language > > > to > > > >> > >> > leak > > > >> > >> > into Gremlin itself which introduces friction when > implemented > > > >> > >> > outside of > > > >> > >> > the JVM. I think that there is some advantage to designing > > > Gremlin > > > >> > >> > more > > > >> > >> > with just graphs/usage in mind and then determining how that > > > design > > > >> > >> > choice > > > >> > >> > looks in each programming language. > > > >> > >> > > > > >> > >> > I think that using an ANTLR grammar to drive that design work > > for > > > >> > >> > Gremlin > > > >> > >> > makes a lot of sense in this context. We would effectively > have > > > >> > >> > something > > > >> > >> > like a gremlin-script which would become the new language > > > >> archetype. > > > >> > >> > New > > > >> > >> > steps, language changes, etc. would be discussed in its > context > > > and > > > >> > >> > then > > > >> > >> > implemented in the grammar and later in each programming > > language > > > >> we > > > >> > >> > support in the style a developer would expect. An interesting > > > >> upside > > > >> > >> > of > > > >> > >> > this approach is that we can implement gremlin-script in the > > > >> > >> > ScriptEngine > > > >> > >> > and replace GremlinGroovyScriptEngine which would help us > > > >> strengthen > > > >> > >> > our > > > >> > >> > security story in Gremlin Server. Groovy processing would > just > > > be a > > > >> > >> > fallback to Gremlin scripts that could not be processed by > the > > > AST. > > > >> > >> > In fact > > > >> > >> > users who didn't need Groovy could simply not install it at > all > > > and > > > >> > >> > thus > > > >> > >> > boast a much more secure system. > > > >> > >> > > > > >> > >> > I think that inclusion of a grammar in our project is an > > exciting > > > >> new > > > >> > >> > direction for us to take and will help in a variety of areas > > > beyond > > > >> > >> > those > > > >> > >> > I've already related. > > > >> > >> > > > > >> > >> > If we like this direction, Amazon Neptune already maintains > > such > > > a > > > >> > >> > grammar > > > >> > >> > and would be willing to contribute it to the project to live > in > > > >> open > > > >> > >> > source. The contribution would go through the same IP > Clearance > > > >> > >> > process > > > >> > >> > gremlint is going through since it was developed outside of > > > >> > >> > TinkerPop. I'd > > > >> > >> > be happy to guide that process through if we draw to > consensus > > > >> here. > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > > > > >> > > > > > > > > > >
