I think that we should make programming language/environment more of a first class citizen. We've been saying that Gremlin is at first a specification for TP4. If we have that specification, we then provide implementations of that specification in various forms:
+ java + dotnet + python + javascript The degree to which each of those languages supports the specification may differ based on demand and community participation. For our purposes, obviously Java is the most full/complete implementation which would means that you drop your suggested structure under Java: + java +-- gremlin-core +-- gremlin-data +-- gremlin-machines I think we've referred to Gremlin Console/Server and "Gremlin Applications" long enough to do: + java +-- gremlin-core +-- gremlin-data +-- gremlin-machines +-- gremlin-app +---- gremlin-console +---- gremlin-server gremlin-console would hopefully be a JShell - rather than groovysh - when we get around to that. I think gremlin-core could hold IO related stuff for now as well as any protocol level interfaces that may be required. gremlin-server could just be our base implementation of those interfaces. not sure if that will stick or not - i guess we'd need to see how all this develops a bit further. as far as gremlin-variants goes, i think that groovy is a part of the jvm and it thus makes sense to keep that where it is and leaves open the possibility of other jvm languages being under that umbrella/ecosystem: + java +-- gremlin-core +-- gremlin-data +-- gremlin-machines +-- gremlin-languages +---- gremlin-variants +------ gremlin-groovy +---- gremlin-uniques +-- gremlin-app +---- gremlin-console +---- gremlin-server Looking back at the top-level I imagine we'd have one global set of docs as we have now and right from the start write them so that they directly address users coming from any programming language.I think we'd also have a global "test" that would contain the general language agnostic test kit that validates our Gremlin specification, protocols, etc. + docs + java + dotnet + python + javascript + test A fully developed Gremlin implementation might have their own specific test suites I suppose (we'd have to see how that develops) but that would put gremlin-test back in our "java" tree. + docs + java +-- gremlin-core +-- gremlin-data +-- gremlin-machines +-- gremlin-languages +---- gremlin-variants +------ gremlin-groovy +---- gremlin-uniques +-- gremlin-app +---- gremlin-console +---- gremlin-server +-- gremlin-test + dotnet + python + javascript + test I think this approach also opens the door to allowing each top-level language/ecosystem to build/deploy under its own preferred system rather than forcing into maven. We've managed to get that to work for TP3, but in the end you still had to have your local environment setup to do the build. I suppose that there was some benefit to that process (like versioning) but it hasn't always been easy to maintain/develop. I think that we should rely more heavily on docker for cross-language testing instead of mvn clean install. And docker tests work really nicely right now in Travis so that pattern allows folks to make sure their changes work across the board without having to have every single language environment setup. I think that this project structure creates a sense that all Gremlin is equal which I think we have struggled to convince folks of completely in TP3. I think it also lowers the barrier to potential committers outside of the java world who aren't familiar with the JVM and find the build system confusing. It also allows each programming language to grow on its own using java as its model. On Sun, Mar 3, 2019 at 1:29 PM Marko Rodriguez <[email protected]> wrote: > Hello, > > The last two days I’ve been working to get TinkerPop4’s project structure > spec’d (see README): > > https://github.com/apache/tinkerpop/tree/tp4 > > There are three main sub-modules: > > gremlin-data: operators (“steps”) to process different data > structures. > gremlin-machines: implementations of the Gremlin traversal machine. > gremlin-languages: implementations of the Gremlin traversal > language. > > Given that we have 3-degrees of freedom in TinkerPop4, I decided to make > use of sub-sub-…-modules in Maven so that our sub-projects are organized > better than what we have in TinkerPop3. > > With respects to the critiques on gremlin-users@, you will note that: > > * gremlin-graph will provide what we currently know to be Gremlin > in TinkerPop3. > * gremlin-pipes will provide what we currently know as > Traversal/Step/etc. in TinkerPop3. > > gremlin-data: There is no need for us to add more gremlin-data sub-modules > until we feel comfortable with where this is all going. Moreover, I think > we stick with all our current concepts of traversal, traverser regardless > of “graph” application. Traversers traverse data structures. Thus, it will > still be called the Gremlin traversal machine and Gremlin traversal > language. > > gremlin-machines: One of the major problems with TinkerPop3 is that we > mixed the description of the computation with the computation itself (i.e. > Traversal). This caused various problems with Gremlin executing on other > engines such as Giraph, Spark, etc. (hence the introduction of > TraversalMatrix). My goal is to separate these concepts such that it will > be much more natural and less error prone to have other execution engines > like RxJava, Flink, Akka, Spark, etc. executing Gremlin. > > gremlin-languages: I think TinkerPop3’s language work is pretty top notch. > Besides some refactoring of bytecode, I/O, and “traversal sources," what we > have in TinkerPop3, will map over to TinkerPop4 pretty naturally. > > *** Note that I haven’t figured out how gremlin-console and gremlin-io > (server + serialization) will play out yet so I haven’t added them to the > README. > > My next steps are to get gremlin-graph and gremlin-pipes working with the > concepts from the TinkerPop4 and Stream Ring papers. > > Take care, > Marko. > > http://markorodriguez.com > > > >
