> I’m almost tempted to say we don’t need “core,” just “data”, “machine”, and “language”
I sense that core will be a bit of inevitability - every project has some shared code somewhere. plus i think serialization code (presumably GraphBinary) would end up in there and not in its own module. > Finally, note that our root directory doesn’t have a pom.xml. +1 On Tue, Mar 5, 2019 at 10:19 AM Marko Rodriguez <[email protected]> wrote: > Hi, > > Okay, so this is how it looks when built (from Java): > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Apache TinkerPop (Java) :: Core .................... SUCCESS [ > 6.609 s] > [INFO] Apache TinkerPop (Java) :: Machine ................. SUCCESS [ > 0.126 s] > [INFO] Apache TinkerPop (Java) :: Machine :: Pipes ........ SUCCESS [ > 0.257 s] > [INFO] Apache TinkerPop ................................... SUCCESS [ > 0.090 s] > [INFO] > ------------------------------------------------------------------------ > > I was looking through our TP4 article (https://zenodo.org/record/1476234 < > https://zenodo.org/record/1476234>) and noted that we were going to try > and do away with “gremlin-“ everywhere. In that spirit, I made it so we > have it as: > > + java > +— core > +— machine > +—— pipes > +—— beam (not there yet) > +—— akka (not there yet) > > **Sidenote**: I’m almost tempted to say we don’t need “core,” just “data”, > “machine”, and “language”, but we will figure it out as it develops. > > Finally, note that our root directory doesn’t have a pom.xml. The java/ > directory has the pom.xml with the group id being org.apache.tinkerpop. > python/ will have its Python specific build stuff. dotnet/ will have its C# > specific build stuff. etc. Thus, this is what it looks like when building > from my terminal: > > ~/software/tinkerpop/java> mvn clean install > > Marko. > > http://markorodriguez.com > > > > > On Mar 5, 2019, at 7:32 AM, Marko Rodriguez <[email protected]> > wrote: > > > > Yes, having the languages be the main directories is the way to go. Just > to stress — for instance, this means that the C# community (e.g. ComosDB) > can provide Apache a GremlinVM implementation and console. > > > > I’ll tweak the structure accordingly. > > > > Marko. > > > > > >> On Mar 5, 2019, at 6:21 AM, Stephen Mallette <[email protected]> > wrote: > >> > >> I think that we should make programming language/environment more of a > >> first class citizen. We've been saying that Gremlin is at first a > >> specification for TP4. If we have that specification, we then provide > >> implementations of that specification in various forms: > >> > >> + java > >> + dotnet > >> + python > >> + javascript > >> > >> The degree to which each of those languages supports the specification > may > >> differ based on demand and community participation. For our purposes, > >> obviously Java is the most full/complete implementation which would > means > >> that you drop your suggested structure under Java: > >> > >> + java > >> +-- gremlin-core > >> +-- gremlin-data > >> +-- gremlin-machines > >> > >> I think we've referred to Gremlin Console/Server and "Gremlin > Applications" > >> long enough to do: > >> > >> + java > >> +-- gremlin-core > >> +-- gremlin-data > >> +-- gremlin-machines > >> +-- gremlin-app > >> +---- gremlin-console > >> +---- gremlin-server > >> > >> gremlin-console would hopefully be a JShell - rather than groovysh - > when > >> we get around to that. I think gremlin-core could hold IO related stuff > for > >> now as well as any protocol level interfaces that may be required. > >> gremlin-server could just be our base implementation of those > interfaces. > >> not sure if that will stick or not - i guess we'd need to see how all > this > >> develops a bit further. > >> > >> as far as gremlin-variants goes, i think that groovy is a part of the > jvm > >> and it thus makes sense to keep that where it is and leaves open the > >> possibility of other jvm languages being under that umbrella/ecosystem: > >> > >> + java > >> +-- gremlin-core > >> +-- gremlin-data > >> +-- gremlin-machines > >> +-- gremlin-languages > >> +---- gremlin-variants > >> +------ gremlin-groovy > >> +---- gremlin-uniques > >> +-- gremlin-app > >> +---- gremlin-console > >> +---- gremlin-server > >> > >> Looking back at the top-level I imagine we'd have one global set of > docs as > >> we have now and right from the start write them so that they directly > >> address users coming from any programming language.I think we'd also > have a > >> global "test" that would contain the general language agnostic test kit > >> that validates our Gremlin specification, protocols, etc. > >> > >> + docs > >> + java > >> + dotnet > >> + python > >> + javascript > >> + test > >> > >> A fully developed Gremlin implementation might have their own specific > test > >> suites I suppose (we'd have to see how that develops) but that would put > >> gremlin-test back in our "java" tree. > >> > >> + docs > >> + java > >> +-- gremlin-core > >> +-- gremlin-data > >> +-- gremlin-machines > >> +-- gremlin-languages > >> +---- gremlin-variants > >> +------ gremlin-groovy > >> +---- gremlin-uniques > >> +-- gremlin-app > >> +---- gremlin-console > >> +---- gremlin-server > >> +-- gremlin-test > >> + dotnet > >> + python > >> + javascript > >> + test > >> > >> I think this approach also opens the door to allowing each top-level > >> language/ecosystem to build/deploy under its own preferred system rather > >> than forcing into maven. We've managed to get that to work for TP3, but > in > >> the end you still had to have your local environment setup to do the > build. > >> I suppose that there was some benefit to that process (like versioning) > but > >> it hasn't always been easy to maintain/develop. I think that we should > rely > >> more heavily on docker for cross-language testing instead of mvn clean > >> install. And docker tests work really nicely right now in Travis so that > >> pattern allows folks to make sure their changes work across the board > >> without having to have every single language environment setup. > >> > >> I think that this project structure creates a sense that all Gremlin is > >> equal which I think we have struggled to convince folks of completely in > >> TP3. I think it also lowers the barrier to potential committers outside > of > >> the java world who aren't familiar with the JVM and find the build > system > >> confusing. It also allows each programming language to grow on its own > >> using java as its model. > >> > >> > >> > >> > >> On Sun, Mar 3, 2019 at 1:29 PM Marko Rodriguez <[email protected]> > wrote: > >> > >>> Hello, > >>> > >>> The last two days I’ve been working to get TinkerPop4’s project > structure > >>> spec’d (see README): > >>> > >>> https://github.com/apache/tinkerpop/tree/tp4 > >>> > >>> There are three main sub-modules: > >>> > >>> gremlin-data: operators (“steps”) to process different data > >>> structures. > >>> gremlin-machines: implementations of the Gremlin traversal > machine. > >>> gremlin-languages: implementations of the Gremlin traversal > >>> language. > >>> > >>> Given that we have 3-degrees of freedom in TinkerPop4, I decided to > make > >>> use of sub-sub-…-modules in Maven so that our sub-projects are > organized > >>> better than what we have in TinkerPop3. > >>> > >>> With respects to the critiques on gremlin-users@, you will note that: > >>> > >>> * gremlin-graph will provide what we currently know to be Gremlin > >>> in TinkerPop3. > >>> * gremlin-pipes will provide what we currently know as > >>> Traversal/Step/etc. in TinkerPop3. > >>> > >>> gremlin-data: There is no need for us to add more gremlin-data > sub-modules > >>> until we feel comfortable with where this is all going. Moreover, I > think > >>> we stick with all our current concepts of traversal, traverser > regardless > >>> of “graph” application. Traversers traverse data structures. Thus, it > will > >>> still be called the Gremlin traversal machine and Gremlin traversal > >>> language. > >>> > >>> gremlin-machines: One of the major problems with TinkerPop3 is that we > >>> mixed the description of the computation with the computation itself > (i.e. > >>> Traversal). This caused various problems with Gremlin executing on > other > >>> engines such as Giraph, Spark, etc. (hence the introduction of > >>> TraversalMatrix). My goal is to separate these concepts such that it > will > >>> be much more natural and less error prone to have other execution > engines > >>> like RxJava, Flink, Akka, Spark, etc. executing Gremlin. > >>> > >>> gremlin-languages: I think TinkerPop3’s language work is pretty top > notch. > >>> Besides some refactoring of bytecode, I/O, and “traversal sources," > what we > >>> have in TinkerPop3, will map over to TinkerPop4 pretty naturally. > >>> > >>> *** Note that I haven’t figured out how gremlin-console and gremlin-io > >>> (server + serialization) will play out yet so I haven’t added them to > the > >>> README. > >>> > >>> My next steps are to get gremlin-graph and gremlin-pipes working with > the > >>> concepts from the TinkerPop4 and Stream Ring papers. > >>> > >>> Take care, > >>> Marko. > >>> > >>> http://markorodriguez.com > >>> > >>> > >>> > >>> > > > >
