Hi Stephen, I cleaned up the code a bit and then, I tried testing the code merge yesterday and I ran into some issues for 3.3.1-SNAPSHOT version.
- I forked apache/tinkerpop repository to my local account and loaded the same using an IDE (as maven project). This immediately threw errors in the native repositories such as gremlin-core, stating that it is not able to find org.apache.tinkerpop.shaded.kryo.Kryo. - When I try building the code with 3.3.0 api it works perfectly without any error, however for 3.3.1-SNAPSHOT version it is not able to find various files and throws errors in the core modules of tinkerpop. Thus, I cannot test my module (sparql-gremlin) with the 3.3.1-SNAPSHOT version. - Also, when I did a man clean install on 3.3.1-SNAPSHOT, it did get build success but a majority of the test cases failed. Not sure if this is worth mentioning. What do you suggest? How do I fix this? Also, is it okay if I use the 3.3.0 api version for my module or it is absolutely necessary that we have to only use the 3.3.1-SNAPSHOT api version? Thanks in advance! On 2017-12-12 12:58, Stephen Mallette <[email protected]> wrote: > yes - please post questions here. i don't think you need to know much about > TinkerPop internal structure. I'd think that sparql-gremlin is expected to > be included in the root of the TinkerPop source as a sub-module to the > top-level pom. That just means some minor changes to your pom.xml to get it > to build along with everything else. See other projects for examples: > > https://github.com/apache/tinkerpop/blob/f5687ee4497bfbaef4ae89233e4c29f07001ed2c/gremlin-core/pom.xml#L20-L24 > > You can drop all of this because it is already defined in the root pom.xml: > > https://github.com/LITMUS-Benchmark-Suite/sparql-to-gremlin/blob/master/pom.xml#L30-L70 > > Looking at the rest of your pom.xml now, I'm not sure I understand > everything your <build> section is doing and if it's all necessary: The > root pom.xml should handle the most common build/deploy options and they > will be thus inherited to your sub-module pom which is why, for example, > the gremlin-core pom is pretty simple for the <build> section: > > https://github.com/apache/tinkerpop/blob/f5687ee4497bfbaef4ae89233e4c29f07001ed2c/gremlin-core/pom.xml#L119-L151 > > If there's anything you're sure can be removed from the sparql-gremlin > pom.xml <build> section based on how the TinkerPop root pom.xml is setup, > the please feel free to cleanup as much as possible there. > > As for the general project structure of sparql-gremlin, I don't fully > understand how it is arranged. There's > > /Queries > /doc > /docs/images > /output > /src > > and all of that is repeated inside of the /bin directory. something seems > amiss there. maybe once that's cleared up a bit I can think more clearly on > what additional changes you might need. > > Another important thing to consider....documentation. Right now, it's all > in the README. I think we will want a new section to the Reference > Documentation, probably appearing after Gremlin Variants: > > http://tinkerpop.apache.org/docs/current/reference/#gremlin-variants > > Perhaps that could be named "Query Languages" where sparql-gremlin would be > the first sub-section. That would set up for some future where we also had > sql-gremlin. And perhaps a section for cypher-gremlin which could point to > Neo4j's work in this area. You can find reference docs for TinkerPop here: > > https://github.com/apache/tinkerpop/tree/f5687ee4497bfbaef4ae89233e4c29f07001ed2c/docs/src/reference > > The easiest way to generate docs is with docker via: > > docker/build.sh -d > > without that you need hadoop running with appropriate configurations: > > http://tinkerpop.apache.org/docs/current/dev/developer/#documentation-environment > > Well, hope that gives you a few things to work on and think about for your > first round of changes in your fork. Looking forward to seeing how this PR > shapes up! > > > On Tue, Dec 12, 2017 at 3:40 AM, Harsh Thakkar <[email protected]> wrote: > > > Hi Stephen, > > > > Very well then, we will start the migration from today. Also we will > > submit the signed iclas' today. > > > > If we have some questions regarding building the code properly, can we > > feel free to ask them here? I assume we might need some guidance on how to > > getting things plugged in correctly. We both are not much aware of the > > internal structure of TinkerPop, so that is why. > > > > Also, if there is any specific documentation to help us with this, please > > lend a pointer. > > > > Many thanks! > > > > On 2017-12-11 21:33, Stephen Mallette <[email protected]> wrote: > > > As there hasn't been any other opinions, It seems we have a lazy > > consensus > > > to accept sparql-gremlin into TinkerPop's code base. Cool! > > > > > > Harsh, I think you and Dharmen should proceed with the steps I listed > > > above. Once you have the code integrated and building properly in your > > > fork, please reply back and point us to it and we can start with some > > > coarse grained review of what you have. > > > > > > Thanks, > > > > > > Stephen > > > > > > On Fri, Dec 8, 2017 at 8:44 AM, [email protected] <[email protected]> > > wrote: > > > > > > > Hi Stephen, > > > > > > > > Thanks for the insight on the process of this integration. I will > > reply to > > > > your comments in the same manner. > > > > > > > > 1. Yes, I will do the fork and migrate the code to the Tinkerpop > > > > repository, after cleaning the code a bit. We also need to prepare a > > > > detailed doc (how-to) for the plugin. This can also be done in > > parallel, > > > > depending upon the urgency. > > > > > > > > 2. Yes, we both are contributing to the v0.2 of the sparql-gremlin > > plugin. > > > > We will both submit the ICLAs. > > > > > > > > Yes, we (both) will continue to provide support for the 0.2 plugin and > > > > also extend it in the future (trying to cover SPARQL 1.1 specification, > > > > also fix the OPTIONAL fix in the current version). > > > > > > > > Looking forward to hear more on this from the devs :) > > > > > > > > Cheers! > > > > > > > > On 2017-12-08 13:41, Stephen Mallette <[email protected]> wrote: > > > > > I agree with Marko's thoughts, both on this topic of including > > > > > sparql-gremlin as well as the wider topic of what should be included > > in > > > > > TinkerPop code base more generally. Providing a path for rdf/sparql > > folks > > > > > to get into the TinkerPop world seems like a smart direction. > > > > > > > > > > Now, assuming that we have consensus to include sparql-gremlin in the > > > > > TinkerPop code base, the process will look something like this: > > > > > > > > > > 1. I think that Harsh should fork the TinkerPop repository and > > migrate > > > > > sparql-gremlin into its structure. From there we will provide > > > > > feedback/review to get that fork into best shape possible prior to > > his > > > > > submitting a pull request. I think we can handle initial feedback > > through > > > > > the dev list in a separate thread. > > > > > > > > > > 2. In parallel to the above item, it appears as though there are two > > > > > contributors on sparql-gremlin: > > > > > > > > > > https://github.com/LITMUS-Benchmark-Suite/sparql-to- > > > > gremlin/graphs/contributors > > > > > > > > > > Both contributors, Harsh and Dharmen, should submit ICLAs: > > > > > > > > > > http://apache.org/licenses/icla.pdf > > > > > > > > > > and send them to [email protected]. > > > > > > > > > > 3. Once ICLAs are confirmed by secretary, Harsh can submit a pull > > request > > > > > from his fork where it can under go final review. > > > > > > > > > > Does that sound sensible to everyone? > > > > > > > > > > btw, Harsh, it sounds as though you intend to continue development on > > > > > sparql-gremlin after it is part of the TinkerPop repository...does > > > > Dharmen > > > > > intend to do the same? > > > > > > > > > > On Fri, Dec 8, 2017 at 6:54 AM, Stephen Mallette < > > [email protected]> > > > > > wrote: > > > > > > > > > > > linking marko's reply from the user list: > > > > > > > > > > > > https://groups.google.com/d/msg/gremlin-users/zK9jj7bWvrQ/ > > nE1VvhmeAAAJ > > > > > > > > > > > > On Thu, Dec 7, 2017 at 1:52 PM, [email protected] < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > >> Hello, dear Gremlin people! > > > > > >> > > > > > >> Apologies for raising this topic a bit late. I planned to start > > this > > > > > >> thread quite earlier but wasnât able to due to some reasons. > > > > > >> > > > > > >> ======= short ============================== > > > > > >> ============================== > > > > > >> I seek your guidance and also help for polishing and integrating > > the > > > > > >> sparql-gremlin 0.2 (https://github.com/LITMUS-Ben > > > > > >> chmark-Suite/sparql-to-gremlin) plugin in the apache tinkerpop > > code > > > > > >> base, succeeding its predecessor developed by Daniel Kupitz ( > > > > > >> https://github.com/dkuppitz/sparql-gremlin). The new plugin > > offers > > > > > >> support for a wide range of SPARQL queries from the SPARQL 1.0 > > > > features. > > > > > >> > > > > > >> > > > > > >> ============ long ============================== > > > > > >> =============================== > > > > > >> > > > > > >> I am a Ph.D. student at the University of Bonn and work at the > > > > > >> intersection of semantic web and graph databases. My thesis is > > > > focused on > > > > > >> bridging the gap between these two domains by enabling support for > > > > SPARQL > > > > > >> querying of Property Graph databases. Thus, working on the > > > > SPARQL-Gremlin > > > > > >> interoperability was an obvious idea given the wide popularity of > > > > Gremlin > > > > > >> amongst the Graph DB vendors. > > > > > >> > > > > > >> The sparql-gremlin 0.1 (link - https://github.com/dkuppitz/ > > > > sparql-gremlin) > > > > > >> plugin was developed by Daniel Kupitz, which we have extended to > > > > support > > > > > >> various features of the SPARQL 1.0 specification and have tested > > using > > > > > >> various synthetic datasets (such as Northwind dataset and the > > Berlin > > > > sprawl > > > > > >> benchmark [BSBM] dataset) and a wide range of SPARQL queries. > > > > > >> > > > > > >> The extended version of the plugin (sparql-gremlin 0.2, link - > > > > > >> https://github.com/LITMUS-Benchmark-Suite/sparql-to-gremlin) > > > > supports a > > > > > >> variety of query modifiers (group-by, order-by, counts, etc) and > > > > complex > > > > > >> query features such as union, aggregation, etc. It does not > > currently > > > > > >> support SPARQL optional queries though. It needs a minor fix. > > > > > >> > > > > > >> I wish to integrate this updated version to the apache tinkerpop > > > > codebase > > > > > >> and wish to see it roll out as a functional plugin (like the old > > one, > > > > > >> replacing it with the updated version) in the next version of > > tinker > > > > pop > > > > > >> (or even before, however it works out). > > > > > >> > > > > > >> ============================================================ > > > > > >> ================== > > > > > >> > > > > > >> I am not much aware of how to do it and what steps I need to > > follow, > > > > so I > > > > > >> seek input from you all and have started this thread (as > > suggested by > > > > > >> Stephen Mallette) and already discussed with Marko Rodriguez > > during > > > > Graph > > > > > >> Day SF 2017 and in other informal communications. > > > > > >> > > > > > >> Please guide me through the same and let me know what all I will > > need > > > > to > > > > > >> do and/or what you will need to get this done. I am happy to > > > > collaborate > > > > > >> and be a part of this awesome project :) > > > > > >> > > > > > >> Cheers, > > > > > >> Harsh > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > >
