Hi Stephen, Thanks for the pointers. We have good news. Turn out that the error was mostly because of the IDE environment and some other shady stuff going wrong. We finally managed to merge the sparql-gremlin work into the tinkerpop code base. I merged forked tinkerpop repository can be found here - https://github.com/harsh9t/tinkerpop
Please have a look at it and let us know what is to be done next. Meanwhile, we are having a look at the documentation on how to generate and other specifics. If we have questions, we will get back to you. I haven't done this before so expect some :D Cheers! On 2017-12-13 13:03, Stephen Mallette <[email protected]> wrote: > I suggest that you read through the dev docs a bit. There's lots of little > odds/ends there about how to develop on the TinkerPop code base. For > example, for intellij issues, please have a look at this: > > http://tinkerpop.apache.org/docs/current/dev/developer/#_ide_setup_with_intellij > > > - Also, when I did a man clean install on 3.3.1-SNAPSHOT, it did get > build success but a majority of the test cases failed. Not sure if this is > worth mentioning. > > there should be no failures on master and 3.3.1-SNAPSHOT. hard to say what > is wrong without some error logs > > > Also, is it okay if I use the 3.3.0 api version for my module or it is > absolutely necessary that we have to only use the 3.3.1-SNAPSHOT api > version? > > you should be on 3.3.1-SNAPSHOT for your API version now that you've > integrated sparql-gremlin into the tinkerpop code base. ultimately, it will > all release together as part of a single package, so it should all be on > the same version. > > On Wed, Dec 13, 2017 at 6:43 AM, Harsh Thakkar <[email protected]> wrote: > > > Hi Stephen, > > > > I cleaned up the code a bit and then, I tried testing the code merge > > yesterday and I ran into some issues for 3.3.1-SNAPSHOT version. > > > > - I forked apache/tinkerpop repository to my local account and loaded the > > same using an IDE (as maven project). This immediately threw errors in the > > native repositories such as gremlin-core, stating that it is not able to > > find org.apache.tinkerpop.shaded.kryo.Kryo. > > - When I try building the code with 3.3.0 api it works perfectly without > > any error, however for 3.3.1-SNAPSHOT version it is not able to find > > various files and throws errors in the core modules of tinkerpop. Thus, I > > cannot test my module (sparql-gremlin) with the 3.3.1-SNAPSHOT version. > > - Also, when I did a man clean install on 3.3.1-SNAPSHOT, it did get build > > success but a majority of the test cases failed. Not sure if this is worth > > mentioning. > > > > What do you suggest? How do I fix this? > > Also, is it okay if I use the 3.3.0 api version for my module or it is > > absolutely necessary that we have to only use the 3.3.1-SNAPSHOT api > > version? > > > > Thanks in advance! > > > > On 2017-12-12 12:58, Stephen Mallette <[email protected]> wrote: > > > yes - please post questions here. i don't think you need to know much > > about > > > TinkerPop internal structure. I'd think that sparql-gremlin is expected > > to > > > be included in the root of the TinkerPop source as a sub-module to the > > > top-level pom. That just means some minor changes to your pom.xml to get > > it > > > to build along with everything else. See other projects for examples: > > > > > > https://github.com/apache/tinkerpop/blob/f5687ee4497bfbaef4ae89233e4c29 > > f07001ed2c/gremlin-core/pom.xml#L20-L24 > > > > > > You can drop all of this because it is already defined in the root > > pom.xml: > > > > > > https://github.com/LITMUS-Benchmark-Suite/sparql-to- > > gremlin/blob/master/pom.xml#L30-L70 > > > > > > Looking at the rest of your pom.xml now, I'm not sure I understand > > > everything your <build> section is doing and if it's all necessary: The > > > root pom.xml should handle the most common build/deploy options and they > > > will be thus inherited to your sub-module pom which is why, for example, > > > the gremlin-core pom is pretty simple for the <build> section: > > > > > > https://github.com/apache/tinkerpop/blob/f5687ee4497bfbaef4ae89233e4c29 > > f07001ed2c/gremlin-core/pom.xml#L119-L151 > > > > > > If there's anything you're sure can be removed from the sparql-gremlin > > > pom.xml <build> section based on how the TinkerPop root pom.xml is setup, > > > the please feel free to cleanup as much as possible there. > > > > > > As for the general project structure of sparql-gremlin, I don't fully > > > understand how it is arranged. There's > > > > > > /Queries > > > /doc > > > /docs/images > > > /output > > > /src > > > > > > and all of that is repeated inside of the /bin directory. something seems > > > amiss there. maybe once that's cleared up a bit I can think more clearly > > on > > > what additional changes you might need. > > > > > > Another important thing to consider....documentation. Right now, it's all > > > in the README. I think we will want a new section to the Reference > > > Documentation, probably appearing after Gremlin Variants: > > > > > > http://tinkerpop.apache.org/docs/current/reference/#gremlin-variants > > > > > > Perhaps that could be named "Query Languages" where sparql-gremlin would > > be > > > the first sub-section. That would set up for some future where we also > > had > > > sql-gremlin. And perhaps a section for cypher-gremlin which could point > > to > > > Neo4j's work in this area. You can find reference docs for TinkerPop > > here: > > > > > > https://github.com/apache/tinkerpop/tree/f5687ee4497bfbaef4ae89233e4c29 > > f07001ed2c/docs/src/reference > > > > > > The easiest way to generate docs is with docker via: > > > > > > docker/build.sh -d > > > > > > without that you need hadoop running with appropriate configurations: > > > > > > http://tinkerpop.apache.org/docs/current/dev/developer/# > > documentation-environment > > > > > > Well, hope that gives you a few things to work on and think about for > > your > > > first round of changes in your fork. Looking forward to seeing how this > > PR > > > shapes up! > > > > > > > > > On Tue, Dec 12, 2017 at 3:40 AM, Harsh Thakkar <[email protected]> > > wrote: > > > > > > > Hi Stephen, > > > > > > > > Very well then, we will start the migration from today. Also we will > > > > submit the signed iclas' today. > > > > > > > > If we have some questions regarding building the code properly, can we > > > > feel free to ask them here? I assume we might need some guidance on > > how to > > > > getting things plugged in correctly. We both are not much aware of the > > > > internal structure of TinkerPop, so that is why. > > > > > > > > Also, if there is any specific documentation to help us with this, > > please > > > > lend a pointer. > > > > > > > > Many thanks! > > > > > > > > On 2017-12-11 21:33, Stephen Mallette <[email protected]> wrote: > > > > > As there hasn't been any other opinions, It seems we have a lazy > > > > consensus > > > > > to accept sparql-gremlin into TinkerPop's code base. Cool! > > > > > > > > > > Harsh, I think you and Dharmen should proceed with the steps I listed > > > > > above. Once you have the code integrated and building properly in > > your > > > > > fork, please reply back and point us to it and we can start with some > > > > > coarse grained review of what you have. > > > > > > > > > > Thanks, > > > > > > > > > > Stephen > > > > > > > > > > On Fri, Dec 8, 2017 at 8:44 AM, [email protected] <[email protected] > > > > > > > wrote: > > > > > > > > > > > Hi Stephen, > > > > > > > > > > > > Thanks for the insight on the process of this integration. I will > > > > reply to > > > > > > your comments in the same manner. > > > > > > > > > > > > 1. Yes, I will do the fork and migrate the code to the Tinkerpop > > > > > > repository, after cleaning the code a bit. We also need to prepare > > a > > > > > > detailed doc (how-to) for the plugin. This can also be done in > > > > parallel, > > > > > > depending upon the urgency. > > > > > > > > > > > > 2. Yes, we both are contributing to the v0.2 of the sparql-gremlin > > > > plugin. > > > > > > We will both submit the ICLAs. > > > > > > > > > > > > Yes, we (both) will continue to provide support for the 0.2 plugin > > and > > > > > > also extend it in the future (trying to cover SPARQL 1.1 > > specification, > > > > > > also fix the OPTIONAL fix in the current version). > > > > > > > > > > > > Looking forward to hear more on this from the devs :) > > > > > > > > > > > > Cheers! > > > > > > > > > > > > On 2017-12-08 13:41, Stephen Mallette <[email protected]> > > wrote: > > > > > > > I agree with Marko's thoughts, both on this topic of including > > > > > > > sparql-gremlin as well as the wider topic of what should be > > included > > > > in > > > > > > > TinkerPop code base more generally. Providing a path for > > rdf/sparql > > > > folks > > > > > > > to get into the TinkerPop world seems like a smart direction. > > > > > > > > > > > > > > Now, assuming that we have consensus to include sparql-gremlin > > in the > > > > > > > TinkerPop code base, the process will look something like this: > > > > > > > > > > > > > > 1. I think that Harsh should fork the TinkerPop repository and > > > > migrate > > > > > > > sparql-gremlin into its structure. From there we will provide > > > > > > > feedback/review to get that fork into best shape possible prior > > to > > > > his > > > > > > > submitting a pull request. I think we can handle initial feedback > > > > through > > > > > > > the dev list in a separate thread. > > > > > > > > > > > > > > 2. In parallel to the above item, it appears as though there are > > two > > > > > > > contributors on sparql-gremlin: > > > > > > > > > > > > > > https://github.com/LITMUS-Benchmark-Suite/sparql-to- > > > > > > gremlin/graphs/contributors > > > > > > > > > > > > > > Both contributors, Harsh and Dharmen, should submit ICLAs: > > > > > > > > > > > > > > http://apache.org/licenses/icla.pdf > > > > > > > > > > > > > > and send them to [email protected]. > > > > > > > > > > > > > > 3. Once ICLAs are confirmed by secretary, Harsh can submit a pull > > > > request > > > > > > > from his fork where it can under go final review. > > > > > > > > > > > > > > Does that sound sensible to everyone? > > > > > > > > > > > > > > btw, Harsh, it sounds as though you intend to continue > > development on > > > > > > > sparql-gremlin after it is part of the TinkerPop > > repository...does > > > > > > Dharmen > > > > > > > intend to do the same? > > > > > > > > > > > > > > On Fri, Dec 8, 2017 at 6:54 AM, Stephen Mallette < > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > linking marko's reply from the user list: > > > > > > > > > > > > > > > > https://groups.google.com/d/msg/gremlin-users/zK9jj7bWvrQ/ > > > > nE1VvhmeAAAJ > > > > > > > > > > > > > > > > On Thu, Dec 7, 2017 at 1:52 PM, [email protected] < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > >> Hello, dear Gremlin people! > > > > > > > >> > > > > > > > >> Apologies for raising this topic a bit late. I planned to > > start > > > > this > > > > > > > >> thread quite earlier but wasnât able to due to some reasons. > > > > > > > >> > > > > > > > >> ======= short ============================== > > > > > > > >> ============================== > > > > > > > >> I seek your guidance and also help for polishing and > > integrating > > > > the > > > > > > > >> sparql-gremlin 0.2 (https://github.com/LITMUS-Ben > > > > > > > >> chmark-Suite/sparql-to-gremlin) plugin in the apache > > tinkerpop > > > > code > > > > > > > >> base, succeeding its predecessor developed by Daniel Kupitz ( > > > > > > > >> https://github.com/dkuppitz/sparql-gremlin). The new plugin > > > > offers > > > > > > > >> support for a wide range of SPARQL queries from the SPARQL 1.0 > > > > > > features. > > > > > > > >> > > > > > > > >> > > > > > > > >> ============ long ============================== > > > > > > > >> =============================== > > > > > > > >> > > > > > > > >> I am a Ph.D. student at the University of Bonn and work at the > > > > > > > >> intersection of semantic web and graph databases. My thesis is > > > > > > focused on > > > > > > > >> bridging the gap between these two domains by enabling > > support for > > > > > > SPARQL > > > > > > > >> querying of Property Graph databases. Thus, working on the > > > > > > SPARQL-Gremlin > > > > > > > >> interoperability was an obvious idea given the wide > > popularity of > > > > > > Gremlin > > > > > > > >> amongst the Graph DB vendors. > > > > > > > >> > > > > > > > >> The sparql-gremlin 0.1 (link - https://github.com/dkuppitz/ > > > > > > sparql-gremlin) > > > > > > > >> plugin was developed by Daniel Kupitz, which we have extended > > to > > > > > > support > > > > > > > >> various features of the SPARQL 1.0 specification and have > > tested > > > > using > > > > > > > >> various synthetic datasets (such as Northwind dataset and the > > > > Berlin > > > > > > sprawl > > > > > > > >> benchmark [BSBM] dataset) and a wide range of SPARQL queries. > > > > > > > >> > > > > > > > >> The extended version of the plugin (sparql-gremlin 0.2, link - > > > > > > > >> https://github.com/LITMUS-Benchmark-Suite/sparql-to-gremlin) > > > > > > supports a > > > > > > > >> variety of query modifiers (group-by, order-by, counts, etc) > > and > > > > > > complex > > > > > > > >> query features such as union, aggregation, etc. It does not > > > > currently > > > > > > > >> support SPARQL optional queries though. It needs a minor fix. > > > > > > > >> > > > > > > > >> I wish to integrate this updated version to the apache > > tinkerpop > > > > > > codebase > > > > > > > >> and wish to see it roll out as a functional plugin (like the > > old > > > > one, > > > > > > > >> replacing it with the updated version) in the next version of > > > > tinker > > > > > > pop > > > > > > > >> (or even before, however it works out). > > > > > > > >> > > > > > > > >> ============================================================ > > > > > > > >> ================== > > > > > > > >> > > > > > > > >> I am not much aware of how to do it and what steps I need to > > > > follow, > > > > > > so I > > > > > > > >> seek input from you all and have started this thread (as > > > > suggested by > > > > > > > >> Stephen Mallette) and already discussed with Marko Rodriguez > > > > during > > > > > > Graph > > > > > > > >> Day SF 2017 and in other informal communications. > > > > > > > >> > > > > > > > >> Please guide me through the same and let me know what all I > > will > > > > need > > > > > > to > > > > > > > >> do and/or what you will need to get this done. I am happy to > > > > > > collaborate > > > > > > > >> and be a part of this awesome project :) > > > > > > > >> > > > > > > > >> Cheers, > > > > > > > >> Harsh > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
