Hi Josh, I already wrote an elaborate reply to your comment. I think it went somewhere but didn't show up :(
I will summarize my reply here now.. Yes, I am of the same opinion of having a continuous SPARQL implementation on top of Gremlin. Also, I am working on a custom interface, (as we speak) in my current research, on proposing an information preserving RDF <-> PG converter. This will allow interoperability between the semantic web and graph database communities to leverage the advantages of one another. i.e. the earlier can traverse and the later can have a more diverse access portfolio to rich datasets. My Ph.D. thesis is more or less focused on this. It started from proposing a robust open and extensible benchmarking platform "LITMUS" [], which eventually led me to address all these issues and thus my keen interest :) If I am not getting it wrong, the other interfaces you mentioned, about that, do you wish to see them eventually integrated into tinkerpop? or are you implying that this should be already done before the next release? Thanks for your pointers! Cheers! On 2017-12-13 16:46, Joshua Shinavier <[email protected]> wrote: > Hi Harsh, > > Glad you are taking Daniel's work forward. In porting the code to the > TinkerPop code base, might I suggest we allow for not only SPARQL-Gremlin, > but a whole suite of RDF tools as in TP2. Perhaps call the module > rdf-gremlin. Then we could have all of: > > * SPARQL-Gremlin: executes standard SPARQL queries over a Property Graph > database > * GraphSail [1,2]: stores RDF quads in the database, explicitly, and > enables SPARQL and triple pattern queries over the quads > * PropertyGraphSail [3]: exposes a Property Graph with of two mappings to > the RDF data model > * SailGraph [4]: takes an RDF triple store (not natively supporting > Gremlin) and enables Gremlin queries > * others? I have often thought that a continuous SPARQL implementation > built on Gremlin would be powerful > > The biggest mismatch between the TP2 suite and what might be built for > Apache TinkerPop is that the previous suite was implemented using (Eclipse) > RDF4j, whereas things seem to be leaning towards (Apache) Jena now. > However, the same principles could be applied. > > Josh > > > [1] https://github.com/tinkerpop/blueprints/wiki/Sail-Ouplementation > [2] https://github.com/joshsh/graphsail > [3] > https://github.com/tinkerpop/blueprints/wiki/PropertyGraphSail-Ouplementation > [4] https://github.com/tinkerpop/blueprints/wiki/Sail-Implementation > > > > > On Wed, Dec 13, 2017 at 4:03 AM, Stephen Mallette <[email protected]> > wrote: > > > I suggest that you read through the dev docs a bit. There's lots of little > > odds/ends there about how to develop on the TinkerPop code base. For > > example, for intellij issues, please have a look at this: > > > > http://tinkerpop.apache.org/docs/current/dev/developer/#_ > > ide_setup_with_intellij > > > > > - Also, when I did a man clean install on 3.3.1-SNAPSHOT, it did get > > build success but a majority of the test cases failed. Not sure if this is > > worth mentioning. > > > > there should be no failures on master and 3.3.1-SNAPSHOT. hard to say what > > is wrong without some error logs > > > > > Also, is it okay if I use the 3.3.0 api version for my module or it is > > absolutely necessary that we have to only use the 3.3.1-SNAPSHOT api > > version? > > > > you should be on 3.3.1-SNAPSHOT for your API version now that you've > > integrated sparql-gremlin into the tinkerpop code base. ultimately, it will > > all release together as part of a single package, so it should all be on > > the same version. > > > > On Wed, Dec 13, 2017 at 6:43 AM, Harsh Thakkar <[email protected]> wrote: > > > > > Hi Stephen, > > > > > > I cleaned up the code a bit and then, I tried testing the code merge > > > yesterday and I ran into some issues for 3.3.1-SNAPSHOT version. > > > > > > - I forked apache/tinkerpop repository to my local account and loaded the > > > same using an IDE (as maven project). This immediately threw errors in > > the > > > native repositories such as gremlin-core, stating that it is not able to > > > find org.apache.tinkerpop.shaded.kryo.Kryo. > > > - When I try building the code with 3.3.0 api it works perfectly without > > > any error, however for 3.3.1-SNAPSHOT version it is not able to find > > > various files and throws errors in the core modules of tinkerpop. Thus, I > > > cannot test my module (sparql-gremlin) with the 3.3.1-SNAPSHOT version. > > > - Also, when I did a man clean install on 3.3.1-SNAPSHOT, it did get > > build > > > success but a majority of the test cases failed. Not sure if this is > > worth > > > mentioning. > > > > > > What do you suggest? How do I fix this? > > > Also, is it okay if I use the 3.3.0 api version for my module or it is > > > absolutely necessary that we have to only use the 3.3.1-SNAPSHOT api > > > version? > > > > > > Thanks in advance! > > > > > > On 2017-12-12 12:58, Stephen Mallette <[email protected]> wrote: > > > > yes - please post questions here. i don't think you need to know much > > > about > > > > TinkerPop internal structure. I'd think that sparql-gremlin is expected > > > to > > > > be included in the root of the TinkerPop source as a sub-module to the > > > > top-level pom. That just means some minor changes to your pom.xml to > > get > > > it > > > > to build along with everything else. See other projects for examples: > > > > > > > > https://github.com/apache/tinkerpop/blob/ > > f5687ee4497bfbaef4ae89233e4c29 > > > f07001ed2c/gremlin-core/pom.xml#L20-L24 > > > > > > > > You can drop all of this because it is already defined in the root > > > pom.xml: > > > > > > > > https://github.com/LITMUS-Benchmark-Suite/sparql-to- > > > gremlin/blob/master/pom.xml#L30-L70 > > > > > > > > Looking at the rest of your pom.xml now, I'm not sure I understand > > > > everything your <build> section is doing and if it's all necessary: The > > > > root pom.xml should handle the most common build/deploy options and > > they > > > > will be thus inherited to your sub-module pom which is why, for > > example, > > > > the gremlin-core pom is pretty simple for the <build> section: > > > > > > > > https://github.com/apache/tinkerpop/blob/ > > f5687ee4497bfbaef4ae89233e4c29 > > > f07001ed2c/gremlin-core/pom.xml#L119-L151 > > > > > > > > If there's anything you're sure can be removed from the sparql-gremlin > > > > pom.xml <build> section based on how the TinkerPop root pom.xml is > > setup, > > > > the please feel free to cleanup as much as possible there. > > > > > > > > As for the general project structure of sparql-gremlin, I don't fully > > > > understand how it is arranged. There's > > > > > > > > /Queries > > > > /doc > > > > /docs/images > > > > /output > > > > /src > > > > > > > > and all of that is repeated inside of the /bin directory. something > > seems > > > > amiss there. maybe once that's cleared up a bit I can think more > > clearly > > > on > > > > what additional changes you might need. > > > > > > > > Another important thing to consider....documentation. Right now, it's > > all > > > > in the README. I think we will want a new section to the Reference > > > > Documentation, probably appearing after Gremlin Variants: > > > > > > > > http://tinkerpop.apache.org/docs/current/reference/#gremlin-variants > > > > > > > > Perhaps that could be named "Query Languages" where sparql-gremlin > > would > > > be > > > > the first sub-section. That would set up for some future where we also > > > had > > > > sql-gremlin. And perhaps a section for cypher-gremlin which could > > point > > > to > > > > Neo4j's work in this area. You can find reference docs for TinkerPop > > > here: > > > > > > > > https://github.com/apache/tinkerpop/tree/ > > f5687ee4497bfbaef4ae89233e4c29 > > > f07001ed2c/docs/src/reference > > > > > > > > The easiest way to generate docs is with docker via: > > > > > > > > docker/build.sh -d > > > > > > > > without that you need hadoop running with appropriate configurations: > > > > > > > > http://tinkerpop.apache.org/docs/current/dev/developer/# > > > documentation-environment > > > > > > > > Well, hope that gives you a few things to work on and think about for > > > your > > > > first round of changes in your fork. Looking forward to seeing how this > > > PR > > > > shapes up! > > > > > > > > > > > > On Tue, Dec 12, 2017 at 3:40 AM, Harsh Thakkar <[email protected]> > > > wrote: > > > > > > > > > Hi Stephen, > > > > > > > > > > Very well then, we will start the migration from today. Also we will > > > > > submit the signed iclas' today. > > > > > > > > > > If we have some questions regarding building the code properly, can > > we > > > > > feel free to ask them here? I assume we might need some guidance on > > > how to > > > > > getting things plugged in correctly. We both are not much aware of > > the > > > > > internal structure of TinkerPop, so that is why. > > > > > > > > > > Also, if there is any specific documentation to help us with this, > > > please > > > > > lend a pointer. > > > > > > > > > > Many thanks! > > > > > > > > > > On 2017-12-11 21:33, Stephen Mallette <[email protected]> wrote: > > > > > > As there hasn't been any other opinions, It seems we have a lazy > > > > > consensus > > > > > > to accept sparql-gremlin into TinkerPop's code base. Cool! > > > > > > > > > > > > Harsh, I think you and Dharmen should proceed with the steps I > > listed > > > > > > above. Once you have the code integrated and building properly in > > > your > > > > > > fork, please reply back and point us to it and we can start with > > some > > > > > > coarse grained review of what you have. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Stephen > > > > > > > > > > > > On Fri, Dec 8, 2017 at 8:44 AM, [email protected] < > > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > > > Hi Stephen, > > > > > > > > > > > > > > Thanks for the insight on the process of this integration. I will > > > > > reply to > > > > > > > your comments in the same manner. > > > > > > > > > > > > > > 1. Yes, I will do the fork and migrate the code to the Tinkerpop > > > > > > > repository, after cleaning the code a bit. We also need to > > prepare > > > a > > > > > > > detailed doc (how-to) for the plugin. This can also be done in > > > > > parallel, > > > > > > > depending upon the urgency. > > > > > > > > > > > > > > 2. Yes, we both are contributing to the v0.2 of the > > sparql-gremlin > > > > > plugin. > > > > > > > We will both submit the ICLAs. > > > > > > > > > > > > > > Yes, we (both) will continue to provide support for the 0.2 > > plugin > > > and > > > > > > > also extend it in the future (trying to cover SPARQL 1.1 > > > specification, > > > > > > > also fix the OPTIONAL fix in the current version). > > > > > > > > > > > > > > Looking forward to hear more on this from the devs :) > > > > > > > > > > > > > > Cheers! > > > > > > > > > > > > > > On 2017-12-08 13:41, Stephen Mallette <[email protected]> > > > wrote: > > > > > > > > I agree with Marko's thoughts, both on this topic of including > > > > > > > > sparql-gremlin as well as the wider topic of what should be > > > included > > > > > in > > > > > > > > TinkerPop code base more generally. Providing a path for > > > rdf/sparql > > > > > folks > > > > > > > > to get into the TinkerPop world seems like a smart direction. > > > > > > > > > > > > > > > > Now, assuming that we have consensus to include sparql-gremlin > > > in the > > > > > > > > TinkerPop code base, the process will look something like this: > > > > > > > > > > > > > > > > 1. I think that Harsh should fork the TinkerPop repository and > > > > > migrate > > > > > > > > sparql-gremlin into its structure. From there we will provide > > > > > > > > feedback/review to get that fork into best shape possible prior > > > to > > > > > his > > > > > > > > submitting a pull request. I think we can handle initial > > feedback > > > > > through > > > > > > > > the dev list in a separate thread. > > > > > > > > > > > > > > > > 2. In parallel to the above item, it appears as though there > > are > > > two > > > > > > > > contributors on sparql-gremlin: > > > > > > > > > > > > > > > > https://github.com/LITMUS-Benchmark-Suite/sparql-to- > > > > > > > gremlin/graphs/contributors > > > > > > > > > > > > > > > > Both contributors, Harsh and Dharmen, should submit ICLAs: > > > > > > > > > > > > > > > > http://apache.org/licenses/icla.pdf > > > > > > > > > > > > > > > > and send them to [email protected]. > > > > > > > > > > > > > > > > 3. Once ICLAs are confirmed by secretary, Harsh can submit a > > pull > > > > > request > > > > > > > > from his fork where it can under go final review. > > > > > > > > > > > > > > > > Does that sound sensible to everyone? > > > > > > > > > > > > > > > > btw, Harsh, it sounds as though you intend to continue > > > development on > > > > > > > > sparql-gremlin after it is part of the TinkerPop > > > repository...does > > > > > > > Dharmen > > > > > > > > intend to do the same? > > > > > > > > > > > > > > > > On Fri, Dec 8, 2017 at 6:54 AM, Stephen Mallette < > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > linking marko's reply from the user list: > > > > > > > > > > > > > > > > > > https://groups.google.com/d/msg/gremlin-users/zK9jj7bWvrQ/ > > > > > nE1VvhmeAAAJ > > > > > > > > > > > > > > > > > > On Thu, Dec 7, 2017 at 1:52 PM, [email protected] < > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > >> Hello, dear Gremlin people! > > > > > > > > >> > > > > > > > > >> Apologies for raising this topic a bit late. I planned to > > > start > > > > > this > > > > > > > > >> thread quite earlier but wasnât able to due to some > > > > > > > > >> reasons. > > > > > > > > >> > > > > > > > > >> ======= short ============================== > > > > > > > > >> ============================== > > > > > > > > >> I seek your guidance and also help for polishing and > > > integrating > > > > > the > > > > > > > > >> sparql-gremlin 0.2 (https://github.com/LITMUS-Ben > > > > > > > > >> chmark-Suite/sparql-to-gremlin) plugin in the apache > > > tinkerpop > > > > > code > > > > > > > > >> base, succeeding its predecessor developed by Daniel Kupitz > > ( > > > > > > > > >> https://github.com/dkuppitz/sparql-gremlin). The new plugin > > > > > offers > > > > > > > > >> support for a wide range of SPARQL queries from the SPARQL > > 1.0 > > > > > > > features. > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> ============ long ============================== > > > > > > > > >> =============================== > > > > > > > > >> > > > > > > > > >> I am a Ph.D. student at the University of Bonn and work at > > the > > > > > > > > >> intersection of semantic web and graph databases. My thesis > > is > > > > > > > focused on > > > > > > > > >> bridging the gap between these two domains by enabling > > > support for > > > > > > > SPARQL > > > > > > > > >> querying of Property Graph databases. Thus, working on the > > > > > > > SPARQL-Gremlin > > > > > > > > >> interoperability was an obvious idea given the wide > > > popularity of > > > > > > > Gremlin > > > > > > > > >> amongst the Graph DB vendors. > > > > > > > > >> > > > > > > > > >> The sparql-gremlin 0.1 (link - https://github.com/dkuppitz/ > > > > > > > sparql-gremlin) > > > > > > > > >> plugin was developed by Daniel Kupitz, which we have > > extended > > > to > > > > > > > support > > > > > > > > >> various features of the SPARQL 1.0 specification and have > > > tested > > > > > using > > > > > > > > >> various synthetic datasets (such as Northwind dataset and > > the > > > > > Berlin > > > > > > > sprawl > > > > > > > > >> benchmark [BSBM] dataset) and a wide range of SPARQL > > queries. > > > > > > > > >> > > > > > > > > >> The extended version of the plugin (sparql-gremlin 0.2, > > link - > > > > > > > > >> https://github.com/LITMUS-Benchmark-Suite/sparql-to-gremlin > > ) > > > > > > > supports a > > > > > > > > >> variety of query modifiers (group-by, order-by, counts, etc) > > > and > > > > > > > complex > > > > > > > > >> query features such as union, aggregation, etc. It does not > > > > > currently > > > > > > > > >> support SPARQL optional queries though. It needs a minor > > fix. > > > > > > > > >> > > > > > > > > >> I wish to integrate this updated version to the apache > > > tinkerpop > > > > > > > codebase > > > > > > > > >> and wish to see it roll out as a functional plugin (like the > > > old > > > > > one, > > > > > > > > >> replacing it with the updated version) in the next version > > of > > > > > tinker > > > > > > > pop > > > > > > > > >> (or even before, however it works out). > > > > > > > > >> > > > > > > > > >> ============================== > > ============================== > > > > > > > > >> ================== > > > > > > > > >> > > > > > > > > >> I am not much aware of how to do it and what steps I need to > > > > > follow, > > > > > > > so I > > > > > > > > >> seek input from you all and have started this thread (as > > > > > suggested by > > > > > > > > >> Stephen Mallette) and already discussed with Marko Rodriguez > > > > > during > > > > > > > Graph > > > > > > > > >> Day SF 2017 and in other informal communications. > > > > > > > > >> > > > > > > > > >> Please guide me through the same and let me know what all I > > > will > > > > > need > > > > > > > to > > > > > > > > >> do and/or what you will need to get this done. I am happy to > > > > > > > collaborate > > > > > > > > >> and be a part of this awesome project :) > > > > > > > > >> > > > > > > > > >> Cheers, > > > > > > > > >> Harsh > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
