as I said Romain you are probably best advised to roll-your-own DAG code. I initially thought you could model the DAG in RDF (Bob DuCharme wrote a blog post about using Spark with SPARQL* in 2015) but since that's not your focus Jena adds much more stuff, stuff you most likely wont need on the lower graph level processing in your case.
http://www.snee.com/bobdc.blog/2015/03/spark-and-sparql-rdf-graphs-an.html On Mon, Sep 9, 2019 at 5:44 AM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Hmm, the DAG in my case is about modelising a problem and the algorithm > modify it to make it simpler and closer to a solution. > > If it helps, see it as a generic algorithm and iterations. > > So RDF and backends will not help since it is not a goal and if so the > backend would be a plain json serialization for example. > > So I see how a DAG can be enriched to be jena compatible - change vertices > and edges impl - but the opposite is still less clear for me. > > Le dim. 8 sept. 2019 à 23:31, Marco Neumann <marco.neum...@gmail.com> a > écrit : > > > On Sun, Sep 8, 2019 at 10:15 PM Romain Manni-Bucau < > rmannibu...@gmail.com> > > wrote: > > > > > Le dim. 8 sept. 2019 à 23:03, Marco Neumann <marco.neum...@gmail.com> > a > > > écrit : > > > > > > > Romain, if I understand you correctly here, and since you are at the > > > > beginning of this process I think I would like to see this designed > as > > > > another custom graph in Jena that in its most basic form could also > > work > > > as > > > > a stand-alone graph with eventually zero jena dependencies. > > > > > > > > > > A bit as a subproject? = Jena would deliver it but not use it? > > > > > > there are a number of adapters that implement vendor specefic backends. > > orcale, virtuoso etc > > > > > > > > > > > > > > said said it sounds like there is some overlap in your effort that > > could > > > > benefit significantly from Jena infratructure features directly. > > > > > > > > > > Hmm, do you have some pointers handy? > > > > > > search for jena adapters and custom implementations of GraphBase. > > > > I do say so since you explictly mentioned that you are not yet ready to > go > > RDF here and a basic DAG impl might be feasible for your anticipated use > > case. but if this takes off in context of the Commons implementation I'd > > like to be ready to use these bridges to the RDF world and use Jena to > > scale for enterprise and web appplication. > > > > > > > > > > > > > > > > > On Sun, Sep 8, 2019 at 4:35 PM Romain Manni-Bucau < > > rmannibu...@gmail.com > > > > > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > I'm coming from dev@commons list and was redirected there by > Marco, > > > the > > > > > discussion is available on our archives ([1]) > > > > > > > > > > Long story short, I'm looking for a core DAG library which is > > > dependency > > > > > free (~100K is ok for the stack, more will need to bring a lot of > > value > > > > > ;)). > > > > > > > > > > My need is "simple" but very generic (so shareable) and should be > > > strong > > > > > (so better if shared and reused instead of coded by app): > > > > > > > > > > 1. load/build a graph (DAG actually) model > > > > > 2. be able to navigate in the model to capture nodes (mainly > upstream > > > > > browsing in my case but downstream will be useful quite quickly) > > > > > 3. create a new DAG by "mutating" (copy of an immutable model is > not > > an > > > > > issue) the original graph > > > > > 4. dump back the newly obtained graph > > > > > > > > > > I did a quick prototype at [2] - please ignore the specificity of > the > > > > model > > > > > loading using JSON-B and modelisation using ports which is likely > > > > specific > > > > > to my case (?). > > > > > The interesting piece of code (core logic) is this class [3] and > the > > > > > findUpStream method usage of the DAG. > > > > > In terms of logic it is very close to what Spark does when it > "pushes > > > > down" > > > > > some filter/predicates in a source during a plan computation > (that's > > > why > > > > it > > > > > is called this way in the PoC). > > > > > > > > > > Therefore my question to Jena are: > > > > > > > > > > 1. Does it somehow overlap with Jena goals (even if low level)? > > > > > 2. Can it be extracted in a jena-graph (core would maybe depend on > > it?) > > > > > without guava, slf4j implementation (api is ok) and base > > dependencies? > > > > > Concretely jena-graph+slf4j-api are ok for me, more wouldn't. > > > > > 3. Is a graph modelization not RDF specific ok and would it be > stable > > > for > > > > > consumers? > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > https://markmail.org/search/?q=graph%20status%20list%3Aorg.apache.commons.dev%2F#query:graph%20status%20list%3Aorg.apache.commons.dev%2F+page:1+mid:pxecs5y3ezmfztbj+state:results > > > > > [2] https://github.com/rmannibucau/drag > > > > > [3] > > > > > > > > > > > > > > > > > > > > https://github.com/rmannibucau/drag/blob/master/src/main/java/com/github/rmannibucau/graph/drag/rule/PushdownRule.java > > > > > > > > > > Romain Manni-Bucau > > > > > @rmannibucau <https://twitter.com/rmannibucau> | Blog > > > > > <https://rmannibucau.metawerx.net/> | Old Blog > > > > > <http://rmannibucau.wordpress.com> | Github < > > > > > https://github.com/rmannibucau> | > > > > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book > > > > > < > > > > > > > > > > > > > > > https://www.packtpub.com/application-development/java-ee-8-high-performance > > > > > > > > > > > > > > > -- > > > > > > > > > > > > --- > > > > Marco Neumann > > > > KONA > > > > > > > > > -- > > > > > > --- > > Marco Neumann > > KONA > > > -- --- Marco Neumann KONA
