The DERI work is supposed to be open source, however, I don't believe the papers have been written yet so I suspect they are not yet releasing the code.
In my case we were building queries across multiple endpoints based upon the topics in an original query, so I was more concerned about writing a properly constructed federated query than about efficiency of the specific query. That being said, there were a couple of issues that we had to solve (most notably for optional values shared across the endpoints) to make the response time acceptable. In any case we simply passed the query transformed query to the default optimiser after we were done. The only optimisation that we did was to look at statements and run the ones with the fewest unknowns before queries with multiple unknowns. We probably could have added another layer of efficiency by placing statements with unknown predicates later in the query. We also did some work on response times from endpoints (we had cases where multiple SPARQL endpoints contained the same graphs so we had a choice of query points) and picking the most efficient for our query. We did the evaluation of responses on a schedule so we always had reasonably fresh data. This also allowed us to determine if an endpoint was down and switch to a backup or simply not generate the query for that endpoint. Hope this helps, Claude On Thu, Aug 29, 2013 at 1:52 PM, Diogo FC Patrao <[email protected]>wrote: > Hi Claude > > Actually I did some testing with Transformer, which I'm more used to than > OpVisitor, and with it I can evaluate a cost function for each node. > However I would need several different trees to be evaluated in order to > find the best plan, and I can't see how Transformer could help on it. > > I was thinking that maybe ARQ has a default query planner that could be > substituted on configuration, or by applying a custom cost function. > > Was your work on DERI opensourced? Can you talk about it? I'm curious about > which strategies did you used on your planner. > > Cheers, > > > > -- > diogo patrão > > > > > On Wed, Aug 28, 2013 at 6:24 PM, Claude Warren <[email protected]> wrote: > > > This discussion reminds me that a friend and I discussed the possibility > of > > adding pluggable OpVisitors to allow chained rewrites of the query before > > execution. > > > > > > On Wed, Aug 28, 2013 at 10:23 PM, Claude Warren <[email protected]> > wrote: > > > > > When I did the work on the query engine described above I used an > > > OpVisitor implementation in modifyOp to rewrite the query. It should > be > > > possible to implement the optimization there, but I don't know that it > is > > > the best place. > > > > > > Claude > > > > > > > > > On Wed, Aug 28, 2013 at 5:19 PM, Diogo FC Patrao < > [email protected] > > >wrote: > > > > > >> Hi Claude > > >> > > >> I meant the first option, optimize queries to run at federated > > >> environments. Although I'm working on a query "federator" as well, and > > am > > >> also interested in similar projects as well. > > >> > > >> Thanks > > >> > > >> -- > > >> diogo patrão > > >> > > >> > > >> > > >> > > >> On Wed, Aug 28, 2013 at 12:50 PM, Claude Warren <[email protected]> > > wrote: > > >> > > >> > I did some work on query planning for federated queries when I > worked > > >> for > > >> > DERI. (http://deri.ie). However, that work may have been a bit > more > > >> > complex than you are considering as we had some sources for > vocabulary > > >> > equivalence and were attempting to handle that at the same time. > > >> > > > >> > When you say a query planner for federated queries, are you looking > to > > >> > optimise queries that contain federated queries, or attempting to > > build > > >> a > > >> > federated query from a non-federated one? > > >> > > > >> > Claude > > >> > > > >> > > > >> > On Wed, Aug 28, 2013 at 1:54 PM, Diogo FC Patrao < > > [email protected] > > >> > >wrote: > > >> > > > >> > > Hello > > >> > > > > >> > > I was thinking about writing a query planner for federated > queries. > > >> > AFAIK, > > >> > > there's no such thing on ARQ, and couldn't find any third-party > libs > > >> for > > >> > > that. > > >> > > > > >> > > So my questions are: > > >> > > > > >> > > 1) do you know of any open source projects out there that deals > with > > >> it? > > >> > > I'm willing to contribute. > > >> > > > > >> > > 2) If there is no such project, is a Transformer the right place > to > > go > > >> > for > > >> > > it? I thought about extending TransformerCopy to reorder OpJoins > to > > >> solve > > >> > > first those with smaller costs. > > >> > > > > >> > > Thanks! > > >> > > > > >> > > > > >> > > -- > > >> > > diogo patrão > > >> > > > > >> > > > >> > > > >> > > > >> > -- > > >> > I like: Like Like - The likeliest place on the web< > > >> > http://like-like.xenei.com> > > >> > Identity: https://www.identify.nu/[email protected] > > >> > LinkedIn: http://www.linkedin.com/in/claudewarren > > >> > > > >> > > > > > > > > > > > > -- > > > I like: Like Like - The likeliest place on the web< > > http://like-like.xenei.com> > > > Identity: https://www.identify.nu/[email protected] > > > LinkedIn: http://www.linkedin.com/in/claudewarren > > > > > > > > > > > -- > > I like: Like Like - The likeliest place on the web< > > http://like-like.xenei.com> > > Identity: https://www.identify.nu/[email protected] > > LinkedIn: http://www.linkedin.com/in/claudewarren > > > -- I like: Like Like - The likeliest place on the web<http://like-like.xenei.com> Identity: https://www.identify.nu/[email protected] LinkedIn: http://www.linkedin.com/in/claudewarren
