Re: [Wikidata] SPARQL endpoint caching

Hi! > > I'll do a presentation next week, in which I intend to demonstrate > that I can add a Wikidata value online, which then is available > immediately for my application - as well as for the whole rest of the > world. (In Library Land, that's a real blast, because business > processes

Re: [Wikidata] SPARQL endpoint caching

-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Am 17.02.2016 um 09:54 schrieb Katie Filbert: > On Wed, Feb 17, 2016 at 9:39 AM, Markus Krötzsch > > wrote: > > On 17.02.2016 08:16, Stas Malyshev wrote: > > Hi! > > (2)

Re: [Wikidata] SPARQL endpoint caching

If you add a proxy cache like Varnish in front of the endpoint, it will cache based on the Cache-Control: max-age and ETag headers sent by the endpoint, which I guess can be configured. But you can also PURGE and BAN specific cache entries from Varnish to force fresh retrieval. On Wed, Feb 17,

Re: [Wikidata] SPARQL endpoint caching

On 17.02.2016 10:34, Magnus Manske wrote: On Wed, Feb 17, 2016 at 7:16 AM Stas Malyshev > wrote: Well, again the problem is that one use case that I think absolutely needs caching - namely, exporting data to graphs, maps, etc.

Re: [Wikidata] SPARQL endpoint caching

On Wed, Feb 17, 2016 at 7:16 AM Stas Malyshev wrote: > > Well, again the problem is that one use case that I think absolutely > needs caching - namely, exporting data to graphs, maps, etc. deployed on > wiki pages - is also the one not implemented yet because we don't

Re: [Wikidata] SPARQL endpoint caching

On 17.02.2016 09:54, Katie Filbert wrote: ... I think it would be nice if having a graph with query on a page does not too much adversely affect the time it takes to save a page. (e.g. if running the query takes 20 seconds..., and instead reuse cached query results) And not have such usage

Re: [Wikidata] SPARQL endpoint caching

Basicly I have two use-cases of the SPARQL endpoint. 1. Concept finding for bot activities, 2. example /tutorial/show-case queries. Starting with the second, especially if it is on prototyping an (extensive) caching time is totally acceptable to me and definitely worth it would it improve the

Re: [Wikidata] SPARQL endpoint caching

On Wed, Feb 17, 2016 at 9:39 AM, Markus Krötzsch < mar...@semantic-mediawiki.org> wrote: > On 17.02.2016 08:16, Stas Malyshev wrote: > >> Hi! >> >> (2) Shouldn't BlazeGraph do the caching (too)? It knows how much a query >>> costs to re-run and it could even know if a query is affected by a data

Re: [Wikidata] SPARQL endpoint caching

Hi! > I agree, we should look at some actual traffic to see how many queries > /could/ be cached in a 2/5/10/60 min window. Maybe remove the example > queries from those numbers, to separate the "production" and testing > usage. Also, look at query runtime; if only "cheap" queries would be >

Re: [Wikidata] SPARQL endpoint caching

Hi! > How often does *exactly* the same query get run within 2 minutes ? Depends where the query is coming from. E.g. if there's a graph backed by query, then a lot of people can be seeing the graph and running a query. Same if somebody publishes a link to some query e.g. during a talk or in

Re: [Wikidata] SPARQL endpoint caching

I agree, we should look at some actual traffic to see how many queries /could/ be cached in a 2/5/10/60 min window. Maybe remove the example queries from those numbers, to separate the "production" and testing usage. Also, look at query runtime; if only "cheap" queries would be cached, there is no

Re: [Wikidata] SPARQL endpoint caching

I have to say that I am dubious. How often does *exactly* the same query get run within 2 minutes ? Does the same query ever get run ? The first thing to do, surely, is to create a hash for each query, (or better, perhaps, something like a tinyurl so then the lookup is reversible, record a

[Wikidata] SPARQL endpoint caching

Hi! With Wikidata Query Service usage raising and more use cases being found, it is time to consider caching infrastructure for results, since queries are expensive. One of the questions I would like to solicit feedback on is the following: Should we have default SPARQL endpoint cached or