[Wikidata-bugs] [Maniphest] [Commented On] T126730: Caching for results of wikidatasparql queries for Graphs
Yurik added a comment. I think Graph extension was shown as an example that can easily overwhelm the system without proper caching. More elaborate, research-oriented usages are usually different because they are being done interactively by the query developer. TASK DETAIL https://phabricator.wikimedia.org/T126730 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik Cc: Christopher, Yurik, hoo, Aklapper, aude, Izno, Wikidata-bugs, Mbch331, Jay8g, Ltrlg ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T126730: Caching for results of wikidatasparql queries for Graphs
hoo added a comment. Please note that a SPARQL query takes at least a few hundred ms to run (and up to 30s even), thus doing that during page save/ render is not an option. TASK DETAIL https://phabricator.wikimedia.org/T126730 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hoo Cc: hoo, Aklapper, aude, Izno, Wikidata-bugs, Yurik, Mbch331, Jay8g, Ltrlg ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T126730: Caching for results of wikidatasparql queries for Graphs
Yurik added a subscriber: Yurik. Yurik added a comment. I agree that the query should go through varnish, but as usual - what is the purging policy? It will be very complicated to track each piece of data that appears in the result back to the original, and to invalidate it when original changes. We could introduce "its ok to be stale" argument, and allow manual cache flushing. - In VCL, check if request headers `Allow-Stale` is set, return the cached value. Or use some max age setting. - If request has no special headers, treat it as "cache for a minute maximum" - to prevent DOSing TASK DETAIL https://phabricator.wikimedia.org/T126730 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik Cc: Yurik, hoo, Aklapper, aude, Izno, Wikidata-bugs, Mbch331, Jay8g, Ltrlg ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T126730: Caching for results of wikidatasparql queries for Graphs
Christopher added a subscriber: Christopher. Christopher added a comment. question: why is this task limited in scope to the Graph extension? TASK DETAIL https://phabricator.wikimedia.org/T126730 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Christopher Cc: Christopher, Yurik, hoo, Aklapper, aude, Izno, Wikidata-bugs, Mbch331, Jay8g, Ltrlg ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T126730: Caching for results of wikidatasparql queries for Graphs
hoo added a comment. If we want a broad solution (and not just a stop gap, that might work for graphs), I think we should go for Query entities again. These would address the problems brought up over here and should also allow for better maintainability. TASK DETAIL https://phabricator.wikimedia.org/T126730 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hoo Cc: Christopher, Yurik, hoo, Aklapper, aude, Izno, Wikidata-bugs, Mbch331, Jay8g, Ltrlg ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
