Gehel added a comment.
While we’d love to support this use case, the limits on volume are there to ensure the stability of the service. Federation on large volumes (via SERVICE or otherwise) is problematic. It increases variability in response time, consumes idle resources on intermediate servers, and increases the likelihood of failure and the cost of retries. In general, for a synchronous service like WDQS, we want to have strong limits on resource consumption. In the case of working through a large number of items using WDQS (for example for the detection of redirects use cases), you would need to find a way to batch those requests in smaller chunks. I don’t know the details of what is involved here, but it sounds like this is a case where offline processing via the dumps might be a better option (Wikidata <https://dumps.wikimedia.org/wikidatawiki/latest/> / Commons <https://dumps.wikimedia.org/other/wikibase/commonswiki/>). This could allow for more specialized processing that would be more efficient than relying on a generic SPARQL endpoint. I'm keeping this open for now, so that we can continue this discussion as needed. TASK DETAIL https://phabricator.wikimedia.org/T261716 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Gehel, Jarekt, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Ramsey-WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Poyekhali, _jensen, rosalieper, Taiwania_Justo, Scott_WUaS, Jonas, Xmlizer, Ixocactus, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, El_Grafo, Dinoguy1000, Manybubbles, Steinsplitter, Mbch331, Keegan
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
