Gehel created this task. Gehel added projects: Operations, Traffic, Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata.
TASK DESCRIPTION wdqs-updater adds a `nocache=<timestamp>` parameter to request to wikidata. This is done to ensure that when processing recent changes, the data fetched from wikidata is not older than the update. This is quite aggressive and we should have a better way to ensure freshness. This issue is raised as part of T217893 <https://phabricator.wikimedia.org/T217893>, where an external wdqs-updater was generating a significant amount of traffic. A few idea: - disable cache busting by default, enable it internally - use the event date instead of the current date as timestamp (would enable caching the fetch for the same event from multiple clients) - don't do cache busting on events older than X - back off when data received is older than the event There are probably better ways to do this, but I'm not really sure what is available on the wikidata API to ensure freshness. TASK DETAIL https://phabricator.wikimedia.org/T217897 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Aklapper, Gehel, alaa_wmde, Legado_Shulgin, Nandana, thifranc, AndyTan, Davinaclare77, Qtn1293, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Jonas, Xmlizer, Wong128hk, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
