| Gehel added a comment. |
I'm late to the party, so a few notes in no particular order:
- WDQS queries from Kartotherian are arbitrary, and it is not really possible to restrict them without heavily impacting functionality. In most cases they will come from a user editing a <mapframe/> tag, so we have some level of control there, but anyone could also access Kartotherian directly. The external cluster seems to be the right place to send those queries.
- We have varnish in front of Kartotherian, so there is already some higher level caching in place, caching the requests themselves is probably not necessary. We could configure Kartotherian to use wdqs.discovery.wmnet which is the internal endpoint behind query.wikidata.org. This internal traffic should not need to go through our web proxy.
- The problem described in this task is that the traffic from kartotherian is currently being throttled, as wdqs is throttling based on UA and IP, and Kartotherian funnels traffic from a multitude of clients. Note that at the moment, this appears when wdqs is already overloaded. Since Kartotherian is acting as a proxy to wdqs in this case, it make sense to have it publish an X-Client-IP header.
TASK DETAIL
EMAIL PREFERENCES
To: Mholloway, Gehel
Cc: Stashbot, mobrovac, MSantos, Gehel, Aklapper, MaxSem, Pnorman, Mholloway, Smalyshev, Nandana, Amatissart, Lahi, Gq86, Looniverse, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Orienteerix, merbst, LawExplorer, JGirault, Jonas, phabyogi, Xmlizer, Susannaanas, lxbarth, Eevans, jkroll, Planemad, Hardikj, Wikidata-bugs, Jdouglas, bearND, aude, Tobias1984, Manybubbles, Yurik, Jdforrester-WMF, Mbch331, Jay8g
Cc: Stashbot, mobrovac, MSantos, Gehel, Aklapper, MaxSem, Pnorman, Mholloway, Smalyshev, Nandana, Amatissart, Lahi, Gq86, Looniverse, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Orienteerix, merbst, LawExplorer, JGirault, Jonas, phabyogi, Xmlizer, Susannaanas, lxbarth, Eevans, jkroll, Planemad, Hardikj, Wikidata-bugs, Jdouglas, bearND, aude, Tobias1984, Manybubbles, Yurik, Jdforrester-WMF, Mbch331, Jay8g
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
