| mobrovac added a comment. |
In T200594#4469972, @Smalyshev wrote:Aren't clients making requests to Kartotherian? If they are, then X-Client-IP will be set to the external (to our prod environment) client issuing the request.
Yes. But when Kartotherian calls to WDQS, it calls external Varnish endpoint (now), which might replace X-Client-IP - not 100% sure about it, needs checking, but I think judging from the logs that's what is happening since the IP I see in the logs is one of Kartotherian.
The flow is: client -> Nginx -> Varnish -> Kartotherian -> WDQS. Hence, when Kartotherian receives the request, that request will have the external client's IP in the X-Client-IP header and that's immutable regardless of what Kartotherian does next (i.e. whether the requests it issues pass through Varnish or not).
The ideal solution here would be for Kartotherian to call directly WDQS (via its discovery end point mentioned in the previous comments) and supply it the aforementioned header if the idea is to throttle requests on WDQS.
On the technical side, though, changing the UA for every client is IMHO a bad practice and should not even be considered a solution here.
I am not sure why. What's wrong with changing the UA?
But I am not hard-set on the UA, that'd just make it work automatically, but if it puts it in other place as long as Java can extract it it'd be fine I think.
By definition, the User-Agent header identifies the user agent making the request, not the client. Furthermore, putting semi-random information in that header (like IPs) makes analytics, metrics, logging and other use-cases harder, amongst other things.
The differentiator here is not who initiates the query, but who writes the query itself. As I understand, Kartotherian runs arbitrary user queries, which can be very heavy and unoptimized, and we have little visibility into who produces these queries.
Ah I see. In that case, perhaps it would worth investigating whether the set of queries could be restricted somehow?
Cc: mobrovac, MSantos, Gehel, Aklapper, MaxSem, Pnorman, Mholloway, Smalyshev, Amatissart, Lahi, Gq86, Looniverse, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Orienteerix, merbst, LawExplorer, debt, JGirault, Jonas, phabyogi, Xmlizer, Susannaanas, lxbarth, Eevans, jkroll, Planemad, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Yurik, Jdforrester-WMF, Mbch331, Jay8g
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
