| Smalyshev added a comment. |
So it sounds like the only place to fix this is within blazegraph itself.?
One of the solutions may be to try and figure out how to do faster updates. Another would be to add servers to production cluster to spread query load. The pattern seems to be very dependant on query load, and does not happen on non-public cluster.
Again looking at the edit rate and creation rate on wikidata in relation to the lags on wdqs I don't really see a correlation.
I don't think it's edit load by itself. It's edit load PLUS data size PLUS query load that pushes the public servers over the edge where we see lags. Edit load alone doesn't do it. That said, I notice significant increase pattern here: https://grafana.wikimedia.org/dashboard/db/wikidata-edits?refresh=1m&panelId=7&fullscreen&orgId=1 (note that both RC and Kafka stream process revisions, so RC count is the metric to look at).
f the result of this is that the request timeout etc are lowered it might be worth revisiting
Yes, but it's a project for which currently we don't have enough bandwidth. We're
working on changing that. As a side note, I have a bot that is meant to do queued stored requests: https://commons.wikimedia.org/wiki/User:TabulistBot but looks like nobody is interested in using it.
Cc: Tarrow, Krenair, Addshore, TerraCodes, Liuxinyu970226, Aklapper, Smalyshev, Nandana, Lahi, Gq86, Darkminds3113, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Vali.matei, D3r1ck01, Jonas, Xmlizer, Volker_E, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Dinoguy1000, Manybubbles, Mbch331, Jay8g
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
