| jcrespo added a comment. |
how would we generate the kind of estimates you would need in order to sign off on this type of change?
Measure the QPS writes/rows written/percentage of write IOPS you have now, evaluate what is the increase with the new method, and scale with a worst-case-scenario factor. Or think on how you would do it on a separate set of servers, and I would instantly accept it because you would at least reduce the write load by half. I do not approve or deny things, I provide feedback and I would love if more developers had into account and discussed physical or architecture issues.
Personally, I'd like to see watchlists on client wikis notify you when a change is made to a item that actually affects one of the pages you watch
That seems like a good fit to combine it with the purchase of the externals servers for the T126641: [RFC] Devise plan for a cross-wiki watchlist back-end feature. Budgets for next fiscal year are being prepared right now I would ask as soon as possible for them.
without the ability to join
What is the problem? I see 3 possibilities for that :
- You are doing OLTP-like queries, so generally speaking you convert SELECT a JOIN B into SELECT A; SELECT B IN (). which should be almost as performant (except for an extra round trip) because you should only be selecting a few rows
- You are doing analytics, so you just just a consolidated slave with all data though multi-source replication, as I mentioned before
- You are running large (millon-sized) ranges on production- just don't do that in the first place
Future cases/hypothetical cases can be discussed later, we can backtrack, it is easy to move things around with no downtime. It is not easy to separate later. Also, new servers usually means better, more powerful servers! Take advantage of that if you can.
This is a distributed environment- we have to keep the writes low so there are capacity for reads, otherwise replication load balancing do not work.
Cc: Halfak, jcrespo, TomT0m, Hall1467, hoo, zhuyifei1999, Eloquence, Lydia_Pintscher, Sannita, Ainali, Liuxinyu970226, MZMcBride, Ricordisamoa, Micru, jayvdb, Daniel_Mietchen, Tobi_WMDE_SW, Legoktm, Abraham, Wikidata-bugs, liangent, jeremyb, aude, Candalua, Bianjiang, Aklapper, DixonD, daniel, D3r1ck01, Izno, Mbch331
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
