ori added a subscriber: EBernhardson. ori added a comment.
@EBernhardson made it so when a job fragments into a number of child jobs, each child job has the same request ID as its parent. This also makes it possible to aggregate PURGEs by individual parent job: Top PURGE issuers by orig. request id ------------------------------------- | Orig. request id | Percent | | ------------------------ | ------- | | V07acApAEK4AAA-p8t4AAACK | 24.43% | | V09G2wpAIDEAAAoe07kAAABE | 10.05% | | V0xKtwpAEK8AAFSGAicAAACI | 3.59% | | V0xKtQpAEIEAAAvjcJIAAAAY | 3.45% | | V08jWwpAEK4AAHA7yEsAAAAV | 3.44% | | V08P2QpAADUAABlSU9sAAAAB | 2.49% | | V08o4QpAICwAABlKSrUAAACJ | 2.39% | | fe4a69ca01a1c0e76f782cd9 | 2.34% | | 73fd09f28d554f56484bed95 | 1.97% | | ceeea6f37cefae6a4a86478b | 1.90% | This tells us that of the 10,718,138 PURGEs issued during the observation period, approximately 2,618,441 PURGEs were issued by a single htmlCacheUpdate job for svwiki. svwiki only has 6,464,684 pages in total (of which 3,199,790 are content pages), so that means nearly half of svwiki was purged over the course of four hours. My hunch is that purges are not getting correctly de-duplicated, so a single page can get purged multiple times by children of a single root job. I wonder whether we correctly de-duplicate the case when a template is indirectly included on a page via two or more different intermediate templates. For example, on enwiki, [[Template:Navbox]] is included in [[Barack Obama]] both through [[Template:US Presidents]] and [[Template:Current U.S. Governors]]. TASK DETAIL https://phabricator.wikimedia.org/T124418 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: ori Cc: EBernhardson, Smalyshev, gerritbot, Legoktm, Addshore, daniel, hoo, aude, Lydia_Pintscher, JanZerebecki, MZMcBride, Luke081515, aaron, faidon, Joe, ori, BBlack, Aklapper, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, Mbch331, Jay8g _______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
