[Wikidata-bugs] [Maniphest] [Changed Subscribers] T124418: Investigate massive increase in htmlCacheUpdate jobs in Dec/Jan

2016-06-17 Thread BBlack 
BBlack added a subscriber: GWicke.BBlack added a comment.
@aaron and @GWicke - both patches sound promising, thanks for digging into this topic!TASK DETAILhttps://phabricator.wikimedia.org/T124418EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: BBlackCc: GWicke, ArielGlenn, Krinkle, Peter, EBernhardson, Smalyshev, gerritbot, Legoktm, Addshore, daniel, hoo, aude, Lydia_Pintscher, JanZerebecki, MZMcBride, Luke081515, aaron, faidon, Joe, ori, BBlack, Aklapper, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, Mbch331, Jay8g___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T124418: Investigate massive increase in htmlCacheUpdate jobs in Dec/Jan

2016-06-01 Thread ori 
ori added a subscriber: EBernhardson.
ori added a comment.


  @EBernhardson made it so when a job fragments into a number of child jobs, 
each child job has the same request ID as its parent. This also makes it 
possible to aggregate PURGEs by individual parent job:
  
  Top PURGE issuers by orig. request id
  -
  
  | Orig. request id | Percent |
  |  | --- |
  | V07acApAEK4AAA-p8t4AAACK | 24.43%  |
  | V09G2wpAIDEAAAoe07kAAABE | 10.05%  |
  | V0xKtwpAEK8AAFSGAicAAACI | 3.59%   |
  | V0xKtQpAEIEAAAvjcJIY | 3.45%   |
  | V08jWwpAEK4AAHA7yEsV | 3.44%   |
  | V08P2QpAADUAABlSU9sB | 2.49%   |
  | V08o4QpAICwAABlKSrUAAACJ | 2.39%   |
  | fe4a69ca01a1c0e76f782cd9| 2.34%   |
  | 73fd09f28d554f56484bed95| 1.97%   |
  | ceeea6f37cefae6a4a86478b| 1.90%   |
  
  This tells us that of the 10,718,138 PURGEs issued during the observation 
period, approximately 2,618,441 PURGEs were issued by a single htmlCacheUpdate 
job for svwiki.
  
  svwiki only has 6,464,684 pages in total (of which 3,199,790 are content 
pages), so that means nearly half of svwiki was purged over the course of four 
hours.
  
  My hunch is that purges are not getting correctly de-duplicated, so a single 
page can get purged multiple times by children of a single root job.
  
  I wonder whether we correctly de-duplicate the case when a template is 
indirectly included on a page via two or more different intermediate templates. 
For example, on enwiki, [[Template:Navbox]] is included in [[Barack Obama]] 
both through [[Template:US Presidents]] and [[Template:Current U.S. Governors]].

TASK DETAIL
  https://phabricator.wikimedia.org/T124418

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: ori
Cc: EBernhardson, Smalyshev, gerritbot, Legoktm, Addshore, daniel, hoo, aude, 
Lydia_Pintscher, JanZerebecki, MZMcBride, Luke081515, aaron, faidon, Joe, ori, 
BBlack, Aklapper, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, Mbch331, 
Jay8g



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T124418: Investigate massive increase in htmlCacheUpdate jobs in Dec/Jan

2016-01-25 Thread Lydia_Pintscher 
Lydia_Pintscher added a subscriber: daniel.

TASK DETAIL
  https://phabricator.wikimedia.org/T124418

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lydia_Pintscher
Cc: daniel, hoo, aude, Lydia_Pintscher, JanZerebecki, MZMcBride, Luke081515, 
Denniss, aaron, faidon, Joe, ori, BBlack, Aklapper, Wikidata-bugs, Mbch331



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T124418: Investigate massive increase in htmlCacheUpdate jobs in Dec/Jan

2016-01-25 Thread Lydia_Pintscher 
Lydia_Pintscher added subscribers: Lydia_Pintscher, aude, hoo.

TASK DETAIL
  https://phabricator.wikimedia.org/T124418

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lydia_Pintscher
Cc: hoo, aude, Lydia_Pintscher, JanZerebecki, MZMcBride, Luke081515, Denniss, 
aaron, faidon, Joe, ori, BBlack, Aklapper, Wikidata-bugs, Mbch331



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T124418: Investigate massive increase in htmlCacheUpdate jobs in Dec/Jan

2016-01-25 Thread BBlack 
BBlack added a subscriber: JanZerebecki.

TASK DETAIL
  https://phabricator.wikimedia.org/T124418

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: BBlack
Cc: JanZerebecki, MZMcBride, Luke081515, Denniss, aaron, faidon, Joe, ori, 
BBlack, Aklapper, Wikidata-bugs, aude, Mbch331



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs