https://bugzilla.wikimedia.org/show_bug.cgi?id=54406
--- Comment #9 from Gabriel Wicke <[email protected]> --- (In reply to comment #6) > (In reply to comment #3) > > We dequeue Parsoid jobs in a throttled manner to avoid overloading the API > > during edit spikes. This means that abnormal edit rates especially to > > templates can create a large backlog of jobs in the Parsoid queue. > > Where does Parsoid fit in to the general MediaWiki ecosystem? Are Parsoid > jobs > generated on every edit? If so, why? After an edit, the Parsoid HTML for each affected article is generated / updated with jobs that perform HTTP requests to the Parsoid cluster. This ensures that requests from VisualEditor and other users are normally served straight from cache. We have been processing all edits from all Wikipedias since June. As expected, VE deployments have not made a noticeable difference to the load on the Parsoid cluster. It seems that the Parsoid dequeue rate was slightly lower than the average enqueue rate since the end of July, which allowed the job backlog to build up a bit. During MZMcBride's null edit episode the backlog doubled in size. Since that stopped yesterday, the enwiki Parsoid job queue has drained by 10% (200k jobs). So overall, the Parsoid job dequeue rate is slightly too low to absorb abnormal edit rates in a timely manner. It might be sufficient to slightly de-throttle the Parsoid dequeue rate while keeping an eye on the API cluster load (https://ganglia.wikimedia.org/latest/?r=hour&cs=&ce=&m=cpu_report&s=by+name&c=API+application+servers+eqiad&h=&host_regex=&max_graphs=0&tab=m&vn=&sh=1&z=small&hc=4). -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. _______________________________________________ Wikibugs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
