On Thu, Oct 31, 2013 at 1:57 AM, Yuri Astrakhan <yastrak...@wikimedia.org>wrote:
> == Background == > ZeroRatedMobileAccess has always depended on MobileFrontend and used it > liberally, including calls to its classes. However, it was done in hooks > called by MF so Zero simply stopped working in absence of MF. This, > however, changed in [1] where Zero started using a ResourceLoader module > from MF. > > Yuri, Thanks for the detailed writeup and for monitoring / quickly resolving the problem. Cheers, Katie > == What happened == > At 23:02pm UTC, after deploying Zero extension updates, fatal monitor was > flooded with: > > -- Fatal error: Class 'MFResourceLoaderModule' not found in /usr/local/ > > apache/common-local/php-1.23wmf1/includes/resourceloader/ResourceLoader.phpon > line 408 > > The issue was tracked down to Wikidata having MobileFrontend disabled, > while ZeroRatedMobileAccess was enabled. It didn't impact page views > directly, however all load.php calls that requested the startup module > caused fatals because it attempted to instantiate MFResourceLoader class > and couldn't find it. As a consequence, people might have seen pages > without styles or scripts. > > A number of people (MaxSem, Reedy, Roan, and Greg, and possibly others) > gave great assistance to track down the issue and rapidly disable the > ZeroRatedMobileAccess extension in Wikidata. Furthermore, mobile > configuration [2] will add an additional guard against calling > ZeroRatedMobileAccess.php unless it's explicitly within the context of MF. > > Thank you to everyone!!! > > == Timeline == > All times in UTC > > * 22:48 Zero 1.22wmf22 deployed, no errors > * 23:02 Zero 1.23wmf1 deployed, first errors appear - initially unnoticed > * 23:08 A small MobileFrontend change deployed > * 23:09 Errors noticed, initially linked with MobileFrontend push > * 23:17 Max reverts his MobileFrontend changes, errors don't go away > * 23:22 Problem narrowed down > * 23:27 Fix deployed > > == Recomendations == > * Allow a bit more time between deployments and observe fatalmonitor before > and after > * Ensure Zero extension checks if Mobile extension is loaded before > enabling itself if it relies on MFResourceLoader. > > ------ > [1] https://gerrit.wikimedia.org/r/#/c/83133 > [2] https://gerrit.wikimedia.org/r/#/c/92811 > _______________________________________________ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- @wikimediadc / @wikidata _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
