https://bugzilla.wikimedia.org/show_bug.cgi?id=66600
[email protected] changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |64749 --- Comment #4 from [email protected] --- (In reply to Dan Andreescu from comment #0) > The cron job that updates the mobile dashboards has not been working since > June 2nd. We've enabled logging with > https://gerrit.wikimedia.org/r/#/c/138884/ but we don't see a log file yet. I think, those are actually two separate issues: * Cron job not working, and * Seeing no log file. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * Cron job not working: For me, the relevant command bails out with: [...] Generating Uploads cancelled by user (per day) Traceback (most recent call last): File "/srv/limn-mobile-data/generate.py", line 321, in <module> dg.execute() File "/srv/limn-mobile-data/generate.py", line 171, in execute ok = self.generate_graph_full(key, value) File "/srv/limn-mobile-data/generate.py", line 285, in generate_graph_full self.save_graph_as_csv(key, headers, rows) File "/srv/limn-mobile-data/generate.py", line 303, in save_graph_as_csv with open(csv_filename, 'wb') as csv_file: IOError: [Errno 13] Permission denied: '/a/limn-public-data/mobile/datafiles/cancelled-uploads.csv' As that file can only be writted by 610:wikidev [1], but the job being run as stats [2] (without group "wikidev") the error looks plausible. Owner 610 is bug 64749. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * Seeing no log file: The patch that added logging for this job was broken, and broke puppet on stat1003 [3]. Broken puppet should have been fixed by https://gerrit.wikimedia.org/r/139775 (see comment 1 and comment 2). Thanks springle for the prompt merge. Puppet is ok again on stat1003. But it seems that the log generation will fail regardless, due to log file creation happening on the wrong end. The corresponding patch is https://gerrit.wikimedia.org/r/#/c/139785/ (for which I did not yet escalate merging, as it does not seem to affect this or other services ... just logging. So it can sit in the queue.) [1] _________________________________________________________________ qchris@stat1003 // jobs: 0 // time: 06:26:20 // exit code: 0 cwd: ~ ll /a/limn-public-data/mobile/datafiles/cancelled-uploads.csv -rw-rw-r-- 1 610 wikidev 7956 Jun 2 18:07 /a/limn-public-data/mobile/datafiles/cancelled-uploads.csv [2] https://git.wikimedia.org/blob/operations%2Fpuppet/79e00984efb6e23cfc59e9fab13be14223baab7a/manifests%2Fmisc%2Fstatistics.pp#L696 [3] For example [05:21:18] <icinga-wm> PROBLEM - Puppet freshness on stat1003 is CRITICAL: Last successful Puppet run was Fri 13 Jun 2014 20:03:25 UTC in http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-operations/20140616.txt -- You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ Wikibugs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
