[Bug 66600] Cron that runs mobile queries is dead
https://bugzilla.wikimedia.org/show_bug.cgi?id=66600 Dan Andreescu dandree...@wikimedia.org changed: What|Removed |Added Whiteboard|u=dandree...@wikimedia.org |u=caistleit...@wikimedia.or |c=General/Unknown p=0 |g c=General/Unknown p=0 |s=2014-06-12|s=2014-06-12 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 66600] Cron that runs mobile queries is dead
https://bugzilla.wikimedia.org/show_bug.cgi?id=66600 Gerrit Notification Bot gerritad...@wikimedia.org changed: What|Removed |Added Status|NEW |PATCH_TO_REVIEW -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 66600] Cron that runs mobile queries is dead
https://bugzilla.wikimedia.org/show_bug.cgi?id=66600 --- Comment #1 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 139775 had a related patch set uploaded by QChris: Fix log target for mobile apps data job https://gerrit.wikimedia.org/r/139775 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 66600] Cron that runs mobile queries is dead
https://bugzilla.wikimedia.org/show_bug.cgi?id=66600 --- Comment #2 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 139775 merged by Springle: Fix log target for mobile apps data job https://gerrit.wikimedia.org/r/139775 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 66600] Cron that runs mobile queries is dead
https://bugzilla.wikimedia.org/show_bug.cgi?id=66600 --- Comment #3 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 139785 had a related patch set uploaded by QChris: Ensure log file for mobile data job exists https://gerrit.wikimedia.org/r/139785 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 66600] Cron that runs mobile queries is dead
https://bugzilla.wikimedia.org/show_bug.cgi?id=66600 christ...@quelltextlich.at changed: What|Removed |Added Depends on||64749 --- Comment #4 from christ...@quelltextlich.at --- (In reply to Dan Andreescu from comment #0) The cron job that updates the mobile dashboards has not been working since June 2nd. We've enabled logging with https://gerrit.wikimedia.org/r/#/c/138884/ but we don't see a log file yet. I think, those are actually two separate issues: * Cron job not working, and * Seeing no log file. * Cron job not working: For me, the relevant command bails out with: [...] Generating Uploads cancelled by user (per day) Traceback (most recent call last): File /srv/limn-mobile-data/generate.py, line 321, in module dg.execute() File /srv/limn-mobile-data/generate.py, line 171, in execute ok = self.generate_graph_full(key, value) File /srv/limn-mobile-data/generate.py, line 285, in generate_graph_full self.save_graph_as_csv(key, headers, rows) File /srv/limn-mobile-data/generate.py, line 303, in save_graph_as_csv with open(csv_filename, 'wb') as csv_file: IOError: [Errno 13] Permission denied: '/a/limn-public-data/mobile/datafiles/cancelled-uploads.csv' As that file can only be writted by 610:wikidev [1], but the job being run as stats [2] (without group wikidev) the error looks plausible. Owner 610 is bug 64749. * Seeing no log file: The patch that added logging for this job was broken, and broke puppet on stat1003 [3]. Broken puppet should have been fixed by https://gerrit.wikimedia.org/r/139775 (see comment 1 and comment 2). Thanks springle for the prompt merge. Puppet is ok again on stat1003. But it seems that the log generation will fail regardless, due to log file creation happening on the wrong end. The corresponding patch is https://gerrit.wikimedia.org/r/#/c/139785/ (for which I did not yet escalate merging, as it does not seem to affect this or other services ... just logging. So it can sit in the queue.) [1] _ qchris@stat1003 // jobs: 0 // time: 06:26:20 // exit code: 0 cwd: ~ ll /a/limn-public-data/mobile/datafiles/cancelled-uploads.csv -rw-rw-r-- 1 610 wikidev 7956 Jun 2 18:07 /a/limn-public-data/mobile/datafiles/cancelled-uploads.csv [2] https://git.wikimedia.org/blob/operations%2Fpuppet/79e00984efb6e23cfc59e9fab13be14223baab7a/manifests%2Fmisc%2Fstatistics.pp#L696 [3] For example [05:21:18] icinga-wm PROBLEM - Puppet freshness on stat1003 is CRITICAL: Last successful Puppet run was Fri 13 Jun 2014 20:03:25 UTC in http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-operations/20140616.txt -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 66600] Cron that runs mobile queries is dead
https://bugzilla.wikimedia.org/show_bug.cgi?id=66600 --- Comment #5 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 139785 merged by Ottomata: Ensure log file for mobile data job exists https://gerrit.wikimedia.org/r/139785 -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 66600] Cron that runs mobile queries is dead
https://bugzilla.wikimedia.org/show_bug.cgi?id=66600 christ...@quelltextlich.at changed: What|Removed |Added Status|PATCH_TO_REVIEW |RESOLVED Resolution|--- |FIXED --- Comment #6 from christ...@quelltextlich.at --- Jobs are running again. Logs get written. Charts look good [1]. Resolved. [1] All the graphs of the 'Edits daily' tab [2] look wrong. The graphs drop for 2014-06-12 and die off on 2014-06-13. But that's not a script error, but it closely matches EventLogging's upgrade for the MobileWebEditing schema from revision 7675117 to revision 8599025. To verify that, run select timestamp, count(*) from MobileWebEditing_7675117 group by left(timestamp,8) order by timestamp desc limit 20; select timestamp, count(*) from MobileWebEditing_8599025 group by left(timestamp,8) order by timestamp asc limit 20; against the EventLogging database. Switching schemata is offered in https://gerrit.wikimedia.org/r/#/c/139889/ but I am defering to mobile to decide, when they want to switch to the new schema. [2] http://mobile-reportcard.wmflabs.org/#edits_daily-graphs-tab -- You are receiving this mail because: You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 66600] Cron that runs mobile queries is dead
https://bugzilla.wikimedia.org/show_bug.cgi?id=66600 Dan Andreescu dandree...@wikimedia.org changed: What|Removed |Added Priority|Unprioritized |Highest Assignee|wikibugs-l@lists.wikimedia. |dandree...@wikimedia.org |org | Whiteboard||u=dandree...@wikimedia.org ||c=General/Unknown p=0 ||s=2014-06-12 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
