Hello Ottomata,
I'd like you to do a code review. Please visit
https://gerrit.wikimedia.org/r/84974
to review the following change.
Change subject: Compute geowiki data only for full days
......................................................................
Compute geowiki data only for full days
When running the cronjob on 2013-09-18 12:00, the job computed the
data for both 2013-09-17, and for 2013-09-18. The computed data for
2013-09-18 however can only contain edits up to 12:00 [1]. The data
cannot contain an edit made for example on 2013-09-18 14:00. Hence,
the last data point (i.e.: the data for the current date) might be
wrong. The data for 2013-09-18 gets recomputed (and corrected) when
running the cronjob on 2013-09-19. However, it's unnecessary to
compute the data for 2013-09-18 already on 2013-09-18, if we know that
it's bound to be stale anyways.
Hence, we shift the computed dates back one day. When running the job
on 2013-09-18 12:00, it now computes the data for 2013-09-16, and
2013-09-17. These days have been passed and their data is known to no
longer change.
[1] Or some minutes later depending on how long the job runs.
Change-Id: I7e194a60b2b5247d10601f14dfd5a583eac80239
---
M manifests/misc/statistics.pp
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/74/84974/1
diff --git a/manifests/misc/statistics.pp b/manifests/misc/statistics.pp
index d6f81aa..fa0678e 100644
--- a/manifests/misc/statistics.pp
+++ b/manifests/misc/statistics.pp
@@ -946,7 +946,7 @@
minute => 0,
hour => 12,
user => $geowiki_user,
- command => "/usr/bin/python
${geowiki_path}/geowiki/process_data.py -o ${$geowiki_backups_path} --wpfiles
${geowiki_path}/geowiki/data/all_ids.tsv --daily --start=`date --date='-1 day'
+\\%Y-\\%m-\\%d` --end=`date --date='1 day' +\\%Y-\\%m-\\%d`
--source_sql_cnf=${geowiki_path}/.globaldev.my.cnf
--dest_sql_cnf=${geowiki_path}/.research.my.cnf",
+ command => "/usr/bin/python
${geowiki_path}/geowiki/process_data.py -o ${$geowiki_backups_path} --wpfiles
${geowiki_path}/geowiki/data/all_ids.tsv --daily --start=`date --date='-2 day'
+\\%Y-\\%m-\\%d` --end=`date --date='0 day' +\\%Y-\\%m-\\%d`
--source_sql_cnf=${geowiki_path}/.globaldev.my.cnf
--dest_sql_cnf=${geowiki_path}/.research.my.cnf",
require => File[$geowiki_backups_path],
}
}
--
To view, visit https://gerrit.wikimedia.org/r/84974
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7e194a60b2b5247d10601f14dfd5a583eac80239
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: QChris <[email protected]>
Gerrit-Reviewer: Ottomata <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
