OliverKeyes has uploaded a new change for review.
https://gerrit.wikimedia.org/r/266220
Change subject: Use a single date for Sys.Date()-1
......................................................................
Use a single date for Sys.Date()-1
This passes Sys.Date()-1 into main() from run(). The idea
is very simple; at the moment we're launching the cron job
relatively late in the day to cover for the possibility
of EventLogging replication lag. Each main() function
calculates Sys.Date()-1 on its own. When combined, this
raises the improbable but possible scenario where some
of the code takes long enough to run that the Sys.Date changes,
and some scripts run for day N and others for N+1.
Instead, with this change we'll be passing N through consistently,
for all files, and still retain the ability to run the main()
functions as self-contained blocks. Everyone wins.
Change-Id: Ia4b98b72f115128c435ec7da6610527b869fc91e
---
M run.R
1 file changed, 4 insertions(+), 3 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/wikimedia/discovery/golden
refs/changes/20/266220/1
diff --git a/run.R b/run.R
index 8bdeb2c..6174368 100644
--- a/run.R
+++ b/run.R
@@ -19,18 +19,19 @@
# If the user has not provided dates, just run each file.
if(!length(dates)){
- file_status <- unlist(lapply(source_files, function(x){
+ date <- (Sys.Date() - 1)
+ file_status <- unlist(lapply(source_files, function(x, date){
tryCatch({
source(x)
check_dir(base_path)
- main()
+ main(date)
}, error = function(e){
print(x)
print(e$message)
return(FALSE)
})
return(TRUE)
- }))
+ }, date = date))
} else {
# If the user has provided dates, we need to do more clever stuff.
data_files <- list.files(write_dirs, full.names = TRUE, pattern =
"\\.tsv$")
--
To view, visit https://gerrit.wikimedia.org/r/266220
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia4b98b72f115128c435ec7da6610527b869fc91e
Gerrit-PatchSet: 1
Gerrit-Project: wikimedia/discovery/golden
Gerrit-Branch: master
Gerrit-Owner: OliverKeyes <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
