[MediaWiki-commits] [Gerrit] Add bad dates for projectcount aggregations - change (analytics/aggregator)

[QChris (Code Review)]

QChris has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/172238

Change subject: Add bad dates for projectcount aggregations
......................................................................

Add bad dates for projectcount aggregations

Bad dates depend strongly on the use data source. So bad dates should
not go into this repository.

This is only parked code that is used for backfilling since 2008.

Change-Id: I733bd9d75df96167564ac1e889bcc8e130565fb6
---
M aggregator/projectcounts.py
1 file changed, 28 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/analytics/aggregator 
refs/changes/38/172238/1

diff --git a/aggregator/projectcounts.py b/aggregator/projectcounts.py
index 2f95370..a543230 100644
--- a/aggregator/projectcounts.py
+++ b/aggregator/projectcounts.py
@@ -32,6 +32,32 @@
 
 CSV_LINE_ENDING = '\r\n'
 
+BAD_DATES = [
+    datetime.date(2008, 10, 21),
+    datetime.date(2008, 10, 22),
+    datetime.date(2009, 10, 14),
+    datetime.date(2009, 10, 15),
+    datetime.date(2009, 10, 16),
+    datetime.date(2009, 11, 22),
+    datetime.date(2010, 1, 23),
+    datetime.date(2010, 1, 24),
+    datetime.date(2010, 2, 8),
+    datetime.date(2010, 2, 26),
+    datetime.date(2010, 6, 27),
+    datetime.date(2010, 6, 28),
+    datetime.date(2010, 7, 5),
+    datetime.date(2010, 7, 7),
+    datetime.date(2010, 7, 8),
+    datetime.date(2010, 7, 9),
+    datetime.date(2010, 7, 10),
+    datetime.date(2011, 1, 4),
+    datetime.date(2011, 6, 14),
+    datetime.date(2013, 7, 23),
+    datetime.date(2013, 7, 24),
+    datetime.date(2014, 1, 6),
+    datetime.date(2014, 8, 28),
+    ]
+
 cache = {}
 
 
@@ -165,6 +191,8 @@
                     csv_data[date_str] = line.strip() + CSV_LINE_ENDING
 
         for date in util.generate_dates(first_date, last_date):
+            if date in BAD_DATES:
+                continue
             date_str = date.isoformat()
             logging.debug("Updating csv '%s' for date '%s'" % (
                 dbname, str(date)))

-- 
To view, visit https://gerrit.wikimedia.org/r/172238
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I733bd9d75df96167564ac1e889bcc8e130565fb6
Gerrit-PatchSet: 1
Gerrit-Project: analytics/aggregator
Gerrit-Branch: master
Gerrit-Owner: QChris <christ...@quelltextlich.at>

_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits