ArielGlenn added a comment.
WIkidata has been moved to the list of "big" wikis which means jobs run in
parallel now, cutting down on processing time. It truly is growing leaps and
bounds.
We should be able to do two runs a month as we just did in August, one full run
including revision
Hydriz added a subscriber: Hydriz.
Hydriz added a comment.
Just a update on the dump progress for the last few wikidatawiki dumps:
mysql> SELECT subject,dumpdate,progress FROM archive WHERE
subject="wikidatawiki";
+--++--+
| subject | dumpdate |
Lydia_Pintscher added a comment.
We are talking about moving from how often to once a month?
TASK DETAIL
https://phabricator.wikimedia.org/T85970
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
username.
EMAIL PREFERENCES
Lydia_Pintscher added a comment.
I don't think it is ok for our users to do it less often than it is at the
moment.
TASK DETAIL
https://phabricator.wikimedia.org/T85970
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
username.
EMAIL
Lydia_Pintscher added a comment.
Agreed :)
TASK DETAIL
https://phabricator.wikimedia.org/T85970
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
username.
EMAIL PREFERENCES
hoo added a comment.
We have to distinguish here: Our json dumps will keep running on a weekly
schedule, but the other dumps are apparently monthly (and we need those rather
more often than less often).
TASK DETAIL
https://phabricator.wikimedia.org/T85970
REPLY HANDLER ACTIONS
Reply to
ezachte added a comment.
@Lydia_Pintscher are you referring to wikidata? For all practical purposes the
current rate is once a month for wikidata anyway. One exception since June
2014: two runs completed in Aug.
TASK DETAIL
https://phabricator.wikimedia.org/T85970
REPLY HANDLER ACTIONS
ezachte added a comment.
If budget allows let's run dumps more often. But one monthly cycle starting on
the first date of each month is better than a 3 week continuous cycle (which
grows in length every month anyway). The current scheme frustrates all those
users who want monthly stats with
ArielGlenn added a comment.
Wikidata needs to be moved to the 'big wikis' queue at some point and there are
other not so small wikis that should be moved over as well. A question for
wikiata dumps users; is once a month often enough for the run or do people need
two complete runs? Once a
ArielGlenn added a comment.
I had a look at the previous failed runs to get a sense of what was going on.
The causes are various: the dataset1001 host or the snapshot host being
rebooted for security updates; the db server being either hung or having been
depooled (I didn't check which); a
ezachte added a subscriber: ezachte.
ezachte added a comment.
Here are recent dump times and outcomes:
wiki,date,run time in hms,run time in secs,,result,
wikidatawiki,20140612,195:38:06,704286,-,done
wikidatawiki,20140705,278:35:10,1002910,-,done
wikidatawiki,20140731,78:04:51,281091,-,failed
11 matches
Mail list logo
