ArielGlenn has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/338754 )
Change subject: for page content dumps, for each numbered part do either ranges
or whole dump
......................................................................
for page content dumps, for each numbered part do either ranges or whole dump
also:
get rid of inaccurate fixme
get rid of extraneous int conversions for value that's already an int
Bug: T158517
Change-Id: I015584c265a84e39c2c1082052554453779e8f88
---
M xmldumps-backup/dumps/xmljobs.py
1 file changed, 11 insertions(+), 11 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/dumps
refs/changes/54/338754/1
diff --git a/xmldumps-backup/dumps/xmljobs.py b/xmldumps-backup/dumps/xmljobs.py
index 2168b67..fd1dcd6 100644
--- a/xmldumps-backup/dumps/xmljobs.py
+++ b/xmldumps-backup/dumps/xmljobs.py
@@ -474,23 +474,23 @@
todo = outfiles
else:
todo = []
+ missing_ranges = self.find_missing_ranges(stub_ranges,
checkpoint_ranges)
parts = self.get_fileparts_list()
for partnum in parts:
if not [1 for chkpt_range in checkpoint_ranges
- if int(chkpt_range[2]) == int(partnum)]:
- # no page ranges covered by checkpoints for a
particular
- # file part (subjob) so do that output file the
- # regular way
+ if int(chkpt_range[2]) == partnum]:
+ # entire page range for a particular file part
(subjob)
+ # is missing so generate the regular output file
todo.extend([outfile for outfile in outfiles
- if int(outfile.partnum) ==
int(partnum)])
-
- missing = self.find_missing_ranges(stub_ranges,
checkpoint_ranges)
- todo.extend([self.chkpt_file_from_page_range((first,
last), partnum)
- for (first, last, partnum) in missing])
-
+ if int(outfile.partnum) == partnum])
+ else:
+ # at least some page ranges are covered, just do
those that
+ # are missing (maybe none are and list is empty)
+
todo.extend([self.chkpt_file_from_page_range((first, last), part)
+ for (first, last, part) in
missing_ranges
+ if int(part) == partnum])
else:
# do the missing files only
- # FIXME public or private depending on the wiki!
todo = [outfile for outfile in outfiles
if not
os.path.exists(runner.dump_dir.filename_public_path(outfile))]
--
To view, visit https://gerrit.wikimedia.org/r/338754
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I015584c265a84e39c2c1082052554453779e8f88
Gerrit-PatchSet: 1
Gerrit-Project: operations/dumps
Gerrit-Branch: master
Gerrit-Owner: ArielGlenn <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
