jenkins-bot has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/473940 )
Change subject: [FIX] site.preloadpages: split pagelist in at most max_ids
elements
......................................................................
[FIX] site.preloadpages: split pagelist in at most max_ids elements
In site.preloadpages(), max_ids, i.e. the API query limit is
computed after pagelist has been splitted in chunks of groupsize
elements.
Pagelist shall be splitted in chunks of min(groupsize, max_ids)
elements.
Bug: T209111
Change-Id: I7ae76e26d2500cc8abea679b5e80465fb07690c9
---
M pywikibot/site.py
1 file changed, 7 insertions(+), 7 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site.py b/pywikibot/site.py
index 8033eeb..58fb89f 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -3370,7 +3370,13 @@
rvprop = ['ids', 'flags', 'timestamp', 'user', 'comment', 'content']
- for sublist in itergroup(pagelist, groupsize):
+ parameter = self._paraminfo.parameter('query+info', 'prop')
+ if self.logged_in() and self.has_right('apihighlimits'):
+ max_ids = int(parameter['highlimit'])
+ else:
+ max_ids = int(parameter['limit']) # T78333, T161783
+
+ for sublist in itergroup(pagelist, min(groupsize, max_ids)):
# Do not use p.pageid property as it will force page loading.
pageids = [str(p._pageid) for p in sublist
if hasattr(p, '_pageid') and p._pageid > 0]
@@ -3388,12 +3394,6 @@
rvgen = api.PropertyGenerator(props, site=self)
rvgen.set_maximum_items(-1) # suppress use of "rvlimit" parameter
- parameter = self._paraminfo.parameter('query+info', 'prop')
- if self.logged_in() and self.has_right('apihighlimits'):
- max_ids = int(parameter['highlimit'])
- else:
- max_ids = int(parameter['limit']) # T78333, T161783
-
if len(pageids) == len(sublist) and len(set(pageids)) <= max_ids:
# only use pageids if all pages have them
rvgen.request['pageids'] = set(pageids)
--
To view, visit https://gerrit.wikimedia.org/r/473940
To unsubscribe, or for help writing mail filters, visit
https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7ae76e26d2500cc8abea679b5e80465fb07690c9
Gerrit-Change-Number: 473940
Gerrit-PatchSet: 2
Gerrit-Owner: Mpaa <[email protected]>
Gerrit-Reviewer: John Vandenberg <[email protected]>
Gerrit-Reviewer: Xqt <[email protected]>
Gerrit-Reviewer: jenkins-bot (75)
_______________________________________________
Pywikibot-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/pywikibot-commits
