jenkins-bot has submitted this change. (
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/432772 )
Change subject: [IMPR] use roundrobin_generators to combine generators when
limit is given
......................................................................
[IMPR] use roundrobin_generators to combine generators when limit is given
Change-Id: I11957c120c911b701581e60d6c3b965980478999
---
M pywikibot/pagegenerators/__init__.py
M pywikibot/pagegenerators/_factory.py
2 files changed, 24 insertions(+), 4 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/pagegenerators/__init__.py
b/pywikibot/pagegenerators/__init__.py
index fa30b7d..4608b6c 100644
--- a/pywikibot/pagegenerators/__init__.py
+++ b/pywikibot/pagegenerators/__init__.py
@@ -432,8 +432,10 @@
-intersect Work on the intersection of all the provided generators.
--limit When used with any other argument -limit:n specifies a set
- of pages, work on no more than n pages in total.
+-limit When used with any other argument ``-limit:n``
+ specifies a set of pages, work on no more than n
+ pages in total. If used with multiple generators,
+ pages are yielded in a roundrobin way.
-namespaces Filter the page generator to only yield pages in the
-namespace specified namespaces. Separate multiple namespace
diff --git a/pywikibot/pagegenerators/_factory.py
b/pywikibot/pagegenerators/_factory.py
index 0b42c73..f32c8e3 100644
--- a/pywikibot/pagegenerators/_factory.py
+++ b/pywikibot/pagegenerators/_factory.py
@@ -54,7 +54,11 @@
WikidataSPARQLPageGenerator,
)
from pywikibot.tools.collections import DequeGenerator
-from pywikibot.tools.itertools import filter_unique, intersect_generators
+from pywikibot.tools.itertools import (
+ filter_unique,
+ intersect_generators,
+ roundrobin_generators,
+)
HANDLER_RETURN_TYPE = Union[None, bool, Iterable['pywikibot.page.BasePage']]
@@ -200,6 +204,10 @@
.. versionchanged:: 7.3
set the instance variable :attr:`is_preloading` to True or False.
+ .. versionchanged:: 8.0
+ if ``limit`` option is set and multiple generators are given,
+ pages are yieded in a :func:`roundrobin
+ <tools.itertools.roundrobin_generators>` way.
:param gen: Another generator to be combined with
:param preload: preload pages using PreloadingGenerator
@@ -247,7 +255,8 @@
# By definition no duplicates are possible.
dupfiltergen = intersect_generators(*self.gens)
else:
- dupfiltergen = _filter_unique_pages(itertools.chain(*self.gens))
+ combine = roundrobin_generators if self.limit else itertools.chain
+ dupfiltergen = _filter_unique_pages(combine(*self.gens))
# Add on subpage filter generator
if self.subpage_max_depth is not None:
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/432772
To unsubscribe, or for help writing mail filters, visit
https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I11957c120c911b701581e60d6c3b965980478999
Gerrit-Change-Number: 432772
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <[email protected]>
Gerrit-Reviewer: John Vandenberg <[email protected]>
Gerrit-Reviewer: Xqt <[email protected]>
Gerrit-Reviewer: Zhuyifei1999 <[email protected]>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
_______________________________________________
Pywikibot-commits mailing list -- [email protected]
To unsubscribe send an email to [email protected]
