jayvdb added a comment.
For the general problem wrt in `PreloadingGenerator`, using `groupsize=1` solves the problem. Pages will be emitted immediately. This was previously called `step`, but was renamed in https://phabricator.wikimedia.org/rPWBC65b75413be0168b73a4576c2de94d1ad58a1dc9c. Also any solution to this bug must not be designed only for `-liverecentchanges` ; any page generators could be emitting changes 'live' from the wiki, and we already have another one `RepeatingGenerator`. The bugs we need to solve are 1. within GeneratorFactory, when PreloadingGenerator is used when `-liverecentchanges` (or similar) has been specified. e.g. a very specific problem in `getCombinedGenerator`, where if self.articlefilter_list: dupfiltergen = RegexBodyFilterPageGenerator( PreloadingGenerator(dupfiltergen), self.articlefilter_list) We need to not call `PreloadingGenerator` there if `-liverecentchanges` is activated, or call it with `groupsize=1`. IMO the simplest solution is for GeneratorFactory to have an attribute `groupsize` initially None, which `-liverecentchanges` would set to `1`. 2. scripts often call `PreloadingGenerator` after the factory has given them a page generator, any thus they will inadvertently be trying to preload liverecentchanges. My preferred solution for that is that the script tells the GeneratorFactory, in the constructor, that the script believes preloading would be beneficial, and the script does not add its own `PreloadingGenerator`. (which is already implemented in https://gerrit.wikimedia.org/r/#/c/172577/2 ) TASK DETAIL https://phabricator.wikimedia.org/T135331 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Xqt, jayvdb Cc: jayvdb, gerritbot, Aklapper, Zppix, pywikibot-bugs-list, valhallasw, Xqt, Lewizho99, Mdupont, Maathavan _______________________________________________ pywikibot-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
