Erutuon created this task. Erutuon added a project: Pywikibot. Restricted Application added subscribers: pywikibot-bugs-list, Aklapper.
TASK DESCRIPTION So I had a list of pages in the English Wiktionary and wanted to list only the ones that belonged to a certain category. My script preloaded the pages themselves with `APISite.preloadpages` and then iterated over their categories with `BasePage.categories`. But this is very inefficient and takes a long time, because `BasePage.categories` has to send a separate request to get the list of categories for a page each time it is called. Ideally these would be preloaded along with the rest of the page properties so that the script sends only one request per group of pages. I searched through the Pywikibot docs <https://doc.wikimedia.org/pywikibot/master/api_ref/pywikibot.html> and didn't find any way to do this. I also asked @valhallasw on IRC and he confirmed that there's no way to do this currently. A good solution would be to add a parameter to `APISite.preloadpages` that would tell it to preload the categories using API:Categories <https://www.mediawiki.org/wiki/API:Categories> and make them accessible to `BasePage.categories` on the pages yielded by the iterator. I'm not familiar enough with the internals of Pywikibot to fully work this out though. @valhallasw on IRC kindly showed me some modifications to `site.py` that make `APISite.preloadpages` preload the categories and place the decoded JSON under `page._preloaded["categories"]` in the page objects, but that's not a long-term solution. TASK DETAIL https://phabricator.wikimedia.org/T241689 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Erutuon Cc: Aklapper, Erutuon, valhallasw, pywikibot-bugs-list, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Dalba, Masti, Alchimista, Rxy
_______________________________________________ pywikibot-bugs mailing list pywikibot-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
