I recently migrated most of pywikibot jobs from grid engine to k8s and to my surprise, it was actually quite easy. I have lots of tasks (been running bots since 2008). So much that it showed redaction in the total number of jobs in SGE.
One thing that helped me that I collapsed everything into a bash file, like hourly.sh and made that a job (it's better to have one pod doing a batch of work as overhead of creating a pod is bigger than a SGE job) Also, make sure your bash file is readable by others (see the doc, chmod ug+x hourlysh), that would have saved me a bit of time. Thank you for doing this. I use only the new fancy infrastructure from now on ^^ On Fri, Feb 18, 2022 at 8:28 PM Arturo Borrero Gonzalez < [email protected]> wrote: > > On 2/16/22 17:34, Russell Blau wrote: > > Also, it is not possible to load Pywikibot in the tf-python39 runtime > because a required module (requests, fromhttps://python-requests.org) is > not available. What is the process for requesting (no pun intended) that > this (or any other resource) be added to the image? > > See some documentation here: > > > https://wikitech.wikimedia.org/wiki/Help:Toolforge/Python#Kubernetes_python_jobs > > I just created it, and may need some polishing, but it should work! > > We will review pywikibot specific workflows and documents soon. > > > regards. > -- > Arturo Borrero Gonzalez > Site Reliability Engineer > Wikimedia Cloud Services > Wikimedia Foundation > _______________________________________________ > Cloud mailing list -- [email protected] > List information: > https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ > -- Amir (he/him)
_______________________________________________ Cloud mailing list -- [email protected] List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
