Tangent: is it worthwhile to establish a consensus for best practices with package pinning and package management for Python projects in the Wikimedia ecosystem? When I last worked on a python project (https://wikitech.wikimedia.org/wiki/Add_Link) I found it confusing that we have so many different tools and approaches for doing these things, and seems like we'd benefit from having a standard, supported way. (Or maybe that already exists and I haven't found it?)
Kosta > On 5. May 2023, at 13:51, Slavina Stefanova <[email protected]> wrote: > > Poetry is a modern lockfile-based packaging and dependency management tool > worth looking into. It also supports exporting dependencies into a > requirements.txt file, should you need that (nice if you want to containerize > an app without bloating the image with Poetry, for instance). > > https://python-poetry.org/ <https://python-poetry.org/> > > -- > Slavina Stefanova (she/her) > Software Engineer - Technical Engagement > > Wikimedia Foundation > > > On Fri, May 5, 2023 at 1:38 PM Sebastian Berlin > <[email protected] <mailto:[email protected]>> wrote: >> A word of warning: using `pip freeze` to populate requirements.txt can >> result in a hard to read (very long) file and other issues: >> https://medium.com/@tomagee/pip-freeze-requirements-txt-considered-harmful-f0bce66cf895. >> >> Sebastian Berlin >> Utvecklare/Developer >> Wikimedia Sverige (WMSE) >> >> E-post/E-Mail: [email protected] >> <mailto:[email protected]> >> Telefon/Phone: (+46) 0707 - 92 03 84 >> >> >> On Fri, 5 May 2023 at 13:17, Amir Sarabadani <[email protected] >> <mailto:[email protected]>> wrote: >>> You can also create an empty virtual env, install all requirements and then >>> do >>> pip freeze > requirements.txt >>> >>> That should take care of pinning >>> >>> Am Fr., 5. Mai 2023 um 13:11 Uhr schrieb Lucas Werkmeister >>> <[email protected] <mailto:[email protected]>>: >>>> For the general case of Python projects, I’d argue that a better solution >>>> is to adopt the lockfile pattern (package-lock.json, composer.lock, >>>> Cargo.lock, etc.) and pin all dependencies, and only update them when the >>>> new versions have been tested and are known to work. pip-tools >>>> <https://pypi.org/project/pip-tools/> can help with that, for example >>>> (requirements.in <http://requirements.in/> specifies “loose” dependencies; >>>> pip-compile creates a pinned requirements.txt; pip-sync installs it; >>>> pip-compile -U upgrades requirements.txt later; you check both >>>> requirements.in <http://requirements.in/> and requirements.txt into >>>> version control.) But I don’t know if that applies in your >>>> integration/config case. >>>> >>>> Am Do., 4. Mai 2023 um 18:08 Uhr schrieb Antoine Musso <[email protected] >>>> <mailto:[email protected]>>: >>>>> Hello, >>>>> >>>>> This is for python projects. >>>>> >>>>> Today, May 4th, urllib3 <https://pypi.org/project/urllib3/#history> has >>>>> released a new major version 2.0.2 which breaks the extremely popular >>>>> requests <https://pypi.org/project/requests/> library. >>>>> >>>>> The fix is to pin urllib3<2 to prevent the new major version from being >>>>> installed (example >>>>> <https://gerrit.wikimedia.org/r/c/integration/config/+/915736/1/tox.ini>). >>>>> >>>>> https://phabricator.wikimedia.org/T335977 >>>>> >>>>> Upstream issue: https://github.com/psf/requests/issues/6432 >>>>> >>>>> >>>>> >>>>> Antoine "hashar" Musso >>>>> Wikimedia Release Engineering >>>>> _______________________________________________ >>>>> Wikitech-l mailing list -- [email protected] >>>>> <mailto:[email protected]> >>>>> To unsubscribe send an email to [email protected] >>>>> <mailto:[email protected]> >>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/ >>>> >>>> >>>> -- >>>> Lucas Werkmeister (he/er) >>>> Software Engineer >>>> >>>> Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin >>>> Phone: +49 (0)30-577 11 62-0 >>>> https://wikimedia.de <https://wikimedia.de/> >>>> >>>> Imagine a world in which every single human being can freely share in the >>>> sum of all knowledge. Help us to achieve our vision! >>>> https://spenden.wikimedia.de <https://spenden.wikimedia.de/> >>>> >>>> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. >>>> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg >>>> unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt >>>> für Körperschaften I Berlin, Steuernummer 27/029/42207. >>>> _______________________________________________ >>>> Wikitech-l mailing list -- [email protected] >>>> <mailto:[email protected]> >>>> To unsubscribe send an email to [email protected] >>>> <mailto:[email protected]> >>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/ >>> >>> >>> -- >>> Amir (he/him) >>> >>> _______________________________________________ >>> Wikitech-l mailing list -- [email protected] >>> <mailto:[email protected]> >>> To unsubscribe send an email to [email protected] >>> <mailto:[email protected]> >>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/ >> _______________________________________________ >> Wikitech-l mailing list -- [email protected] >> <mailto:[email protected]> >> To unsubscribe send an email to [email protected] >> <mailto:[email protected]> >> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/ > _______________________________________________ > Wikitech-l mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
_______________________________________________ Wikitech-l mailing list -- [email protected] To unsubscribe send an email to [email protected] https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
