> > Tangent: is it worthwhile to establish a consensus for best practices with > package pinning and package management for Python projects in the Wikimedia > ecosystem? When I last worked on a python project ( > https://wikitech.wikimedia.org/wiki/Add_Link) I found it confusing that > we have so many different tools and approaches for doing these things, and > seems like we'd benefit from having a standard, supported way. (Or maybe > that already exists and I haven't found it?)
I'm working on an "Essential Tools for Managing Python Development Environments <https://docs.google.com/document/d/1pl6QCDWGebGjRrHixgQNES8qRf-8PCzwLSsFj7jI1HY/edit#heading=h.i5ou2ywgmb09>" tutorial that will be published to the wikis when ready. Maybe that could be expanded upon? In my experience though, it can be hard to get people to agree on following a standard, especially when there are so many different options and many folks already have their favorite tools and workflows. But it would be nice to have a set of recommendations to reduce the cognitive load. -- Slavina Stefanova (she/her) Software Engineer - Technical Engagement Wikimedia Foundation On Fri, May 5, 2023 at 5:18 PM Kosta Harlan <khar...@wikimedia.org> wrote: > Tangent: is it worthwhile to establish a consensus for best practices with > package pinning and package management for Python projects in the Wikimedia > ecosystem? When I last worked on a python project ( > https://wikitech.wikimedia.org/wiki/Add_Link) I found it confusing that > we have so many different tools and approaches for doing these things, and > seems like we'd benefit from having a standard, supported way. (Or maybe > that already exists and I haven't found it?) > > Kosta > > On 5. May 2023, at 13:51, Slavina Stefanova <sstefan...@wikimedia.org> > wrote: > > Poetry is a modern lockfile-based packaging and dependency management tool > worth looking into. It also supports exporting dependencies into a > requirements.txt file, should you need that (nice if you want to > containerize an app without bloating the image with Poetry, for instance). > > https://python-poetry.org/ <https://python-poetry.org/> > > -- > Slavina Stefanova (she/her) > Software Engineer - Technical Engagement > > Wikimedia Foundation > > > On Fri, May 5, 2023 at 1:38 PM Sebastian Berlin < > sebastian.ber...@wikimedia.se> wrote: > >> A word of warning: using `pip freeze` to populate requirements.txt can >> result in a hard to read (very long) file and other issues: >> https://medium.com/@tomagee/pip-freeze-requirements-txt-considered-harmful-f0bce66cf895 >> . >> >> *Sebastian Berlin* >> Utvecklare/*Developer* >> Wikimedia Sverige (WMSE) >> >> E-post/*E-Mail*: sebastian.ber...@wikimedia.se >> Telefon/*Phone*: (+46) 0707 - 92 03 84 >> >> >> On Fri, 5 May 2023 at 13:17, Amir Sarabadani <ladsgr...@gmail.com> wrote: >> >>> You can also create an empty virtual env, install all requirements and >>> then do >>> pip freeze > requirements.txt >>> >>> That should take care of pinning >>> >>> Am Fr., 5. Mai 2023 um 13:11 Uhr schrieb Lucas Werkmeister < >>> lucas.werkmeis...@wikimedia.de>: >>> >>>> For the general case of Python projects, I’d argue that a better >>>> solution is to adopt the lockfile pattern (package-lock.json, >>>> composer.lock, Cargo.lock, etc.) and pin *all* dependencies, and only >>>> update them when the new versions have been tested and are known to work. >>>> pip-tools <https://pypi.org/project/pip-tools/> can help with that, >>>> for example (requirements.in specifies “loose” dependencies; >>>> pip-compile creates a pinned requirements.txt; pip-sync installs it; >>>> pip-compile >>>> -U upgrades requirements.txt later; you check both requirements.in and >>>> requirements.txt into version control.) But I don’t know if that >>>> applies in your integration/config case. >>>> >>>> Am Do., 4. Mai 2023 um 18:08 Uhr schrieb Antoine Musso <has...@free.fr >>>> >: >>>> >>>>> Hello, >>>>> >>>>> This is for python projects. >>>>> >>>>> Today, May 4th, urllib3 <https://pypi.org/project/urllib3/#history> >>>>> has released a new major version 2.0.2 which breaks the extremely popular >>>>> requests <https://pypi.org/project/requests/> library. >>>>> >>>>> The fix is to pin urllib3<2 to prevent the new major version from >>>>> being installed (example >>>>> <https://gerrit.wikimedia.org/r/c/integration/config/+/915736/1/tox.ini> >>>>> ). >>>>> >>>>> https://phabricator.wikimedia.org/T335977 >>>>> >>>>> Upstream issue: https://github.com/psf/requests/issues/6432 >>>>> >>>>> >>>>> Antoine "hashar" Musso >>>>> Wikimedia Release Engineering >>>>> _______________________________________________ >>>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org >>>>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org >>>>> >>>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/ >>>> >>>> >>>> >>>> -- >>>> Lucas Werkmeister (he/er) >>>> Software Engineer >>>> >>>> Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin >>>> Phone: +49 (0)30-577 11 62-0 >>>> https://wikimedia.de >>>> >>>> Imagine a world in which every single human being can freely share in >>>> the sum of all knowledge. Help us to achieve our vision! >>>> https://spenden.wikimedia.de >>>> >>>> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. >>>> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter >>>> der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für >>>> Körperschaften I Berlin, Steuernummer 27/029/42207. >>>> _______________________________________________ >>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org >>>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org >>>> >>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/ >>> >>> >>> >>> -- >>> Amir (he/him) >>> >>> _______________________________________________ >>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org >>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org >>> >>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/ >> >> _______________________________________________ >> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org >> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org >> >> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/ > > _______________________________________________ > Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org > To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org > https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/ > > > _______________________________________________ > Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org > To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org > https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
_______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
