+1 for uv. Quick question: We will still make the branches for constraints as we used to, correct?
- Avi On Sat, Mar 22, 2025 at 7:24 AM Jarek Potiuk <ja...@potiuk.com> wrote: > Also one more important point - if we drop `pip` support - we will be able > to rely on `uv.lock` for constraint generation. Currently we have our own > mechanism of updating the constraint files, but if we go "full-in" with > `uv` - we should be able to use `uv.lock` for this purpose. It means that > we will be able to commit `uv.lock` (currently it's .gitignored and > everyone has its own uv.lock file). > > As of two weeks or so apparently dependabot supports uv > > https://github.blog/changelog/2025-03-13-dependabot-version-updates-now-support-uv-in-general-availability/ > though I > am not sure if this is full support ( > https://docs.astral.sh/uv/guides/integration/dependency-bots/#uvlock-output > mentions that it is work in progress and the issue > https://github.com/dependabot/dependabot-core/issues/10478 mentioned there > is still open). > > Once we go "full-in" with `uv` and have full support for `uv.lock` upgrade > with dependabot, we will be able to replace the current "canary > auto-upgrade" with "Dependabot PR upgrade" mechanism, which should allow > us to remove a lot of custom code I wrote and will allow to have more > "controlled" upgrades (say daily PR to upgrade deps). But details for that > is another discussion. > > J. > > > On Sat, Mar 22, 2025 at 7:44 AM Kunal Bhattacharya <kunal.ju...@gmail.com> > wrote: > > > +1 on dropping pip support and have simplified dependency management with > > uv. > > > > I agree that managing both is complicated and the quick start docs to > > navigate either uv or pip is not the easiest to follow through. I > strongly > > believe the setup will be orders of magnitude easier to follow along with > > just uv. > > > > Regards, > > Kunal Bhattacharya > > > > On Sat, 22 Mar 2025, 03:20 Jarek Potiuk, <ja...@potiuk.com> wrote: > > > > > Hello here, > > > > > > Following the "airflow-core" move I would like to ask what others think > > > about dropping `pip` as a way to set-up your local dev environment and > > > leaving only `uv`. > > > > > > *Context: * > > > > > > The `uv` tool from https://astral.sh/ is built in ruff and the way > they > > > approached python dev environments is inspired by rust cargo. I.e "one > > tool > > > to do it all". > > > > > > The uv tool does this: > > > > > > * automatically installs python when needed > > > * automatically creates venv when needed > > > * automatically syncs venv with your project dependencies when needed > > > * supports workspaces - i.e. multiple python projects in a single repo, > > > which is something we started to use extensively recently and > culminated > > > with airflow-core separation > > > > > > This is all cool, and based mostly on implementing the standards agreed > > by > > > the Python Packaging team (https://peps.python.org/topic/packaging/) > - > > > which means that we do not force our users and developers to use > certain > > > tools, as long as they follow standards. One can use pip, poetry, > hatch, > > > uv, flit and a number of others as a front-end client to install and > > manage > > > their local environment. > > > > > > This however breaks a bit with workspaces and dependency groups and > lock > > > files: > > > > > > * dependency groups (used for devel/ test dependencies) is already > > approved > > > https://peps.python.org/pep-0735/ but not implemented by `pip` yet > (next > > > version 25.1 will support it) > > > > > > * workspaces do not have a PEP proposal yet. Different tools implement > > them > > > in different ways - I work with Ofek, Hatch creator and he has his own > > > ideas and implementations, currently uv's workspaces (modelled after > the > > > needs of ours) are working very well for us > > > > > > * uv.lock is an incarnation (by uv) of > https://peps.python.org/pep-0751/ > > > which is currently draft but there is a rumour it will be approved soon > > and > > > uv team is committed to support the PEP > > > > > > *Problem:* > > > > > > Currently, a number of decisions in our build/packaging files are > > > supporting the "generic" support. Everything we do (including workspace > > > installation) should be possible with `uv` > > > but we also have `pip` equivalent of it (for example you have a > sequence > > of > > > `pip install -e` commands that you can run to get the same result as > `uv > > > sync`). And it costs a bit - hatch_build.py in the root of Airflow is > > > unnecessarily complex to support it (with dynamic pre-installed > packages > > > and some other dynamic code. We could get rid of it and replace the > > dynamic > > > stuff with static pyproject.toml in the root folder if we do not care > > about > > > `pip` installation for development. It would also simplify "quick > start" > > > docs if we only support `uv` with workspaces. > > > > > > The problem with `uv` is that it's an open-source, but privately > > controlled > > > (astral.sh) by a VC-backed company. We know (personally) and like the > > > people behind `uv`, but at some future point in time, the ownership and > > > change of control might turn astral into not-that-open-source-friendly > > > (this is in a stark contrast with `pip`, `hatch`, `flit` - which are > part > > > of the Python Software Foundation and under the Python Packaging > > Authority > > > working group. > > > > > > *Concerns* > > > > > > I personally have some reservations against exclusively supporting > `uv`, > > > but I recognise that it makes our packaging unnecessarily complex if we > > > continue to support other workflows. It's great DevEX, fantastic > > > "Contributor Journey Optimisation" with `uv`. > > > > > > The risks, however, are small. If anything happens with `astral` - they > > > licence it with Apache 2 and MIT dual licensing, we - or anyone else - > > > could fork it as it happened with multiple other projects in the past > > > including Akka -> Pekko, Terraform -> OpenTofu and numerous others. > > > > > > And it does not affect our users. Packages published from us are > > > installable with anything (including pip, hatch, flit) because we > follow > > > standards, it's only about the dev tooling. And - if all things fail, > we > > > can always redevelop our own "glue" (like we already do in breeze) to > > allow > > > workspace or dependency groups to be easily usable. > > > > > > *Proposal* > > > > > > My proposal - for the sake of simplicity, better contributor's journey > - > > > for now to drop `pip` support and have `uv` as the only supported dev > > > tooling. That would allow us to simplify docs and tooling support. > > > > > > I would love to hear what others think about it. > > > > > > J. > > > > > >
