I am OK with this decision too. I personally am finding `uv` to be nice and super user friendly. No need to go through the past list of commands to install some package, `uv sync` just simplifies things amazingly.
It is always a good idea to have one "solid" way of doing things rather than multiple options which we promise would work but can run into issues. Instead, using something as the "way" as a whole community would be brilliant! Thanks & Regards, Amogh Desai On Sat, Mar 22, 2025 at 4:30 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > Quick question: We will still make the branches for constraints as we > used to, correct? > > Quick answer: Correct. > > Longer answer with more context and future vision. > > The constraints mechanism is currently used for both: > > 1) development (effectively pinning the dependencies for most PRs and only > bumping them to latest in canary builds and in some PRs) > 2) user facing reproducible installation with `pip` > > By - eventually - switching to uv we will be able to replace 1) with > uv.lock, but still continue to generate the constraints as we did for 2). > This means that we will "bind" development with `uv` but the "user" side of > the installation of airflow will not require UV in any scenario, we are > following "de-facto" standard of remote constraint mechanism (it's not > really part of standard but something that both `pip` and `uv` support). > > Eventually we might be able to replace 2) for our users when it becomes > standard and implemented by multiple tools like `pip`. This is essentially > what PEP-751 is about: https://peps.python.org/pep-0751/ - Bret, TP and > other packaging maintainers recognize our case as valid and they are > designing the standard format that will be usable in both 1) and 2) cases. > > Our "constraint" mechanism is essentially a "poor man" version of PEP-751 > implemented by me, in the absence of a similar mechanism in pip and > packaging standards. This is about the last thing that the packaging team > is catching up with all the needs Airflow had with packaging since Airflow > 2 and providers introduction. > > This will be different than uv.lock, the current proposal is to use .toml > format for it and - from what I know - uv team is involved and committed - > you can see Charlie and Zanieb from uv very actively participating in those > 3 super long threads: > > Original restart of PEP-751 (there was a previous PEP=665 > https://peps.python.org/pep-0665/ that was rejected) > * https://discuss.python.org/t/pep-751-lock-files-again/59173 (354 posts) > > Reboot: > * https://discuss.python.org/t/pep-751-now-with-graphs/69721/254 (253 > posts) > > And one last reboot : > * https://discuss.python.org/t/pep-751-one-last-time/77293/133 (136 posts) > > I have not actively participated in those discussions - I was barely > following them, and I think Charlie and Zanieb are already looking at > Airflow as one of the most complex cases they will need to serve, so they > are for sure doing a good job in the discussions led by Bret, Paul Moor and > involving many other people, and I have a full trust the standard they will > come with will be good for us :). You can see by the sheer number of posts > how thorough and detailed the discussion was. > > So - once PEP-751 is approved and implemented by `pip`, `uv`, `poetry` and > others, we will be able to replace the uv.lock mechanism with it and also > use it for reproducible installation for our users. > > But this is again - a few years from now. > > J. > > > On Sat, Mar 22, 2025 at 11:31 AM Abhishek Bhakat > <abhishek.bha...@astronomer.io.invalid> wrote: > > > +1 for uv. > > Quick question: We will still make the branches for constraints as we > used > > to, correct? > > > > - Avi > > > > On Sat, Mar 22, 2025 at 7:24 AM Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > Also one more important point - if we drop `pip` support - we will be > > able > > > to rely on `uv.lock` for constraint generation. Currently we have our > > own > > > mechanism of updating the constraint files, but if we go "full-in" with > > > `uv` - we should be able to use `uv.lock` for this purpose. It means > that > > > we will be able to commit `uv.lock` (currently it's .gitignored and > > > everyone has its own uv.lock file). > > > > > > As of two weeks or so apparently dependabot supports uv > > > > > > > > > https://github.blog/changelog/2025-03-13-dependabot-version-updates-now-support-uv-in-general-availability/ > > > though I > > > am not sure if this is full support ( > > > > > > https://docs.astral.sh/uv/guides/integration/dependency-bots/#uvlock-output > > > mentions that it is work in progress and the issue > > > https://github.com/dependabot/dependabot-core/issues/10478 mentioned > > there > > > is still open). > > > > > > Once we go "full-in" with `uv` and have full support for `uv.lock` > > upgrade > > > with dependabot, we will be able to replace the current "canary > > > auto-upgrade" with "Dependabot PR upgrade" mechanism, which should > allow > > > us to remove a lot of custom code I wrote and will allow to have more > > > "controlled" upgrades (say daily PR to upgrade deps). But details for > > that > > > is another discussion. > > > > > > J. > > > > > > > > > On Sat, Mar 22, 2025 at 7:44 AM Kunal Bhattacharya < > > kunal.ju...@gmail.com> > > > wrote: > > > > > > > +1 on dropping pip support and have simplified dependency management > > with > > > > uv. > > > > > > > > I agree that managing both is complicated and the quick start docs to > > > > navigate either uv or pip is not the easiest to follow through. I > > > strongly > > > > believe the setup will be orders of magnitude easier to follow along > > with > > > > just uv. > > > > > > > > Regards, > > > > Kunal Bhattacharya > > > > > > > > On Sat, 22 Mar 2025, 03:20 Jarek Potiuk, <ja...@potiuk.com> wrote: > > > > > > > > > Hello here, > > > > > > > > > > Following the "airflow-core" move I would like to ask what others > > think > > > > > about dropping `pip` as a way to set-up your local dev environment > > and > > > > > leaving only `uv`. > > > > > > > > > > *Context: * > > > > > > > > > > The `uv` tool from https://astral.sh/ is built in ruff and the way > > > they > > > > > approached python dev environments is inspired by rust cargo. I.e > > "one > > > > tool > > > > > to do it all". > > > > > > > > > > The uv tool does this: > > > > > > > > > > * automatically installs python when needed > > > > > * automatically creates venv when needed > > > > > * automatically syncs venv with your project dependencies when > needed > > > > > * supports workspaces - i.e. multiple python projects in a single > > repo, > > > > > which is something we started to use extensively recently and > > > culminated > > > > > with airflow-core separation > > > > > > > > > > This is all cool, and based mostly on implementing the standards > > agreed > > > > by > > > > > the Python Packaging team ( > https://peps.python.org/topic/packaging/) > > > - > > > > > which means that we do not force our users and developers to use > > > certain > > > > > tools, as long as they follow standards. One can use pip, poetry, > > > hatch, > > > > > uv, flit and a number of others as a front-end client to install > and > > > > manage > > > > > their local environment. > > > > > > > > > > This however breaks a bit with workspaces and dependency groups and > > > lock > > > > > files: > > > > > > > > > > * dependency groups (used for devel/ test dependencies) is already > > > > approved > > > > > https://peps.python.org/pep-0735/ but not implemented by `pip` yet > > > (next > > > > > version 25.1 will support it) > > > > > > > > > > * workspaces do not have a PEP proposal yet. Different tools > > implement > > > > them > > > > > in different ways - I work with Ofek, Hatch creator and he has his > > own > > > > > ideas and implementations, currently uv's workspaces (modelled > after > > > the > > > > > needs of ours) are working very well for us > > > > > > > > > > * uv.lock is an incarnation (by uv) of > > > https://peps.python.org/pep-0751/ > > > > > which is currently draft but there is a rumour it will be approved > > soon > > > > and > > > > > uv team is committed to support the PEP > > > > > > > > > > *Problem:* > > > > > > > > > > Currently, a number of decisions in our build/packaging files are > > > > > supporting the "generic" support. Everything we do (including > > workspace > > > > > installation) should be possible with `uv` > > > > > but we also have `pip` equivalent of it (for example you have a > > > sequence > > > > of > > > > > `pip install -e` commands that you can run to get the same result > as > > > `uv > > > > > sync`). And it costs a bit - hatch_build.py in the root of Airflow > is > > > > > unnecessarily complex to support it (with dynamic pre-installed > > > packages > > > > > and some other dynamic code. We could get rid of it and replace the > > > > dynamic > > > > > stuff with static pyproject.toml in the root folder if we do not > care > > > > about > > > > > `pip` installation for development. It would also simplify "quick > > > start" > > > > > docs if we only support `uv` with workspaces. > > > > > > > > > > The problem with `uv` is that it's an open-source, but privately > > > > controlled > > > > > (astral.sh) by a VC-backed company. We know (personally) and like > the > > > > > people behind `uv`, but at some future point in time, the ownership > > and > > > > > change of control might turn astral into > > not-that-open-source-friendly > > > > > (this is in a stark contrast with `pip`, `hatch`, `flit` - which > are > > > part > > > > > of the Python Software Foundation and under the Python Packaging > > > > Authority > > > > > working group. > > > > > > > > > > *Concerns* > > > > > > > > > > I personally have some reservations against exclusively supporting > > > `uv`, > > > > > but I recognise that it makes our packaging unnecessarily complex > if > > we > > > > > continue to support other workflows. It's great DevEX, fantastic > > > > > "Contributor Journey Optimisation" with `uv`. > > > > > > > > > > The risks, however, are small. If anything happens with `astral` - > > they > > > > > licence it with Apache 2 and MIT dual licensing, we - or anyone > else > > - > > > > > could fork it as it happened with multiple other projects in the > past > > > > > including Akka -> Pekko, Terraform -> OpenTofu and numerous others. > > > > > > > > > > And it does not affect our users. Packages published from us are > > > > > installable with anything (including pip, hatch, flit) because we > > > follow > > > > > standards, it's only about the dev tooling. And - if all things > fail, > > > we > > > > > can always redevelop our own "glue" (like we already do in breeze) > to > > > > allow > > > > > workspace or dependency groups to be easily usable. > > > > > > > > > > *Proposal* > > > > > > > > > > My proposal - for the sake of simplicity, better contributor's > > journey > > > - > > > > > for now to drop `pip` support and have `uv` as the only supported > dev > > > > > tooling. That would allow us to simplify docs and tooling support. > > > > > > > > > > I would love to hear what others think about it. > > > > > > > > > > J. > > > > > > > > > > > > > > >
