+1 to this idea. I considered whether we should use `pylock.toml` instead, but it might not address the issue we intend to resolve based on my reading.
Best, Wei > On Aug 19, 2025, at 5:51 AM, Ferruzzi, Dennis <ferru...@amazon.com.INVALID> > wrote: > > I don't have any particular experience, but based just on what you shared > there, it does feel like a next logical step in the migration to these tools. > Off-loading work and simplifying our CI is also always a plus in my opinion. > > > - ferruzzi > > > ________________________________ > From: Jarek Potiuk <ja...@potiuk.com> > Sent: Monday, August 18, 2025 4:14 AM > To: dev@airflow.apache.org > Subject: [EXT] [DISCUSS] Replace constraints with uv.lock mechanisms for dev > env freeze > > CAUTION: This email originated from outside of the organization. Do not click > links or open attachments unless you can confirm the sender and know the > content is safe. > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne > cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas > confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le > contenu ne présente aucun risque. > > > > Hello here, > > *TL;DR: I would love to hear comments on the proposal to switch to using > uv.lock for our dev env dependency freeze* > > While working on some small follow-ups after prek upgrade, I think I > finally figured out how we can start using `uv.lock` locally and merge it > into our constraint mechanism effectively - without > over-burdening committers with merging a lot of "dependabot-like" PRs. > > The key problem that needed to be resolved was who and when would update > the uv.lock regularly. > > I captured my thoughts in https://github.com/apache/airflow/issues/54609 > > But let me copy the content of it below. I would love to hear the comments > - here or in the GitHub issue if you will (I will bring important things > from the issue here if they will be raised there): > > > *Currently constraints are used for three purposes:* > 1) to take snapshot of latest "working" version of dependencies > 2) to limit PRs to "latest known good constraints" - so that newly released > packages are not failing most PRs, but fail canary builds instead > 3) to give our users a way to install released airflow in reproducible way > > However, after we switched to uv, the case 2) can be better achieved with > uv.lock. Currently we .gitignore uv.lock, but we could commit it to the > repository and handle constraint generation differently > > * constraints could be generated using latest lock > * canary builds could use the --resolution highest as today and diverge > from latest "frozen" uv.lock > * all prs would use the uv.lock and do not use --resolution-highest in > those PRs. This has the added benefit that uv sync in PR would > automatically upgrade dependencies if the dependencies in pyproject.toml > are conflicting with uv.lock - but it would not use --resolution highest, > which means that the PRs that modify dependeNcies do not have to get latest > dependencies when they are run - that would limit the number of PRs that > are experiencing "main/canary" problems > > *Problem* > > The only problem to solve is who and when would upgrade the uv.lock and > commit it to the repo. Currently constraints are automatically bumped after > successful canary runs, but they are in orphan un-protected branches, while > uv.lock shares the code with main airflow repo, and we need a "human with > ICLA" to review and merge commits there (ASF requirement) > > > *Solution* > Proposed solution is to add such a uv.lock upgrade to our > "upgrade-important-versions" prek hook. We run it in CI in canary builds > and we fail the builds when new versions of important installers change > (pip, Python base images, golang) - this happens often-enough to be > eventually-consistent pretty fast, but rarely enough to make it into > annoying "everyday" upgrade chore. > > Regular PRs - especially those that would change dependencies will also > modify uv.lock - for example when a new dependency is added or removed or > bumped. This means that the uv.lock will be generally quite regularly > bumped and "eventually consistent" with constraints. > > > *Benefits:* > * simplification of breeze build image with constraints (currently it uses > complex uv pip install with --constraint, and it could simply use uv sync > --all-packages instead if our uv.lock is checked in and maintained > * better synchronization of the local venv. Currently uv sync does not > guarantee to have the same version of dependencies in local venv. The only > way to guarantee it now is to use breeze image - build with constraints or > run complex commands with constraints added when you run uv sync > * less "3rd-party release" impact on failing PRs. if PRs will not touch the > dependency specification where the new release is released, it will > continue using the dependency stored in .lock and not the latest dependency > released > > > *Cons:* > * Probably we should pay more attention to run upgrade-important-versions > regularly and even add an option there to manually just perform the sync > (for example that will be useful as part of the release procedure, where > release manager would have to make sure to run the > upgrade-important-versions check when preparing release notes. But it has > the advantage that it will be a conscious step that should actually help in > making sure that constraints are updated. > > * sometimes PRs of people will have "big" changes to uv.lock - when they > are updating the dependency and alongside the uv.lock will also bump many > other unrelated dependencies - this could be mitigated by splitting it into > "lock upgrade" and "dependency update" but workflow of that is not very > convenient and natural. > > J. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
