This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new f2a3e5feac7 Add --frozen to uv run commands in run_tests and document
uv lock behavior (#64699)
f2a3e5feac7 is described below
commit f2a3e5feac7c4d5a34f3691daf766ae84b130486
Author: Jarek Potiuk <[email protected]>
AuthorDate: Sat Apr 4 15:35:52 2026 +0200
Add --frozen to uv run commands in run_tests and document uv lock behavior
(#64699)
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
---
contributing-docs/07_local_virtualenv.rst | 75 ++++++++++------------
.../13_airflow_dependencies_and_extras.rst | 11 +++-
dev/breeze/src/airflow_breeze/utils/run_tests.py | 13 +++-
3 files changed, 54 insertions(+), 45 deletions(-)
diff --git a/contributing-docs/07_local_virtualenv.rst
b/contributing-docs/07_local_virtualenv.rst
index e6a0409ac9d..b75db8d4dbd 100644
--- a/contributing-docs/07_local_virtualenv.rst
+++ b/contributing-docs/07_local_virtualenv.rst
@@ -267,57 +267,52 @@ for the provider is as simple as running:
uv run pytest
-Installing "golden" version of dependencies
--------------------------------------------
+Locked versions of dependencies
+-------------------------------
-Whatever virtualenv solution you use, when you want to make sure you are using
the same
-version of dependencies as in main, you can install recommended version of the
dependencies by using pip:
-constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as ``constraint``
file. This might be useful
-to avoid "works-for-me" syndrome, where you use different version of
dependencies than the ones
-that are used in main, CI tests and by other contributors.
+The ``uv.lock`` file is committed to the Airflow repository and is used by
``uv sync`` to ensure
+consistent dependency versions across all developers. When you run ``uv
sync``, it uses the lock file
+to install exact dependency versions, so you don't need to pass constraint
files manually.
-There are different constraint files for different python versions. For
example this command will install
-all basic devel requirements and requirements of google provider as last
successfully tested for Python 3.10:
-
-.. code:: bash
-
- uv pip install -e ".[devel,google]" \
- --constraint
"https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.10.txt";
+The ``uv sync`` command prefers the locked versions of dependencies from
``uv.lock``. It will only
+attempt to resolve new dependencies when ``pyproject.toml`` files change (e.g.
when a new dependency
+is added or version bounds are modified). This means that day-to-day ``uv
sync`` is fast and
+deterministic — it simply installs what the lock file specifies without
re-resolving the dependency
+tree.
+If you want to make sure that ``uv sync`` does not update your lock file at
all (for example in CI
+or when running tests), you can pass the ``--frozen`` flag:
-In the future we will utilise ``uv.lock`` to manage dependencies and
constraints, but for the moment we do not
-commit ``uv.lock`` file to Airflow repository because we need to figure out
automation of updating the ``uv.lock``
-very frequently (few times a day sometimes). With Airflow's 700+ dependencies
it's all but guaranteed that we
-will have 3-4 changes a day and currently automated constraints generation
mechanism in ``canary`` build keeps
-constraints updated, but for ASF policy reasons we cannot update ``uv.lock``
in the same way - but work is in
-progress to fix it.
-
-Make sure to use latest main for such installation, those constraints are
"development constraints" and they
-are refreshed several times a day to make sure they are up to date with the
latest changes in the main branch.
+.. code:: bash
-Note that this might not always work as expected, because the constraints are
not always updated
-immediately after the dependencies are updated, sometimes there is a very
recent change (few hours, rarely more
-than a day) which still runs in ``canary`` build and constraints will not be
updated until the canary build
-succeeds. Usually what works in this case is running your install command
without constraints.
+ uv sync --frozen
-You can upgrade just airflow, without paying attention to provider's
dependencies by using
-the 'constraints-no-providers' constraint files. This allows you to keep
installed provider dependencies
-and install to latest supported ones by pure Airflow core.
+This will fail if the lock file is out of date with respect to
``pyproject.toml``, rather than
+silently updating it. This is useful when you want to guarantee fully
reproducible environments.
-.. code:: bash
+Cooldown via ``exclude-newer``
+..............................
- uv pip install -e ".[devel]" \
- --constraint
"https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.10.txt";
+The ``[tool.uv]`` section in the top-level ``pyproject.toml`` sets
``exclude-newer = "4 days"``.
+This acts as a cooldown period — when ``uv`` resolves new dependencies, it
ignores package versions
+released in the last 4 days. This protects against broken or yanked releases
that might otherwise
+immediately break the dependency resolution for all developers. When ``uv``
writes the lock file, it
+records the resolved ``exclude-newer`` timestamp so that subsequent ``uv
sync`` calls use the same
+cutoff, ensuring consistency across machines.
-These are examples of the development options available with the local
virtualenv in your IDE:
+Constraints generated from the lock file
+.........................................
-* local debugging;
-* Airflow source view;
-* auto-completion;
-* documentation support;
-* unit tests.
+Airflow also publishes traditional ``pip``-style constraint files (see
+`Airflow dependencies and extras <13_airflow_dependencies_and_extras.rst>`_
for details). When
+installing Airflow from sources, these constraint files are generated directly
from ``uv.lock`` using
+``uv export --frozen``, which converts the lock file into a flat list of
pinned versions suitable for
+``pip install --constraint``. This ensures that both the ``uv sync`` workflow
and the ``pip`` constraint
+workflow install the same dependency versions.
-This document describes minimum requirements and instructions for using a
standalone version of the local virtualenv.
+The lock file is updated regularly — whenever dependencies are changed via any
``pyproject.toml`` and
+when ``breeze ci upgrade`` is run. Make sure to use the latest main branch to
get the most
+up-to-date ``uv.lock``.
Running Tests
-------------
diff --git a/contributing-docs/13_airflow_dependencies_and_extras.rst
b/contributing-docs/13_airflow_dependencies_and_extras.rst
index cd43e486ffb..7c59ce25d17 100644
--- a/contributing-docs/13_airflow_dependencies_and_extras.rst
+++ b/contributing-docs/13_airflow_dependencies_and_extras.rst
@@ -318,17 +318,17 @@ example ``pip install apache-airflow==1.10.2
Werkzeug<1.0.0``)
There are several sets of constraints we keep:
-* 'constraints' - these are constraints generated by matching the current
Airflow version from sources
+* ``constraints`` - these are constraints generated by matching the current
Airflow version from sources
and providers that are installed from PyPI. Those are constraints used by
the users who want to
install Airflow with pip, they are named
``constraints-<PYTHON_MAJOR_MINOR_VERSION>.txt``.
-* "constraints-source-providers" - these are constraints generated by using
providers installed from
+* ``constraints-source-providers`` - these are constraints generated by using
providers installed from
current sources. While adding new providers their dependencies might change,
so this set of providers
is the current set of the constraints for Airflow and providers from the
current main sources.
Those providers are used by CI system to keep "stable" set of constraints.
They are named
``constraints-source-providers-<PYTHON_MAJOR_MINOR_VERSION>.txt``
-* "constraints-no-providers" - these are constraints generated from only
Apache Airflow, without any
+* ``constraints-no-providers`` - these are constraints generated from only
Apache Airflow, without any
providers. If you want to manage Airflow separately and then add providers
individually, you can
use them. Those constraints are named
``constraints-no-providers-<PYTHON_MAJOR_MINOR_VERSION>.txt``.
@@ -375,6 +375,11 @@ using ``constraints-no-providers`` constraint files as
well.
--constraint
"https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.10.txt";
+These constraint files are generated from the ``uv.lock`` file committed in
the repository, using
+``uv export --frozen`` to convert the lock file into a flat list of pinned
versions suitable for
+``pip install --constraint``. This means the constraint files always reflect
the same dependency
+versions that ``uv sync`` installs for developers.
+
The ``constraints-<PYTHON_MAJOR_MINOR_VERSION>.txt`` and
``constraints-no-providers-<PYTHON_MAJOR_MINOR_VERSION>.txt``
will be automatically regenerated by CI job every time after the
``pyproject.toml`` is updated and pushed
if the tests are successful.
diff --git a/dev/breeze/src/airflow_breeze/utils/run_tests.py
b/dev/breeze/src/airflow_breeze/utils/run_tests.py
index 5cd4e6692d8..752c3c66cf7 100644
--- a/dev/breeze/src/airflow_breeze/utils/run_tests.py
+++ b/dev/breeze/src/airflow_breeze/utils/run_tests.py
@@ -90,7 +90,16 @@ def verify_an_image(
if slim_image:
env["TEST_SLIM_IMAGE"] = "true"
command_result = run_command(
- ["uv", "run", "--isolated", "pytest", test_path.as_posix(),
*pytest_args, *extra_pytest_args],
+ [
+ "uv",
+ "run",
+ "--frozen",
+ "--isolated",
+ "pytest",
+ test_path.as_posix(),
+ *pytest_args,
+ *extra_pytest_args,
+ ],
env=env,
output=output,
check=False,
@@ -177,7 +186,7 @@ def run_docker_compose_tests(
env["INCLUDE_SUCCESS_OUTPUTS"] = "true"
env["AIRFLOW_UID"] = str(os.getuid())
command_result = run_command(
- ["uv", "run", "pytest", *pytest_args],
+ ["uv", "run", "--frozen", "pytest", *pytest_args],
env=env,
check=False,
cwd=cwd,
