aritra24 commented on code in PR #39946:
URL: https://github.com/apache/airflow/pull/39946#discussion_r1623183969
##########
contributing-docs/testing/unit_tests.rst:
##########
@@ -1251,6 +1251,131 @@ Rebuilding single provider package can be done using
this command:
breeze release-management prepare-provider-packages \
--version-suffix-for-pypi dev0 --package-format wheel <provider>
+Lowest direct dependency resolution tests
+-----------------------------------------
+
+We have special tests that run with the lowest direct resolution of
dependencies for Airflow and providers.
+This is run in order to check whether we are not using a feature that is not
available in an
+older version of some dependencies.
+
+Tests with lowest-direct dependency resolution for Airflow
+----------------------------------------------------------
+
+You can test minimum dependencies that are installed by Airflow by running
(for example to run "Core" tests):
+
+.. code-block::bash
+
+ breeze testing tests --force-lowest-dependencies --test-type "Core"
+
+You can also iterate on the tests and versions of the dependencies by entering
breeze shell and
+running the tests from there:
+
+.. code-block::bash
+
+ breeze shell --force-lowest-dependencies
+
+The way it works - when you run the breeze with
``--force-lowest-dependencies`` flag, breeze will use
+attempt (with the help of ``uv``) to downgrade the dependencies to the lowest
version that is compatible
+with the dependencies specified in airflow dependencies. You will see it in
the output of the breeze
+command as a sequence of downgrades like this:
+
+.. code-block:: diff
+
+ - aiohttp==3.9.5
+ + aiohttp==3.9.2
+ - anyio==4.4.0
+ + anyio==3.7.1
+
+
+Tests with lowest-direct dependency resolution for a Provider
+-------------------------------------------------------------
+
+Similarly we can test if the provider tests are working for lowest
dependencies of specific provider.
+
+Those tests can be easily run locally with breeze (replace PROVIDER_ID with id
of the provider):
+
+.. code-block::bash
+
+ breeze testing tests --force-lowest-dependencies --test-type
"Providers[PROVIDER_ID]"
+
+If you find that the tests are failing for some all dependencies, make sure to
add minimum version for
Review Comment:
some ~all~ dependencies?
##########
contributing-docs/testing/unit_tests.rst:
##########
@@ -1251,6 +1251,131 @@ Rebuilding single provider package can be done using
this command:
breeze release-management prepare-provider-packages \
--version-suffix-for-pypi dev0 --package-format wheel <provider>
+Lowest direct dependency resolution tests
+-----------------------------------------
+
+We have special tests that run with the lowest direct resolution of
dependencies for Airflow and providers.
+This is run in order to check whether we are not using a feature that is not
available in an
+older version of some dependencies.
+
+Tests with lowest-direct dependency resolution for Airflow
+----------------------------------------------------------
+
+You can test minimum dependencies that are installed by Airflow by running
(for example to run "Core" tests):
+
+.. code-block::bash
+
+ breeze testing tests --force-lowest-dependencies --test-type "Core"
+
+You can also iterate on the tests and versions of the dependencies by entering
breeze shell and
+running the tests from there:
+
+.. code-block::bash
+
+ breeze shell --force-lowest-dependencies
+
+The way it works - when you run the breeze with
``--force-lowest-dependencies`` flag, breeze will use
+attempt (with the help of ``uv``) to downgrade the dependencies to the lowest
version that is compatible
+with the dependencies specified in airflow dependencies. You will see it in
the output of the breeze
+command as a sequence of downgrades like this:
+
+.. code-block:: diff
+
+ - aiohttp==3.9.5
+ + aiohttp==3.9.2
+ - anyio==4.4.0
+ + anyio==3.7.1
+
+
+Tests with lowest-direct dependency resolution for a Provider
+-------------------------------------------------------------
+
+Similarly we can test if the provider tests are working for lowest
dependencies of specific provider.
+
+Those tests can be easily run locally with breeze (replace PROVIDER_ID with id
of the provider):
+
+.. code-block::bash
+
+ breeze testing tests --force-lowest-dependencies --test-type
"Providers[PROVIDER_ID]"
+
+If you find that the tests are failing for some all dependencies, make sure to
add minimum version for
+the dependency in the provider.yaml file of the appropriate provider and
re-run it.
+
+You can also iterate on the tests and versions of the dependencies by entering
breeze shell and
+running the tests from there:
+
+.. code-block::bash
+
+ breeze shell --force-lowest-dependencies --test-type
"Providers[PROVIDER_ID]"
+
+Similarly as in case of "Core" tests, the dependencies will be downgraded to
the lowest version that is
+compatible with the dependencies specified in the provider dependencies and
you will see the list of
+downgrades in the output of the breeze command. Note that this will be
combined downgrades of both
+Airflow and selected provider dependencies, so the list will be longer than in
case of "Core" tests
+and longer than **just** dependencies of the provider. For example for a
``google`` provider, part of the
+downgraded dependencies will contain both Airflow and Google Provider
dependencies:
+
+.. code-block:: diff
+
+ - flask-login==0.6.3
+ + flask-login==0.6.2
+ - flask-session==0.5.0
+ + flask-session==0.4.0
+ - flask-wtf==1.2.1
+ + flask-wtf==1.1.0
+ - fsspec==2023.12.2
+ + fsspec==2023.10.0
+ - gcloud-aio-bigquery==7.1.0
+ + gcloud-aio-bigquery==6.1.2
+ - gcloud-aio-storage==9.2.0
+
+
+How to fix failing lowest-direct dependency resolution tests
+------------------------------------------------------------
+
+When your tests pass in regular test, but fail in "lowest-direct" dependency
resolution tests, you need
+to figure out the lower-bindings missing in ``hatch_build.py`` (for Airflow
core dependencies) or
+in the corresponding provider's ``provider.yaml`` file. This is usually very
easy thing that takes a little
Review Comment:
This is usually **a** very easy thing
##########
contributing-docs/testing/unit_tests.rst:
##########
@@ -1251,6 +1251,131 @@ Rebuilding single provider package can be done using
this command:
breeze release-management prepare-provider-packages \
--version-suffix-for-pypi dev0 --package-format wheel <provider>
+Lowest direct dependency resolution tests
+-----------------------------------------
+
+We have special tests that run with the lowest direct resolution of
dependencies for Airflow and providers.
+This is run in order to check whether we are not using a feature that is not
available in an
+older version of some dependencies.
+
+Tests with lowest-direct dependency resolution for Airflow
+----------------------------------------------------------
+
+You can test minimum dependencies that are installed by Airflow by running
(for example to run "Core" tests):
+
+.. code-block::bash
+
+ breeze testing tests --force-lowest-dependencies --test-type "Core"
+
+You can also iterate on the tests and versions of the dependencies by entering
breeze shell and
+running the tests from there:
+
+.. code-block::bash
+
+ breeze shell --force-lowest-dependencies
+
+The way it works - when you run the breeze with
``--force-lowest-dependencies`` flag, breeze will use
+attempt (with the help of ``uv``) to downgrade the dependencies to the lowest
version that is compatible
+with the dependencies specified in airflow dependencies. You will see it in
the output of the breeze
+command as a sequence of downgrades like this:
+
+.. code-block:: diff
+
+ - aiohttp==3.9.5
+ + aiohttp==3.9.2
+ - anyio==4.4.0
+ + anyio==3.7.1
+
+
+Tests with lowest-direct dependency resolution for a Provider
+-------------------------------------------------------------
+
+Similarly we can test if the provider tests are working for lowest
dependencies of specific provider.
+
+Those tests can be easily run locally with breeze (replace PROVIDER_ID with id
of the provider):
+
+.. code-block::bash
+
+ breeze testing tests --force-lowest-dependencies --test-type
"Providers[PROVIDER_ID]"
+
+If you find that the tests are failing for some all dependencies, make sure to
add minimum version for
+the dependency in the provider.yaml file of the appropriate provider and
re-run it.
+
+You can also iterate on the tests and versions of the dependencies by entering
breeze shell and
+running the tests from there:
+
+.. code-block::bash
+
+ breeze shell --force-lowest-dependencies --test-type
"Providers[PROVIDER_ID]"
+
+Similarly as in case of "Core" tests, the dependencies will be downgraded to
the lowest version that is
+compatible with the dependencies specified in the provider dependencies and
you will see the list of
+downgrades in the output of the breeze command. Note that this will be
combined downgrades of both
+Airflow and selected provider dependencies, so the list will be longer than in
case of "Core" tests
+and longer than **just** dependencies of the provider. For example for a
``google`` provider, part of the
+downgraded dependencies will contain both Airflow and Google Provider
dependencies:
+
+.. code-block:: diff
+
+ - flask-login==0.6.3
+ + flask-login==0.6.2
+ - flask-session==0.5.0
+ + flask-session==0.4.0
+ - flask-wtf==1.2.1
+ + flask-wtf==1.1.0
+ - fsspec==2023.12.2
+ + fsspec==2023.10.0
+ - gcloud-aio-bigquery==7.1.0
+ + gcloud-aio-bigquery==6.1.2
+ - gcloud-aio-storage==9.2.0
+
+
+How to fix failing lowest-direct dependency resolution tests
+------------------------------------------------------------
+
+When your tests pass in regular test, but fail in "lowest-direct" dependency
resolution tests, you need
+to figure out the lower-bindings missing in ``hatch_build.py`` (for Airflow
core dependencies) or
+in the corresponding provider's ``provider.yaml`` file. This is usually very
easy thing that takes a little
+time to figure out especially if you just added new feature from a library
that you use, just check in
+the release notes what is the minimum version of the library that you can use
and set it as the
+``>=VERSION`` in the ``hatch_build.py`` or ``provider.yaml`` file. For
``hatch_build.py`` changes you do not
+need to do anything else, for ``provider.yaml`` file you need to regenerate
generated dependencies
+by running ``pre-commit run`` in the provider directory after adding the file
to git or just letting the
+pre-commit to do it's job if you already has pre-commit installed via
``pre-commit install`` - then just
+committing the change will regenerate the dependencies automatically.
+
+After that, re-run the ``breeze shell --force-lowest-dependencies`` command
and see if the tests pass.
+
+.. code-block::bash
+
+ breeze shell --force-lowest-dependencies --test-type
"Providers[PROVIDER_ID]"
+
+Sometimes it might get a bit tricky to know what is the minimum version of the
library you should be using
+but in this case you can easily find it by looking at the error and list of
downgraded packages and
+guessing which one is the one that is causing the problem. You can then look
at the release notes of the
+library and find the minimum version but also you can revert to technique
knowing as bisecting which allows
Review Comment:
revert to bisecting (or) revert to the technique known as bisecting
##########
contributing-docs/testing/unit_tests.rst:
##########
@@ -1251,6 +1251,131 @@ Rebuilding single provider package can be done using
this command:
breeze release-management prepare-provider-packages \
--version-suffix-for-pypi dev0 --package-format wheel <provider>
+Lowest direct dependency resolution tests
+-----------------------------------------
+
+We have special tests that run with the lowest direct resolution of
dependencies for Airflow and providers.
+This is run in order to check whether we are not using a feature that is not
available in an
+older version of some dependencies.
+
+Tests with lowest-direct dependency resolution for Airflow
+----------------------------------------------------------
+
+You can test minimum dependencies that are installed by Airflow by running
(for example to run "Core" tests):
+
+.. code-block::bash
+
+ breeze testing tests --force-lowest-dependencies --test-type "Core"
+
+You can also iterate on the tests and versions of the dependencies by entering
breeze shell and
+running the tests from there:
+
+.. code-block::bash
+
+ breeze shell --force-lowest-dependencies
+
+The way it works - when you run the breeze with
``--force-lowest-dependencies`` flag, breeze will use
+attempt (with the help of ``uv``) to downgrade the dependencies to the lowest
version that is compatible
+with the dependencies specified in airflow dependencies. You will see it in
the output of the breeze
+command as a sequence of downgrades like this:
+
+.. code-block:: diff
+
+ - aiohttp==3.9.5
+ + aiohttp==3.9.2
+ - anyio==4.4.0
+ + anyio==3.7.1
+
+
+Tests with lowest-direct dependency resolution for a Provider
+-------------------------------------------------------------
+
+Similarly we can test if the provider tests are working for lowest
dependencies of specific provider.
+
+Those tests can be easily run locally with breeze (replace PROVIDER_ID with id
of the provider):
+
+.. code-block::bash
+
+ breeze testing tests --force-lowest-dependencies --test-type
"Providers[PROVIDER_ID]"
+
+If you find that the tests are failing for some all dependencies, make sure to
add minimum version for
+the dependency in the provider.yaml file of the appropriate provider and
re-run it.
+
+You can also iterate on the tests and versions of the dependencies by entering
breeze shell and
+running the tests from there:
+
+.. code-block::bash
+
+ breeze shell --force-lowest-dependencies --test-type
"Providers[PROVIDER_ID]"
+
+Similarly as in case of "Core" tests, the dependencies will be downgraded to
the lowest version that is
+compatible with the dependencies specified in the provider dependencies and
you will see the list of
+downgrades in the output of the breeze command. Note that this will be
combined downgrades of both
+Airflow and selected provider dependencies, so the list will be longer than in
case of "Core" tests
+and longer than **just** dependencies of the provider. For example for a
``google`` provider, part of the
+downgraded dependencies will contain both Airflow and Google Provider
dependencies:
+
+.. code-block:: diff
+
+ - flask-login==0.6.3
+ + flask-login==0.6.2
+ - flask-session==0.5.0
+ + flask-session==0.4.0
+ - flask-wtf==1.2.1
+ + flask-wtf==1.1.0
+ - fsspec==2023.12.2
+ + fsspec==2023.10.0
+ - gcloud-aio-bigquery==7.1.0
+ + gcloud-aio-bigquery==6.1.2
+ - gcloud-aio-storage==9.2.0
+
+
+How to fix failing lowest-direct dependency resolution tests
+------------------------------------------------------------
+
+When your tests pass in regular test, but fail in "lowest-direct" dependency
resolution tests, you need
+to figure out the lower-bindings missing in ``hatch_build.py`` (for Airflow
core dependencies) or
+in the corresponding provider's ``provider.yaml`` file. This is usually very
easy thing that takes a little
+time to figure out especially if you just added new feature from a library
that you use, just check in
Review Comment:
little **bit of** time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
