Lee-W commented on code in PR #39513:
URL: https://github.com/apache/airflow/pull/39513#discussion_r1599548092


##########
contributing-docs/testing/unit_tests.rst:
##########
@@ -1108,6 +1108,128 @@ This prepares airflow .whl package in the dist folder.
 
      breeze --use-airflow-version wheel --use-packages-from-dist 
--mount-sources skip
 
+Compatibility Provider unit tests against older airflow releases
+................................................................
+
+Our CI runs provider tests for providers with previous compatible airflow 
releases. This allows to check
+if the providers still work when installed for older airflow versions.
+
+.. note::
+
+  For now it's done for 2.9.1 version only.
+
+Those tests can be used to test compatibility of the providers with past and 
future releases of airflow
+For example it could be used to run latest provider versions with released or 
main
+Airflow 3 if they are developed independently.
+
+The tests use the current source version of ``tests`` folder - so care should 
be taken that the tests
+implemented for providers in the sources allow to run it against previous 
versions of Airflow and
+against Airflow installed from package rather than from the sources.
+
+This can be reproduced locally building providers from tag/commit of the 
airflow repository.
+
+1. Make sure to build latest Breeze ci image
+
+.. code-block:: bash
+
+   breeze ci-image build --python 3.8
+
+2. Build providers from latest sources:
+
+.. code-block:: bash
+
+   rm dist/*
+   breeze release-management prepare-provider-packages 
--include-not-ready-providers \
+      --version-suffix-for-pypi dev0 --package-format wheel
+
+3. Prepare provider constraints
+
+.. code-block:: bash
+
+   breeze release-management generate-constraints --airflow-constraints-mode 
constraints-source-providers --answer yes
+
+3. Enter breeze environment, installing selected airflow version and the 
provider packages prepared from main
+
+.. code-block::bash
+
+  breeze shell --use-packages-from-dist --package-format wheel\
+   --use-airflow-version 2.9.1 --airflow-constraints-reference 
constraints-2.9.1 \
+   --install-airflow-with-constraints \
+   --providers-skip-constraints \
+   --mount-sources tests
+
+4. You can then run tests as usual:
+
+.. code-block::bash
+
+   pytest tests/providers/<provider>/test.py
+
+5. Iterate with the tests
+
+  The tests are run using:
+
+   * airflow installed from PyPI
+   * tests coming from the current airflow sources (they are mounted inside 
the breeze image)
+   * provider packages built from the current airflow sources and placed in 
dist
+
+  This means that you can modify and run tests and re-run them, but if you 
want to modify provider code
+  you need to exit breeze, rebuild the provider package and restart breeze 
using the command above.
+
+  Rebuilding single provider package can be done using this command:
+
+  .. code-block::bash
+
+    breeze release-management prepare-provider-packages \
+      --version-suffix-for-pypi dev0 --package-format wheel <provider>
+
+
+Note that some of the tests if written without taking care about the 
compatibility, might not work with older
+versions of Airflow - this is because of refactorings, renames, and tests 
relying on internals of Airflow that
+are not part of the public API. We deal with it in one of the following ways:
+
+1) if the whole provider is supposed to only work for later airflow version, 
we remove the whole provider
+   by excluding it from compatibility test configuration (see below)
+
+2) Some compatibility shims are defined in ``tests/test_utils/compat.py`` - 
and they can be used to make the
+   tests compatible - for example importing ``ParseImportError`` after the 
exception has been renamed from
+   ``ImportError`` and it would fail in Airflow 2.9, but we have a fallback 
import in ``compat.py`` that
+   falls back to old import automatically, so all tests testing / expecting 
``ParseImportError`` should import
+   it from the ``tests.tests_utils.compat`` module. There are few other 
compatibility shims defined there and
+   you can add more if needed in a similar way.
+
+3) If only some tests are not compatible and use features that are available 
only in newer airflow version,
+    we can mark those tests with appropriate ``AIRFLOW_V_2_X_PLUS`` boolean 
constant defined in ``compat.py``
+    For example:
+
+```python
+from tests.test_utils.compat import AIRFLOW_V_2_7_PLUS
+
+
[email protected](not AIRFLOW_V_2_7_PLUS, reason="The tests should be skipped 
for Airflow < 2.7")
+def some_test_that_only_works_for_airflow_2_7_plus():
+    pass
+```

Review Comment:
   ```suggestion
     .. code-block:: python
       from tests.test_utils.compat import AIRFLOW_V_2_7_PLUS
   
   
       @pytest.mark.skip(not AIRFLOW_V_2_7_PLUS, reason="The tests should be 
skipped for Airflow < 2.7")
       def some_test_that_only_works_for_airflow_2_7_plus():
           pass
   ```
   
   I think we might need to use rst syntax



##########
contributing-docs/testing/unit_tests.rst:
##########
@@ -1108,6 +1108,128 @@ This prepares airflow .whl package in the dist folder.
 
      breeze --use-airflow-version wheel --use-packages-from-dist 
--mount-sources skip
 
+Compatibility Provider unit tests against older airflow releases
+................................................................
+
+Our CI runs provider tests for providers with previous compatible airflow 
releases. This allows to check
+if the providers still work when installed for older airflow versions.
+
+.. note::
+
+  For now it's done for 2.9.1 version only.
+
+Those tests can be used to test compatibility of the providers with past and 
future releases of airflow
+For example it could be used to run latest provider versions with released or 
main
+Airflow 3 if they are developed independently.
+
+The tests use the current source version of ``tests`` folder - so care should 
be taken that the tests
+implemented for providers in the sources allow to run it against previous 
versions of Airflow and
+against Airflow installed from package rather than from the sources.
+
+This can be reproduced locally building providers from tag/commit of the 
airflow repository.
+
+1. Make sure to build latest Breeze ci image
+
+.. code-block:: bash
+
+   breeze ci-image build --python 3.8
+
+2. Build providers from latest sources:
+
+.. code-block:: bash
+
+   rm dist/*
+   breeze release-management prepare-provider-packages 
--include-not-ready-providers \
+      --version-suffix-for-pypi dev0 --package-format wheel
+
+3. Prepare provider constraints
+
+.. code-block:: bash
+
+   breeze release-management generate-constraints --airflow-constraints-mode 
constraints-source-providers --answer yes
+
+3. Enter breeze environment, installing selected airflow version and the 
provider packages prepared from main
+
+.. code-block::bash
+
+  breeze shell --use-packages-from-dist --package-format wheel\
+   --use-airflow-version 2.9.1 --airflow-constraints-reference 
constraints-2.9.1 \
+   --install-airflow-with-constraints \
+   --providers-skip-constraints \
+   --mount-sources tests
+
+4. You can then run tests as usual:
+
+.. code-block::bash
+
+   pytest tests/providers/<provider>/test.py
+
+5. Iterate with the tests
+
+  The tests are run using:
+
+   * airflow installed from PyPI
+   * tests coming from the current airflow sources (they are mounted inside 
the breeze image)
+   * provider packages built from the current airflow sources and placed in 
dist
+
+  This means that you can modify and run tests and re-run them, but if you 
want to modify provider code
+  you need to exit breeze, rebuild the provider package and restart breeze 
using the command above.
+
+  Rebuilding single provider package can be done using this command:
+
+  .. code-block::bash
+
+    breeze release-management prepare-provider-packages \
+      --version-suffix-for-pypi dev0 --package-format wheel <provider>
+
+
+Note that some of the tests if written without taking care about the 
compatibility, might not work with older
+versions of Airflow - this is because of refactorings, renames, and tests 
relying on internals of Airflow that
+are not part of the public API. We deal with it in one of the following ways:
+
+1) if the whole provider is supposed to only work for later airflow version, 
we remove the whole provider
+   by excluding it from compatibility test configuration (see below)
+
+2) Some compatibility shims are defined in ``tests/test_utils/compat.py`` - 
and they can be used to make the
+   tests compatible - for example importing ``ParseImportError`` after the 
exception has been renamed from
+   ``ImportError`` and it would fail in Airflow 2.9, but we have a fallback 
import in ``compat.py`` that
+   falls back to old import automatically, so all tests testing / expecting 
``ParseImportError`` should import
+   it from the ``tests.tests_utils.compat`` module. There are few other 
compatibility shims defined there and
+   you can add more if needed in a similar way.
+
+3) If only some tests are not compatible and use features that are available 
only in newer airflow version,
+    we can mark those tests with appropriate ``AIRFLOW_V_2_X_PLUS`` boolean 
constant defined in ``compat.py``
+    For example:
+
+```python
+from tests.test_utils.compat import AIRFLOW_V_2_7_PLUS
+
+
[email protected](not AIRFLOW_V_2_7_PLUS, reason="The tests should be skipped 
for Airflow < 2.7")
+def some_test_that_only_works_for_airflow_2_7_plus():
+    pass
+```
+
+4) Sometimes the tests should only be run when airflow is installed from the 
sources. Im this case you can

Review Comment:
   ```suggestion
   4) Sometimes, the tests should only be run when airflow is installed from 
the sources. In this case you can
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to