michalslowikowski00 commented on a change in pull request #7774: [AIRFLOW-XXXX]
document system tests mechanism better
URL: https://github.com/apache/airflow/pull/7774#discussion_r395523833
##########
File path: TESTING.rst
##########
@@ -543,31 +587,197 @@ tests whenever an operator/hook/sensor is
added/modified in a given system.
the system class will take care about running the DAG. Note that the
DAG_FOLDER should be
a subdirectory of the ``tests.test_utils.AIRFLOW_MAIN_FOLDER`` +
``providers/<SYSTEM_NAME>/example_dags``.
-An example of a system test is available in:
-``airflow.tests.providers.google.operators.test_natunal_language_system.CloudNaturalLanguageExampleDagsTest``.
+A simple example of a system test is available in:
+
+``tests/providers/google/cloud/operators/test_compute_system.py``.
+
+It runs two DAGs defined in
``airflow.providers.google.cloud.example_dags.example_compute.py`` and
+``airflow.providers.google.cloud.example_dags.example_compute_igm.py``.
+
+Preparing backport packages for System Tests for Airflow 1.10.* series
+----------------------------------------------------------------------
+
+In order to run system tests with old Airflow version you need to prepare
backport packages. This
+can be done by running ``./scripts/ci/ci_prepare_bacport_packages.sh <PACKAGES
TO BUILD>``. For
+example the below command will build google postgres and mysql packages:
+
+.. code-block:: bash
+
+ ./scripts/ci/ci_prepare_bacport_packages.sh google postgres mysql
+
+Those packages will be prepared in ./dist folder. This folder is mapped to
/dist folder
+when you enter Breeze, so it is easy to automate installing of those packages
for testing.
-It runs the DAG defined in
``airflow.providers.google.cloud.example_dags.example_natural_language.py``.
-Running Tests for Older Airflow Versions
-----------------------------------------
+Installing backported for Airflow 1.10.* series
+-----------------------------------------------
The tests can be executed against the master version of Airflow but they also
work
with older versions. This is especially useful to test back-ported operators
from Airflow 2.0 to 1.10.* versions.
To run the tests for Airflow 1.10.* series, you need to run Breeze with
-``--install-airflow-version==<VERSION>`` to install a different version of
Airflow.
+``--install-airflow-version=<VERSION>`` to install a different version of
Airflow.
If ``current`` is specified (default), then the current version of Airflow is
used.
Otherwise, the released version of Airflow is installed.
-The commands make sure that the source version of master Airflow is removed
and the released version of
-Airflow from ``Pypi`` is installed. Note that tests sources are not removed
and they can be used
-to run tests (unit tests and system tests) against the freshly installed
version.
+The ``-install-airflow-version=<VERSION>`` command make sure that the current
(from sources) version of
+Airflow is removed and the released version of Airflow from ``Pypi`` is
installed. Note that tests sources
+are not removed and they can be used to run tests (unit tests and system
tests) against the
+freshly installed version.
+
+You should automate installing of the backport packages in your own
+``./files/airflow-breeze-config/variables.env`` file. You should make it
depend on
+``RUN_AIRFLOW_1_10`` variable value equals to "true" so that
+installation of backport packages is only performed when you install airflow
1.10.*.
+The backport packages are available in ``/dist`` directory if they were
prepared as described
+in the previous chapter.
+
+Typically the command in you variables.env file will be similar to:
+
+.. code-block:: bash
+
+ # install any packages from dist folder if they are available
+ if [[ ${RUN_AIRFLOW_1_10:=} == "true" ]]; then
+ pip install /dist/apache_airflow_providers_{google,postgres,mysql}*.whl
|| true
+ fi
+
+The command above will automatically install backported google, postgres amd
mysql packages if they
+were prepared before entering breeze.
+
+
+Running system tests for backported packages in Airflow 1.10.* series
+---------------------------------------------------------------------
+
+Once you installed 1.10.* Airflow version with ``--install-airflow-version``
and prepared and
+installed the required packages via ``variables.env`` it should be as easy as
running
+``pytest --systems=<SYSTEM_NAME> TEST_NAME``. Note that we have default
timeout for runnning
+system tests set to 8 minutes and some system tests might take much longer to
run and you might
+want to add ``-o faulthandler_timeout=2400`` (2400s = 40 minutes for example)
to your
+pytest command.
+
+Typical system test session
+---------------------------
+
+Here is the typical session that you need to do to run system tests:
+
+1. Prepare backport packages
+
+.. code-block:: bash
+
+ ./scripts/ci/ci_prepare_bacport_packages.sh google postgres mysql
+
+2. Enter breeze with installing Airflow 1.10.*, forwarding credentials and
installing
+ backported packages (you need appropriate line in
``./files/airflow-breeze-config/variables.env``)
+
+.. code-block:: bash
+
+ ./breeze --install-airflow-version 1.10.9 --python 3.6 --db-reset
--forward-credentials restart
+
+This will:
+
+* install Airflow 1.10.9
+* restarts the whole environment (i.e. recreates metadata database from the
scratch)
+* run Breeze with python 3.6 version
+* reset the Airflow database
+* forward your local credentials to Breeze
+
+3. Run the tests:
+
+.. code-block:: bash
+
+ pytest -o faulthandler_timeout=2400 --systems=google
tests/providers/google/cloud/operators/test_compute_system.py
+
+
+Iteration with System Tests if your resources are slow to create
+----------------------------------------------------------------
+
+When you want to iterate on system tests, you might want to create slow
resources first
Review comment:
When you want to iterate on system tests, you might want to create slow
resources first
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
