kaxil commented on a change in pull request #11541:
URL: https://github.com/apache/airflow/pull/11541#discussion_r507188908
##########
File path: CI.rst
##########
@@ -629,6 +628,90 @@ delete old artifacts that are > 7 days old. It only runs
for the 'apache/airflow
We also have a script that can help to clean-up the old artifacts:
`remove_artifacts.sh <dev/remove_artifacts.sh>`_
+Selective CI Checks
+===================
+
+In order to optimise our CI builds, we've implemented optimisations to only
run selected checks for some
+kind of changes. The logic implemented reflects the internal architecture of
Airflow 2.0 packages
+and it helps to keep down both the usage of jobs in GitHub Actions as well as
CI feedback time to
+contributors in case of simpler changes.
+
+We have the following test types (separated by packages in which they are):
+
+* Core - for the core Airflow functionality (core folder)
+* API - Tests for the API of Airflow (api and api_connexion folders)
+* CLI - Tests for the CLI of Airflow (cli folder)
+* WWW - Tests for the webserver of Airflow (www and www_rbac in 1.10 folders)
+* Providers - Tests for all Providers of Airflow (providers folder)
+* Other - all other tests (all other folders that are not part of any of the
above)
+
+We also have several special kinds of tests that are not separated by packages
but they are marked with
+pytest markers. They can be found in any of those packages and they can be
selected by the appropriate
+pylint custom command line options. See `TESTING.rst <TESTING.rst>`_ for
details but those are:
+
+* Integration - tests that require external integration images running in
docker-compose
+* Heisentests - tests that are vulnerable to some side effects and are better
to be run on their own
+* Quarantined - tests that are flaky and need to be fixed
+* Postgres - tests that require Postgres database. They are only run when
backend is Postgres
+* MySQL - tests that require MySQL database. They are only run when backend is
MySQL
+
+Even if the types are separated that share the same backend version/python
version are run sequentially
+on the same CI machine - each of them in a separate docker run command and
with additional docker
+cleaning between the steps to not fall into the trap of using to much of
resources in one big test run
+but also not to increase a number of jobs per each Pull Request.
Review comment:
```suggestion
Even if the types are separated that share the same backend version/python
version is run sequentially
on the same CI machine - each of them in a separate docker run command and
with additional docker
cleaning between the steps to not fall into the trap of using too many
resources in one big test run
but also not to increase the number of jobs per each Pull Request.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
