This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 7f421d1483 Refresh the CI documentation to reflect latest changes
(#29217)
7f421d1483 is described below
commit 7f421d148316a67bf60c614aa81aaceab4e35634
Author: Jarek Potiuk <[email protected]>
AuthorDate: Mon Jan 30 11:35:51 2023 +0100
Refresh the CI documentation to reflect latest changes (#29217)
There were a few changes to our CI workflows that have not been
reflected in the documentation:
* conditional behaviour for non-main branches (some jobs and tests
are skipped when running in v2_* branch)
* Breeze is not "new" any more - there is only one Breeze left
* unit and integration tests have been split to separate jobs
* "summarize warnings" job has been added to summarize warnings
from all tests
* "testing building airflow packages job has been added
* There are only "React WWW tests" now after consolidating UI code
into one folder
* a job has been added to test if images build quickly
* a job has been added to test if Windows git cloning works
* a job has been added to test if ARM CI images build properly
* a jonb for testing docker-compose quick start was missing in the
documentation
This PR synchronizes all the changes and updates CI.rst with more
context and explanation for the conditional behaviours and with
added missing jobs (including updating CI mermaid diagrams)
---
.github/workflows/ci.yml | 2 +-
CI.rst | 128 ++++++++++++++++++++++++++++++++++-------------
CI_DIAGRAMS.md | 85 ++++++++++++++++++++++++-------
3 files changed, 161 insertions(+), 54 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index b079887193..415e39c1ec 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -410,7 +410,7 @@ jobs:
PYTHON_VERSIONS:
${{needs.build-info.outputs.all-python-versions-list-as-string}}
DEBUG_RESOURCES: ${{ needs.build-info.outputs.debug-resources }}
- run-new-breeze-tests:
+ run-breeze-tests:
timeout-minutes: 10
name: Breeze unit tests
runs-on: "${{needs.build-info.outputs.runs-on}}"
diff --git a/CI.rst b/CI.rst
index 178e536376..8f9862f835 100644
--- a/CI.rst
+++ b/CI.rst
@@ -333,47 +333,82 @@ fails to build, it cancels other builds and the source
``Tests`` workflow run
that triggered it.
+Differences for main and release branches
+-----------------------------------------
+
+There are a few differences of what kind of tests are run, depending on which
version/branch the tests are executed for.
+While all our tests run for the "main" development branch to keep Airflow in
check, only a subset of those tests is run
+in older branches when we are releasing patch-level releases. This is because
we never use old branches to release
+providers and helm charts, we only use them to release Airflow and Airflow
image.
+
+This behaviour is controlled by ``default-branch`` output of the build-info
job. Whenever we create a branch for old version
+we update the ``AIRFLOW_BRANCH`` in ``airflow_breeze/branch_defaults.py`` to
point to the new branch and there are a few
+places where selection of tests is based on whether this output is ``main``.
They are marked as - in the "Release branches"
+column of the table below.
+
Tests Workflow
--------------
This workflow is a regular workflow that performs all checks of Airflow code.
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Job | Description
| PR | Canary | Scheduled |
-+=============================+==========================================================+=========+==========+===========+
-| Build info | Prints detailed information about the build
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Build CI/PROD images | Builds images in-workflow (not in the build
images one) | - | Yes | Yes (1) |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Push early cache & images | Pushes early cache/images to GitHub Registry
and test | - | Yes | - |
-| | speed of building breeze images from scratch
| | | |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Test OpenAPI client gen | Tests if OpenAPIClient continues to generate
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| UI tests | React UI tests for new Airflow UI
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Test image building | Tests if PROD image build examples work
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| CI Images | Waits for and verify CI Images (2)
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| (Basic) Static checks | Performs static checks (full or basic)
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Build docs | Builds documentation
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Tests | Run all the Pytest tests for Python code
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Tests provider packages | Tests if provider packages work
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Upload coverage | Uploads test coverage from all the tests
| - | Yes | - |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| PROD Images | Waits for and verify PROD Images (2)
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Tests Kubernetes | Run Kubernetes test
| Yes | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Constraints | Upgrade constraints to latest ones (3)
| - | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
-| Push cache & images | Pushes cache/images to GitHub Registry (3)
| - | Yes | Yes |
-+-----------------------------+----------------------------------------------------------+---------+----------+-----------+
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Job | Description
| PR | Canary | Scheduled | Release branches |
++=================================+==========================================================+==========+==========+===========+===================+
+| Build info | Prints detailed information about the
build | Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Build CI/PROD images | Builds images in-workflow (not in the
build images one) | - | Yes | Yes (1) | Yes (4) |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Check that image builds quickly | Checks that image builds quickly without
taking a lot of | - | Yes | - | Yes |
+| | time for ``pip`` to figure out the right
set of deps. | | | | |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Push early cache & images | Pushes early cache/images to GitHub
Registry and test | - | Yes | - | - |
+| | speed of building breeze images from
scratch | | | | |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Run breeze tests | Run unit tests for Breeze
| Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Test OpenAPI client gen | Tests if OpenAPIClient continues to
generate | Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| React WWW tests | React UI tests for new Airflow UI
| Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Test image building | Tests if PROD image build examples work
| Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Test git clone on Windows | Tests if Git clone for for Windows
| Yes (5) | - | - | Yes (5) |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Waits for CI Images | Waits for and verify CI Images (2)
| Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Static checks | Performs full static checks
| Yes (6) | Yes | Yes | Yes (7) |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Basic static checks | Performs basic static checks
| Yes (6) | - | - | Yes (7) |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Build docs | Builds documentation
| Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Tests | Run the Pytest unit tests (Backend/Python
matrix) | Yes | Yes | Yes | Yes (8) |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Integration tests | Runs integration tests (Postgres/Mysql)
| Yes | Yes | Yes | Yes (9) |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Quarantined tests | Runs quarantined tests (with flakiness and
side-effects) | Yes | Yes | Yes | Yes (8) |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Tests provider packages | Tests if provider packages can be built
and released | Yes | Yes | Yes | - |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Test airflow packages | Tests that Airflow package can be built
and released | Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Helm tests | Run the Helm integration tests
| Yes | Yes | Yes | - |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Summarize warnings | Summarizes warnings from all other tests
| Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Wait for PROD Images | Waits for and verify PROD Images (2)
| Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Tests Kubernetes | Run Kubernetes test
| Yes | Yes | Yes | - |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Test docker-compose | Tests if quick-start docker compose works
| Yes | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Constraints | Upgrade constraints to latest ones (3)
| - | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Push cache & images | Pushes cache/images to GitHub Registry (3)
| - | Yes | Yes | Yes |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
+| Build CI ARM images | Builds CI images for ARM to detect any
problems which | Yes (10) | - | Yes | Yes |
+| | would only appear if we install all
dependencies on ARM | | | | |
++---------------------------------+----------------------------------------------------------+----------+----------+-----------+-------------------+
``(1)`` Scheduled jobs builds images from scratch - to test if everything
works properly for clean builds
@@ -383,6 +418,27 @@ This workflow is a regular workflow that performs all
checks of Airflow code.
upgraded only if all tests are successful. The images are rebuilt in this step
using constraints pushed
in the previous step.
+``(4)`` In main, PROD image uses locally build providers using "latest"
version of the provider code. In the
+non-main version of the build, the latest released providers from PyPI are
used.
+
+``(5)`` Only runs those tests for the builds where public runners are used (so
either when non-committer
+runs it or when ``use public runner`` label is assigned to the PR.
+
+``(6)`` Run full set of static checks when selective-checks determine that
they are needed (basically, when
+Python code has been modified).
+
+``(7)`` On non-main builds some of the static checks that are related to
Providers are skipped via selective checks
+(``skip-pre-commits`` check).
+
+``(8)`` On non-main builds the unit tests for providers are skipped via
selective checks removing the
+"Providers" test type.
+
+``(9)`` On non-main builds the integration tests for providers are skipped via
``skip-provider-tests`` selective
+check output.
+
+``(10)`` Only run the builds in case dependencies are changed
(``upgrade-to-newer-dependencies`` is set).
+
+
CodeQL scan
-----------
diff --git a/CI_DIAGRAMS.md b/CI_DIAGRAMS.md
index e19228a024..0b6fd95f7e 100644
--- a/CI_DIAGRAMS.md
+++ b/CI_DIAGRAMS.md
@@ -42,9 +42,11 @@ sequenceDiagram
and
Note over Tests: OpenAPI client gen
and
- Note over Tests: Test UI
+ Note over Tests: React WWW tests
and
Note over Tests: Test examples<br>PROD image building
+ and
+ Note over Tests: Test git clone on Windows
end
Build Images ->> GitHub Registry: Push CI Images<br>[COMMIT_SHA]
loop Wait for CI images
@@ -68,17 +70,30 @@ sequenceDiagram
end
and
opt
- Note over Tests: Tests
+ Note over Tests: Unit Tests
+ end
+ and
+ opt
+ Note over Tests: Integration Tests
+ end
+ and
+ opt
+ Note over Tests: Quarantined Tests
end
and
opt
Note over Tests: Test provider <br>packages build
end
+ and
+ opt
+ Note over Tests: Test airflow <br>packages build
+ end
and
opt
Note over Tests: Helm tests
end
end
+ Note over Tests: Summarize Warnings
Build Images ->> GitHub Registry: Push PROD Images<br>[COMMIT_SHA]
deactivate Build Images
loop Wait for PROD images
@@ -87,11 +102,11 @@ sequenceDiagram
Note over Tests: Verify PROD Image<br>[COMMIT_SHA]
par
opt
- Note over Tests: Run Kubernetes<br>tests
+ Note over Tests: Run Kubernetes <br>tests
end
and
opt
- Note over Tests: Run Kubernetes<br>upgrade tests
+ Note over Tests: Run docker-compose <br>tests
end
end
opt
@@ -99,6 +114,7 @@ sequenceDiagram
end
Tests -->> Airflow Repo: Status update
deactivate Airflow Repo
+ Note over Tests: Build ARM CI images
deactivate Tests
```
@@ -127,6 +143,8 @@ sequenceDiagram
Note over Tests: Test UI
and
Note over Tests: Test examples<br>PROD image building
+ and
+ Note over Tests: Test git clone on Windows
end
Tests ->> GitHub Registry: Push CI Images<br>[COMMIT_SHA]
GitHub Registry ->> Tests: Pull CI Images<br>[COMMIT_SHA]
@@ -147,17 +165,30 @@ sequenceDiagram
end
and
opt
- Note over Tests: Tests
+ Note over Tests: Unit Tests
+ end
+ and
+ opt
+ Note over Tests: Integration Tests
+ end
+ and
+ opt
+ Note over Tests: Quarantined Tests
end
and
opt
Note over Tests: Test provider <br>packages build
end
+ and
+ opt
+ Note over Tests: Test airflow <br>packages build
+ end
and
opt
Note over Tests: Helm tests
end
end
+ Note over Tests: Summarize Warnings
Tests ->> GitHub Registry: Push PROD Images<br>[COMMIT_SHA]
GitHub Registry ->> Tests: Pull PROD Image<br>[COMIT_SHA]
Note over Tests: Verify PROD Image<br>[COMMIT_SHA]
@@ -167,7 +198,7 @@ sequenceDiagram
end
and
opt
- Note over Tests: Run Kubernetes <br>upgrade tests
+ Note over Tests: Run docker-compose <br>tests
end
end
opt
@@ -175,6 +206,7 @@ sequenceDiagram
end
Tests -->> Airflow Repo: Status update
deactivate Airflow Repo
+ Note over Tests: Build ARM CI images
deactivate Tests
```
@@ -190,16 +222,18 @@ sequenceDiagram
Note over Tests: Build info<br>All tests<br>Full matrix
par
Note over Tests: Build CI Images<br>[COMMIT_SHA]<br>Always upgrade deps
+ and
+ Note over Tests: Check that image builds quickly
+ and
+ Note over Tests: Push early cache and images
and
Note over Tests: OpenAPI client gen
and
- Note over Tests: Test UI
+ Note over Tests: React WWW tests
and
Note over Tests: Test examples<br>PROD image building
and
- Note over Tests: Build CI Images<br>Use original constraints
- Tests ->> GitHub Registry: Push CI Image Early cache + latest
- Note over Tests: Test 'breeze' image build quickly
+ Note over Tests: Test git clone on Windows
end
Tests ->> GitHub Registry: Push CI Images<br>[COMMIT_SHA]
GitHub Registry ->> Tests: Pull CI Images<br>[COMMIT_SHA]
@@ -220,17 +254,26 @@ sequenceDiagram
end
and
opt
- Note over Tests: Tests
+ Note over Tests: Unit Tests
+ end
+ and
+ opt
+ Note over Tests: Integration Tests
end
and
opt
Note over Tests: Test provider <br>packages build
end
+ and
+ opt
+ Note over Tests: Test airflow <br>packages build
+ end
and
opt
Note over Tests: Helm tests
end
end
+ Note over Tests: Summarize Warnings
Tests ->> GitHub Registry: Push PROD Images<br>[COMMIT_SHA]
GitHub Registry ->> Tests: Pull PROD Image<br>[COMMIT_SHA]
Note over Tests: Verify PROD Image<br>[COMMIT_SHA]
@@ -240,7 +283,7 @@ sequenceDiagram
end
and
opt
- Note over Tests: Run Kubernetes <br>upgrade tests
+ Note over Tests: Run docker-compose <br>tests
end
end
Note over Tests: Generate constraints
@@ -274,13 +317,11 @@ sequenceDiagram
and
Note over Tests: OpenAPI client gen
and
- Note over Tests: Test UI
+ Note over Tests: React WWW tests
and
Note over Tests: Test examples<br>PROD image building
and
Note over Tests: Build CI Images<br>Use original constraints
- Tests ->> GitHub Registry: Push CI Image Early cache + latest
- Note over Tests: Test 'breeze' image build quickly
end
Tests ->> GitHub Registry: Push CI Images<br>[COMMIT_SHA]
GitHub Registry ->> Tests: Pull CI Images<br>[COMMIT_SHA]
@@ -302,17 +343,26 @@ sequenceDiagram
end
and
opt
- Note over Tests: Tests
+ Note over Tests: Unit Tests
+ end
+ and
+ opt
+ Note over Tests: Integration Tests
end
and
opt
Note over Tests: Test provider <br>packages build
end
+ and
+ opt
+ Note over Tests: Test airflow <br>packages build
+ end
and
opt
Note over Tests: Helm tests
end
end
+ Note over Tests: Summarize Warnings
Tests ->> GitHub Registry: Push PROD Images<br>[COMMIT_SHA]
GitHub Registry ->> Tests: Pull PROD Image<br>[COMMIT_SHA]
Note over Tests: Verify PROD Image<br>[COMMIT_SHA]
@@ -322,7 +372,7 @@ sequenceDiagram
end
and
opt
- Note over Tests: Run Kubernetes <br>upgrade tests
+ Note over Tests: Run docker-compose <br>tests
end
end
Note over Tests: Generate constraints
@@ -333,5 +383,6 @@ sequenceDiagram
Tests ->> GitHub Registry: Push PROD Image cache + latest
Tests -->> Airflow Repo: Status update
deactivate Airflow Repo
+ Note over Tests: Build ARM CI images
deactivate Tests
```
