This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 589d907 Fix pod_mutation_hook for 1.10.13 (#10850)
new 079203c Breeze was slightly too chatty when there was no dirs created
(#10346)
new 978ab75 More informative description of Breeze's --verbose flag.
(#10348)
new 69f4bb4 Breeze: More fancy environment checking (#10329)
new e0837ac Add redbubble link to Airflow merch (#10359)
new 6ed632c Add to CONTRIBUTING.rst link to backport packages
troubleshooting (#10360)
new d6a6d58 Too much was happening in this pre-commit script (#10345)
new 6b7fb91 Update github flags for Breeze (#10384)
new f15c3e8 Docker images are now consistently labelled and a bit smaller
(#10387)
new 8cba53e Documentation artifact are also uploaded as GitHub Actions
Artifacts (#10158)
new 51beda1 When precommits are run, output is silenced (#10390)
new ad497fc Kubernetes image is extended rather than customized (#10399)
new bbc2cc4 Replaced aliases for common tools with functions. (#10402)
new fe93011 Constraint CI scripts are now separated out (#10404)
new ff38020 Move docker-compose ci.yml to ga.yml as it is GITHUB_* only
(#10405)
new bb7de06 Fixes optimisation where doc only change should build much
faster (#10344)
new 8138baa Fix broken breeze script (#10418)
new 69d13f4 CI Images are now pre-build and stored in registry (#10368)
new 16449f4 Fixes retrieval of correct branch in non-master related builds
new 8ed55e0 Replace file.io with artifacts (#10137)
new 8a6f76c Fix failing dependencies for FAB and Celery (#10828)
new 81cab49 Removes snakebite kerberos dependency (#10865)
new fe320fd Status of quarantined tests is stored in Github Issue (#10119)
new 88c0bef You can disable spellcheck or documentation when building
docs. (#10377)
new 26e716e Enable optimisation of image building. (#10422)
new 7bfc1cc Fix failing breeze (#10424)
new f401118 Switch to released cancel-workflow-runs action (#10423)
new e101e70 Fix Breeze failure on MacOS (#10440)
new 806ec31 Be nice to fork repositories when it comes to scheduled
events (#10448)
new f5572df Do not override in_container scripts when building the image
(#10442)
new bd94540 Fix port number in webserver for kind setup (#10452)
new 7d2a44f Mount gcloud kubeconifg to breeze (#10439)
new c506ba3 Stops running workflow_run for scheduled runs in forks
(#10473)
new da2a0c4 Fixes quoting bug introduced in #10473 (#10477)
new 0bc733e Fixes problem with Python image not ready to be pushed
(#10430)
new 96f831b Optimise production image building during k8s tests on CI
(#10476)
new bed142c GitHub Registry is now lowercase (#10489)
new 4256ba8 Fixes uploading of doc artifacts. (#10441)
new 539cd86 Mounting from sources is disabled for tests (#10472)
new fd62298 Sets default timeout for the job waiting for images (#10517)
new 8590509 Add a possibility to switch back to building images by secret
(#10509)
new ee10510 The PIP version is not pinned to 19.0.2 any more (#10542)
new 3d16f20 Helm Docker image sources are now included in the Airlfow
codebase (#9650)
new 225b31e updated official docker images in README (#10579)
new 6245c03 Add 1.10.12 to breeze-complete
new 37a8535 Improve output of check_environment.sh (#10631)
new bb32248 Move roles to CONTRIBUTING.rst (#10327)
new 97f376a Small fixes in Breeze/Static check/docs documentation (#10658)
new d802355 Improve getting started section (#10680)
new 68124ce Implement Google Shell Conventions for breeze script (#10695)
new 628d9e2 Add packages to function names in bash (#10670) (#10696)
new f2e8b96 Switches to better BATS asserts (#10718)
new 7d9bd00 clean-logs script for Dockerfile: trim logs before sleep
(#10685)
new 395f55a Switch to downloaded pgbouncer_exporter (#10759)
new 3486d32 Check all dockerfiles with hadolint (#10754)
new 2521a10 The verbose functions will not exit immediately if not asked
to (#10731)
new ab9bfbb Make ci/scripts/pre-commit Google Shell Guide compatible
(#10748)
new 9d60646 The scripts to run tests properly initialises constants
(#10769)
new 3d8d447 Make static checks Google Shell Guide compatible (#10750)
new 51ccd28 Fixes pre-commit failing on build step (#10785)
new b7a0b09 Move dev docker images to airflow registry (#9652)
new c4562d6 Check that all pre-commits are synchronized code<>docs
(#10789)
new 25a0a34 Add section for official source code (#10678)
new 0410f26 Refactor official source section to use bullets (#10801)
new e8f5d7f Make script/ci/images Google Shell Guide compatible (#10745)
new 691bfec Make script/ci/kubernetes Google Shell Guide Compatible
(#10746)
new 266e379 Fixed wrong "-e" on md5 file status check (#10803)
new faa9ded Fix integration tests being accidentally excluded (#10807)
new 7fd03c8 Make airflow testing Google Shell Guide compatible (#10813)
new 3d8fe9b Make scripts/ci/tools Google Shell Guide Compatible (#10811)
new 58c8ee8 Make dockerfiles Google Shell Guide Compliant (#10734)
new ca01cc1 Rename "Beyond the Horizon" section and refactor content
(#10802)
new 8321b7a Refactor content to a markdown table (#10863)
new 9770620 Fix `breeze -i` error (#10887)
new 1309c8a Flag --start-airflow for breeze (#10837)
new f209717 The entrypoints in Docker Image should be owned by Airflow
(#10853)
new b6dccfa Remove errors raised during initialiation of virtualenv
(#10896)
new fc05533 Add pre-commit to sort INTHEWILD.md file automatically
(#10851)
new dff8457 Fix syntax error in Dockerfile 'maintainer' Label (#10899)
new a93206e Add missing closing bracket in CI.rst (#10908)
new f3dde36 Make the usage of bash 'shift' consistent across Breeze
(#10907)
new 56e48e3 Fix typos in BREEZE.rst (#10905)
new d5259e2 Fix grammar in BREEZE.rst (#10904)
new fb97a25 Fix typos in scripts/ci/docker-compose/local.yml (#10906)
new ce96783 Make vrious scripts Google Shell Guide compatible (#10812)
new bb52512 Make breeeze-complete Google Shell Guide compatible (#10708)
new 5a41c06 Remove unknown pytest.ini setting (#10923)
new b497c26 Cache for kubernetes tests is updateable
new dd89809 Github repository can be overridden in command line by Breeze
The 88 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.dockerignore | 8 +-
.github/workflows/build-images-workflow-run.yml | 259 ++
.github/workflows/ci.yml | 643 +++--
.github/workflows/scheduled_quarantined.yml | 117 +
.pre-commit-config.yaml | 71 +-
BREEZE.rst | 217 +-
CI.rst | 651 ++++-
CONTRIBUTING.rst | 252 +-
Dockerfile | 61 +-
Dockerfile.ci | 45 +-
IMAGES.rst | 11 +-
INTHEWILD.md | 388 +++
README.md | 420 +--
STATIC_CODE_CHECKS.rst | 90 +-
TESTING.rst | 2 +-
UPDATING.md | 11 +
breeze | 2901 +++++++++++++-------
breeze-complete | 306 ++-
CONTRIBUTING.md => chart/dockerfiles/README.md | 10 +-
chart/dockerfiles/pgbouncer-exporter/.gitignore | 1 +
chart/dockerfiles/pgbouncer-exporter/Dockerfile | 41 +
.../pgbouncer-exporter/build_and_push.sh | 96 +
chart/dockerfiles/pgbouncer/Dockerfile | 73 +
.../dockerfiles/pgbouncer/build_and_push.sh | 29 +-
.../dockerfiles/statsd-exporter/Dockerfile | 29 +-
.../dockerfiles/statsd-exporter/build_and_push.sh | 29 +-
chart/dockerfiles/statsd-exporter/mappings.yml | 87 +
chart/values.yaml | 12 +-
common/_files_for_rebuild_check.sh | 30 -
common/_image_variables.sh | 31 -
confirm | 2 +-
dev/sign.sh | 10 +-
docs/{build => build_docs.py} | 20 +-
docs/start_doc_server.sh | 5 +-
kubernetes_tests/test_kubernetes_pod_operator.py | 2 +-
pytest.ini | 1 -
...ate_constraints.sh => ci_branch_constraints.sh} | 19 +-
.../ci_commit_constraints.sh} | 34 +-
scripts/ci/constraints/ci_generate_constraints.sh | 10 +-
scripts/ci/docker-compose/base.yml | 14 +
.../docker-compose/{backend-sqlite.yml => ci.yml} | 10 +-
.../ci/docker-compose/files.yml | 10 +-
.../docker-compose/{backend-sqlite.yml => ga.yml} | 10 +-
scripts/ci/docker-compose/integration-kerberos.yml | 2 +-
scripts/ci/docker-compose/integration-openldap.yml | 4 +-
...gration-rabbitmq.yml => integration-presto.yml} | 8 +-
scripts/ci/docker-compose/local-prod.yml | 3 +-
scripts/ci/docker-compose/local.yml | 20 +-
scripts/ci/dockerfiles/apache-rat/Dockerfile | 58 +
.../dockerfiles/apache-rat/build_and_push.sh} | 34 +-
scripts/ci/dockerfiles/bats/Dockerfile | 48 +
scripts/ci/dockerfiles/bats/build_and_push.sh | 53 +
.../ci/dockerfiles/bats/load.bash | 12 +-
scripts/ci/dockerfiles/krb5-kdc-server/Dockerfile | 64 +
.../dockerfiles/krb5-kdc-server/build_and_push.sh} | 30 +-
.../krb5-kdc-server/krb-conf/client}/krb5.conf | 9 +-
.../krb5-kdc-server/krb-conf/server/kadm5.acl | 2 +-
.../krb5-kdc-server/krb-conf/server/kdc.conf} | 24 +-
.../krb5-kdc-server/start_kdc.sh} | 50 +-
.../dockerfiles/krb5-kdc-server/supervisord.conf | 40 +
scripts/ci/dockerfiles/stress/Dockerfile | 34 +
.../stress/build_and_push.sh} | 36 +-
scripts/ci/docs/ci_docs.sh | 27 +-
scripts/ci/images/ci_build_dockerhub.sh | 12 +-
scripts/ci/images/ci_prepare_ci_image_on_ci.sh | 42 +
scripts/ci/images/ci_prepare_prod_image_on_ci.sh | 32 +-
.../{ci_push_ci_image.sh => ci_push_ci_images.sh} | 6 +-
scripts/ci/images/ci_push_production_images.sh | 6 +-
scripts/ci/images/ci_wait_for_all_ci_images.sh | 56 +
scripts/ci/images/ci_wait_for_all_prod_images.sh | 58 +
scripts/ci/in_container/check_environment.sh | 182 --
scripts/ci/in_container/run_docs_build.sh | 28 -
scripts/ci/in_container/run_flake8.sh | 38 -
scripts/ci/in_container/run_mypy.sh | 37 -
.../ci/kubernetes/ci_deploy_app_to_kubernetes.sh | 27 +-
scripts/ci/kubernetes/ci_run_helm_testing.sh | 6 +-
scripts/ci/kubernetes/ci_run_kubernetes_tests.sh | 45 +-
scripts/ci/libraries/_all_libs.sh | 10 +-
scripts/ci/libraries/_build_images.sh | 586 ++--
.../ci/libraries/_docker.env | 20 +-
scripts/ci/libraries/_initialization.sh | 735 +++--
scripts/ci/libraries/_kind.sh | 162 +-
scripts/ci/libraries/_local_mounts.sh | 11 +-
scripts/ci/libraries/_md5sum.sh | 54 +-
scripts/ci/libraries/_parameters.sh | 12 +-
scripts/ci/libraries/_permissions.sh | 22 +-
scripts/ci/libraries/_push_pull_remove_images.sh | 237 +-
scripts/ci/libraries/_runs.sh | 32 +-
scripts/ci/libraries/_sanity_checks.sh | 63 +-
scripts/ci/libraries/_script_init.sh | 25 +-
scripts/ci/libraries/_spinner.sh | 2 +-
scripts/ci/libraries/_start_end.sh | 50 +-
.../ci_bat_tests.sh => libraries/_traps.sh} | 33 +-
scripts/ci/libraries/_verbosity.sh | 119 +-
scripts/ci/pre_commit/pre_commit_bat_tests.sh | 10 +-
.../ci/pre_commit/pre_commit_breeze_cmd_line.sh | 34 +-
.../ci/pre_commit/pre_commit_check_integrations.sh | 17 +-
scripts/ci/pre_commit/pre_commit_check_license.sh | 6 +-
.../ci/pre_commit/pre_commit_check_pre_commits.sh | 66 +
scripts/ci/pre_commit/pre_commit_ci_build.sh | 8 +-
scripts/ci/pre_commit/pre_commit_flake8.sh | 7 +-
.../ci/pre_commit/pre_commit_lint_dockerfile.sh | 7 +-
.../ci/pre_commit/pre_commit_local_yml_mounts.sh | 18 +-
scripts/ci/pre_commit/pre_commit_mypy.sh | 7 +-
.../pre_commit_sort_in_the_wild.sh} | 14 +-
.../{ci_bat_tests.sh => bats_tests.sh} | 10 +-
.../{ci_check_license.sh => check_license.sh} | 16 +-
scripts/ci/static_checks/ci_flake8.sh | 65 -
scripts/ci/static_checks/ci_mypy.sh | 52 -
.../static_checks/{ci_bat_tests.sh => flake8.sh} | 24 +-
.../{ci_lint_dockerfile.sh => lint_dockerfile.sh} | 16 +-
.../ci/static_checks/{ci_bat_tests.sh => mypy.sh} | 25 +-
...i_run_static_checks.sh => run_static_checks.sh} | 14 +-
scripts/ci/testing/ci_run_airflow_testing.sh | 141 +-
.../ci/tools/ci_check_if_tests_should_be_run.sh | 17 +-
scripts/ci/tools/ci_clear_tmp.sh | 19 +-
scripts/ci/tools/ci_count_changed_files.sh | 17 +-
scripts/ci/tools/ci_fix_ownership.sh | 19 +-
.../in_container/_in_container_script_init.sh | 4 +-
.../{ci => }/in_container/_in_container_utils.sh | 72 +-
scripts/{ci => }/in_container/airflow_ci.cfg | 0
scripts/in_container/check_environment.sh | 184 ++
.../{ci => }/in_container/configure_environment.sh | 2 +-
scripts/{ci => }/in_container/entrypoint_ci.sh | 62 +-
scripts/{ci => }/in_container/entrypoint_exec.sh | 2 +-
scripts/{ci => }/in_container/krb5/krb5.conf | 0
.../prod}/airflow_scheduler_autorestart.sh | 0
scripts/{ => in_container}/prod/clean-logs.sh | 6 +-
scripts/{ => in_container}/prod/entrypoint_prod.sh | 4 +-
.../in_container/quarantine_issue_header.md | 16 +-
scripts/{ci => }/in_container/run_ci_tests.sh | 29 +-
scripts/{ci => }/in_container/run_clear_tmp.sh | 2 +-
scripts/{ci => }/in_container/run_cli_tool.sh | 17 +-
.../run_docs_build.sh} | 25 +-
scripts/{ci => }/in_container/run_extract_tests.sh | 2 +-
scripts/{ci => }/in_container/run_fix_ownership.sh | 2 +-
.../run_flake8.sh} | 5 +-
.../in_container/run_generate_constraints.sh | 7 +-
.../run_clear_tmp.sh => in_container/run_mypy.sh} | 6 +-
scripts/{ci => }/in_container/run_system_tests.sh | 10 +-
.../in_container/update_quarantined_test_status.py | 243 ++
setup.py | 7 +-
tests/bats/bats_utils.bash | 16 +-
.../bats/mocks/docker.sh | 2 +-
.../_default_branch.sh => tests/bats/mocks/helm.sh | 3 +-
.../_common_values.sh => tests/bats/mocks/kind.sh | 2 +-
.../bats/mocks/kubectl.sh | 2 +-
tests/bats/test_breeze_complete.bats | 151 +
tests/bats/test_breeze_params.bats | 112 +-
tests/bats/test_empty_test.bats | 4 +-
tests/bats/test_local_mounts.bats | 12 +-
tests/kubernetes/pod.yaml | 2 +-
tests/kubernetes/test_pod_generator.py | 4 +-
153 files changed, 8060 insertions(+), 3960 deletions(-)
create mode 100644 .github/workflows/build-images-workflow-run.yml
create mode 100644 .github/workflows/scheduled_quarantined.yml
create mode 100644 INTHEWILD.md
copy CONTRIBUTING.md => chart/dockerfiles/README.md (64%)
create mode 100644 chart/dockerfiles/pgbouncer-exporter/.gitignore
create mode 100644 chart/dockerfiles/pgbouncer-exporter/Dockerfile
create mode 100755 chart/dockerfiles/pgbouncer-exporter/build_and_push.sh
create mode 100644 chart/dockerfiles/pgbouncer/Dockerfile
copy scripts/prod/clean-logs.sh =>
chart/dockerfiles/pgbouncer/build_and_push.sh (58%)
copy airflow/api/auth/backend/deny_all.py =>
chart/dockerfiles/statsd-exporter/Dockerfile (59%)
copy scripts/prod/clean-logs.sh =>
chart/dockerfiles/statsd-exporter/build_and_push.sh (57%)
create mode 100644 chart/dockerfiles/statsd-exporter/mappings.yml
delete mode 100644 common/_files_for_rebuild_check.sh
delete mode 100644 common/_image_variables.sh
rename docs/{build => build_docs.py} (94%)
copy scripts/ci/constraints/{ci_generate_constraints.sh =>
ci_branch_constraints.sh} (75%)
copy scripts/ci/{static_checks/ci_run_static_checks.sh =>
constraints/ci_commit_constraints.sh} (59%)
copy scripts/ci/docker-compose/{backend-sqlite.yml => ci.yml} (84%)
copy yamllint-config.yml => scripts/ci/docker-compose/files.yml (90%)
copy scripts/ci/docker-compose/{backend-sqlite.yml => ga.yml} (84%)
copy scripts/ci/docker-compose/{integration-rabbitmq.yml =>
integration-presto.yml} (91%)
create mode 100644 scripts/ci/dockerfiles/apache-rat/Dockerfile
copy scripts/{prod/clean-logs.sh =>
ci/dockerfiles/apache-rat/build_and_push.sh} (53%)
create mode 100644 scripts/ci/dockerfiles/bats/Dockerfile
create mode 100755 scripts/ci/dockerfiles/bats/build_and_push.sh
copy .bash_completion => scripts/ci/dockerfiles/bats/load.bash (78%)
mode change 100755 => 100644
create mode 100644 scripts/ci/dockerfiles/krb5-kdc-server/Dockerfile
copy scripts/{prod/clean-logs.sh =>
ci/dockerfiles/krb5-kdc-server/build_and_push.sh} (57%)
copy scripts/ci/{in_container/krb5 =>
dockerfiles/krb5-kdc-server/krb-conf/client}/krb5.conf (88%)
copy airflow/api/client/__init__.py =>
scripts/ci/dockerfiles/krb5-kdc-server/krb-conf/server/kadm5.acl (97%)
copy scripts/ci/{in_container/krb5/krb5.conf =>
dockerfiles/krb5-kdc-server/krb-conf/server/kdc.conf} (67%)
copy scripts/ci/{libraries/_script_init.sh =>
dockerfiles/krb5-kdc-server/start_kdc.sh} (53%)
mode change 100755 => 100644
create mode 100644 scripts/ci/dockerfiles/krb5-kdc-server/supervisord.conf
create mode 100644 scripts/ci/dockerfiles/stress/Dockerfile
copy scripts/ci/{libraries/_script_init.sh =>
dockerfiles/stress/build_and_push.sh} (54%)
rename scripts/ci/images/{ci_push_ci_image.sh => ci_push_ci_images.sh} (90%)
create mode 100755 scripts/ci/images/ci_wait_for_all_ci_images.sh
create mode 100755 scripts/ci/images/ci_wait_for_all_prod_images.sh
delete mode 100755 scripts/ci/in_container/check_environment.sh
delete mode 100755 scripts/ci/in_container/run_docs_build.sh
delete mode 100755 scripts/ci/in_container/run_flake8.sh
delete mode 100755 scripts/ci/in_container/run_mypy.sh
copy .bash_completion => scripts/ci/libraries/_docker.env (76%)
mode change 100755 => 100644
copy scripts/ci/{static_checks/ci_bat_tests.sh => libraries/_traps.sh} (56%)
mode change 100755 => 100644
create mode 100755 scripts/ci/pre_commit/pre_commit_check_pre_commits.sh
copy scripts/ci/{constraints/ci_generate_constraints.sh =>
pre_commit/pre_commit_sort_in_the_wild.sh} (77%)
copy scripts/ci/static_checks/{ci_bat_tests.sh => bats_tests.sh} (81%)
rename scripts/ci/static_checks/{ci_check_license.sh => check_license.sh} (80%)
delete mode 100755 scripts/ci/static_checks/ci_flake8.sh
delete mode 100755 scripts/ci/static_checks/ci_mypy.sh
copy scripts/ci/static_checks/{ci_bat_tests.sh => flake8.sh} (62%)
rename scripts/ci/static_checks/{ci_lint_dockerfile.sh => lint_dockerfile.sh}
(76%)
rename scripts/ci/static_checks/{ci_bat_tests.sh => mypy.sh} (65%)
rename scripts/ci/static_checks/{ci_run_static_checks.sh =>
run_static_checks.sh} (81%)
rename scripts/{ci => }/in_container/_in_container_script_init.sh (90%)
rename scripts/{ci => }/in_container/_in_container_utils.sh (79%)
rename scripts/{ci => }/in_container/airflow_ci.cfg (100%)
create mode 100755 scripts/in_container/check_environment.sh
rename scripts/{ci => }/in_container/configure_environment.sh (96%)
rename scripts/{ci => }/in_container/entrypoint_ci.sh (78%)
rename scripts/{ci => }/in_container/entrypoint_exec.sh (92%)
rename scripts/{ci => }/in_container/krb5/krb5.conf (100%)
rename {airflow/bin =>
scripts/in_container/prod}/airflow_scheduler_autorestart.sh (100%)
rename scripts/{ => in_container}/prod/clean-logs.sh (100%)
rename scripts/{ => in_container}/prod/entrypoint_prod.sh (97%)
copy requirements/REMOVE.md => scripts/in_container/quarantine_issue_header.md
(60%)
copy scripts/{ci => }/in_container/run_ci_tests.sh (57%)
copy scripts/{ci => }/in_container/run_clear_tmp.sh (92%)
rename scripts/{ci => }/in_container/run_cli_tool.sh (91%)
rename scripts/{ci/in_container/run_ci_tests.sh =>
in_container/run_docs_build.sh} (64%)
rename scripts/{ci => }/in_container/run_extract_tests.sh (93%)
rename scripts/{ci => }/in_container/run_fix_ownership.sh (92%)
copy scripts/{ci/in_container/run_clear_tmp.sh => in_container/run_flake8.sh}
(89%)
rename scripts/{ci => }/in_container/run_generate_constraints.sh (86%)
rename scripts/{ci/in_container/run_clear_tmp.sh => in_container/run_mypy.sh}
(84%)
rename scripts/{ci => }/in_container/run_system_tests.sh (86%)
create mode 100755 scripts/in_container/update_quarantined_test_status.py
copy common/_common_values.sh => tests/bats/mocks/docker.sh (90%)
mode change 100644 => 100755
rename common/_default_branch.sh => tests/bats/mocks/helm.sh (90%)
mode change 100644 => 100755
copy common/_common_values.sh => tests/bats/mocks/kind.sh (90%)
mode change 100644 => 100755
rename common/_common_values.sh => tests/bats/mocks/kubectl.sh (90%)
mode change 100644 => 100755
create mode 100644 tests/bats/test_breeze_complete.bats
