This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a change to branch v2-6-stable
in repository https://gitbox.apache.org/repos/asf/airflow.git
from 58fca5eb3c Update RELEASE_NOTES.rst
add 0fa7270d95 Add instructions on how to avoid accidental airflow
upgrade/downgrade (#30813)
add 3858c51aa1 Fix table formatting in docker build documentation (#31472)
add 27ec3fb419 Add description in a few more places about adding airflow
to pip install (#31448)
add f6cda8fb63 Mark `[secrets] backend_kwargs` as a sensitive config
(#31788)
add aeb4ef3679 Fix overriding `default_args` in nested task groups (#31608)
add 8f343975a1 Fix wrong update for nested group default args (#31776)
add cc3cd20dce Fix typing for POST user endpoint (#31767)
add 5b41ed8209 Fix failing get_safe_url tests for latest Python 3.8 and
3.9 (#31766)
add b9f531697a Few doc fixes - links, grammer and wording (#31719)
add b87c9b1424 Add description on security issue handling in Airflow
(#31160)
add 3563023554 Add severity rating explanation to our Security Policy
(#31702)
add e39ca99398 Add an example for dynamic task mapping with non-TaskFlow
operator (#29762)
add d73cc40435 Add fullname to owner on logging (#30185)
add fd0719ddf5 Fix DAG.access_control can't sync when clean access_control
(#30340)
add 99241948e8 Fix ExternalTaskSensor to work correctly with task groups
(#30742)
add bb86b468ae Add broad-strokes 'security model' (#30843)
add bd7cd41bfa Add Python 3.11 support (#27264)
add 3594da07e9 apply static checks
add e972144197 Save scheduler execution time during search for queued
dag_runs (#30699)
add 945c20e37d Optimize scheduler by skipping "non-scheduable" DAGs
(#30706)
add 1a8d05d7d9 Executor events are not always "exited" here (#30859)
add 71bcb669ae Include rediss to the list of supported URL schemes (#31028)
add 9682a125a4 Update callbacks documentation (errors and context) (#31116)
add f03324137f Validate connection IDs (#31140)
add 9d28825e98 Added spinner activity while the logs load (#31165)
add 78bf9dd6b0 Use only one line for tmp dir log (#31170)
add c5f79d833b Don't present an undefined execution date (#31196)
add cb785eae47 Fix `url_for_asset` fallback and 404 on DAG Audit Log
(#31233)
add 2e5946c389 Fix task.branch error caused by incorrect initial parameter
(#31265)
add 948a0ce56a Make connection id validation consistent across interface
(#31282)
add 5d9d97291c Make BaseJobRunner a generic on the job class (#31287)
add 53a161bc92 Added metavar and choices to --state flag in airflow dags
list-jobs CLI for suggesting valid state arguments. (#31308)
add d2ef9dd287 Upgrade FAB to 4.3.1 (#31203)
add b1e5021c9b Rephrase comment in setup.py (#31312)
add caab2a95fa Convert dask upper-binding into exclusion (#31329)
add 8f2c6e1205 Add an example for sql_alchemy_connect_args conf (#31332)
add c3ba1d35e1 Update the output of airflow info command in the doc
(#31336)
add 0b9ca46fdb Fix crash when clearing run with task from normal to mapped
(#31352)
add e28c88f1bb Skip auto-document things with :meta private: (#31380)
add f529d150aa Fix dropdown default and adjust tutorial to use 42 as
default for proof (#31400)
add c3e388a8f5 Fix error handling when pre-importing modules in DAGs
(#31401)
add d60e0a55c5 Fix `max_active_tis_per_dagrun` for Dynamic Task Mapping
(#31406)
add 271b33af26 Ensure min backoff in base sensor is at least 1 (#31412)
add 5d2bcd8519 Add missing BashOperator import to documentation example
(#31436)
add 38c290ab15 Wording the next and last run DAG columns better (#31467)
add 557b3765ac Fix version support information (#31468)
add c01e029df0 Use kube_client over default CoreV1Api for deleting pods
(#31477)
add 818f0740ff Remove note about triggerer being 3.7+ only (#31483)
add 46c0da8dbd Fix typing in external task triggers (#31490)
add d26144bb20 Remove dependency already registered for this task warning
(#31502)
add 0d82a367f1 Remove typo in docs serializers.rst (#31507)
add b6e0c36160 Include triggerer health status in Airflow /health endpoint
(#31529)
add bb62a46a38 fix airflow users delete CLI command (#31539)
add a1598ed80d Add the missing `map_index` to the xcom key when skipping
downstream tasks (#31541)
add 7295524620 Fix sorting of tags (#31553)
add 2de0656151 Remove found_descendents param from get_flat_relative_ids
(#31559)
add 243a7ddce2 Resolving issue where Grid won't un-collapse when Details
is collapsed (#31561)
add 1cfdefeba3 Workaround mypy confusing default with necessary arguments
(#31562)
add aa5a0ede3e Updating UI screenshots in docs (#31577)
add bbc2f0becf Add TriggererStatus to OpenAPI spec (#31579)
add bbdc2a4e6f Hide old alert box when testing conn with different value
(#31606)
add 6605a8ffa1 fixing broken links in openapi/v1.yaml (#31619)
add af11a19369 Add docstring and signature for _read_remote_logs (#31623)
add 84353693d2 📝 Update release documentation (#31631)
add a8e1899a99 remove ducplicate entry in the metrics doc (#31654)
add 1a9a652754 Fix wrong cron syntax (#31666)
add 3f808aad6d update path of KubernetesPodOperator in core docs (#31683)
add 8cd9c4a00b Replace unicodecsv with standard csv library (#31693)
add 186df9f1bf Add back accidentally removed `create_session` in
utils/db.py (#31705)
add 3bbf6fd73a Control permissibility of driver config in extra from
airflow.cfg (#31754)
add ea8c0a1f0f Use keyword paramaters for migration methods for mssql
(#31309)
add f2479f48bd Fix Kubernetes executor set wrong task status (#31274)
add 47438559e3 Update RELEASE_NOTES.rst
add cfbe2c2aad Check Logging Handlers and Secret Backends for recursion
(#31385)
add 9ae933f67b Add libgeos-dev to dev dependencies in CI image (#31437)
add c529b069b2 Update SDKs for google provider package (#30067)
add 0eb2722070 Export SUSPENDED_PROVIDERS_FOLDERS for breeze testing
commands (#30780)
add b042a81d41 Speed up test collection (#30801)
add 65ddb288d0 Optimize away pytest collection steps (#30824)
add 3c20888453 Only run mssql tests on self-hosted runners (#30831)
add 2ed643e59c Speed up verification and instalation of providers in CI
(#30839)
add e8fc7197df Bring back unicodecsv as depenendency of Airflow (#31814)
add 1573907d41 Update RELEASE_NOTES.rst
add 9323ab1e2b Synchronize dev scripts with main
add 04b535b326 Remove yandex from 2.6 branch
add aeaf0513d5 Fix ERD diagram generated in 3.7 image
add 17392bcd63 Fix selective checks skipping provider tests in non main
branch (#31821)
add 554eb0e8eb Update release notes
add 4a688fb2fe Fix spelling
add 8ba85baa3a Fix failing AIP-52 test
add 550076b24c Skip instalation of provider sdist packages when provider
tests skipped (#31827)
add f2065f2128 Only run builds on "push" events for v* test branch (#31825)
add c72cecef11 Check that breeze image builds quickly only on main branch
(#31828)
add 5679a01919 Use single source of truth for sensitive config items
(#31820)
add d7ba536645 Update release notes
add e19ff6f620 Cascade update of taskinstance to TaskMap table (#31445)
add 91cbb84bc9 Update release note and revision head
add 0726552448 Fix Update Airflow version to 2.6.2
add 930c862034 Change working directory for breeze unit tests (#31839)
No new revisions were added by this update.
Summary of changes:
.github/CODEOWNERS | 11 +-
.github/ISSUE_TEMPLATE/airflow_bug_report.yml | 2 +-
.../airflow_helmchart_bug_report.yml | 3 +-
.../airflow_providers_bug_report.yml | 3 +
.github/SECURITY.rst | 88 +++-
.github/actions/breeze/action.yml | 6 +-
.github/actions/build-ci-images/action.yml | 8 +
.github/workflows/build-images.yml | 19 +-
.github/workflows/ci.yml | 505 ++++++++++---------
.github/workflows/codeql-analysis.yml | 4 +-
.github/workflows/recheck-old-bug-report.yml | 2 +-
.github/workflows/release_dockerhub_image.yml | 7 +-
.github/workflows/stale.yml | 4 +-
.pre-commit-config.yaml | 15 +-
BREEZE.rst | 225 +++++++--
CI.rst | 8 +-
COMMITTERS.rst | 6 +
CONTRIBUTING.rst | 2 +-
Dockerfile | 4 +-
Dockerfile.ci | 59 ++-
IMAGES.rst | 2 +-
INSTALL | 2 +-
LOCAL_VIRTUALENV.rst | 12 +-
README.md | 147 +-----
RELEASE_NOTES.rst | 78 +++
STATIC_CODE_CHECKS.rst | 14 +-
airflow/__init__.py | 3 +-
.../common/airflow_health.py} | 30 +-
airflow/api/common/mark_tasks.py | 4 +-
.../api_connexion/endpoints/connection_endpoint.py | 5 +
airflow/api_connexion/endpoints/health_endpoint.py | 49 +-
airflow/api_connexion/endpoints/user_endpoint.py | 2 +-
airflow/api_connexion/openapi/v1.yaml | 31 +-
airflow/cli/cli_config.py | 9 +-
airflow/cli/commands/connection_command.py | 14 +-
airflow/cli/commands/kubernetes_command.py | 4 +-
airflow/cli/commands/user_command.py | 3 +
airflow/config_templates/config.yml | 23 +-
airflow/config_templates/default_airflow.cfg | 11 +
airflow/config_templates/default_celery.py | 17 +-
airflow/configuration.py | 32 +-
...amic_task_mapping_with_no_taskflow_operators.py | 62 +++
airflow/example_dags/example_params_ui_tutorial.py | 4 +-
airflow/executors/kubernetes_executor.py | 10 +-
airflow/hooks/subprocess.py | 2 +-
airflow/jobs/backfill_job_runner.py | 4 +-
airflow/jobs/base_job_runner.py | 13 +-
airflow/jobs/dag_processor_job_runner.py | 2 +-
airflow/jobs/local_task_job_runner.py | 2 +-
airflow/jobs/scheduler_job_runner.py | 39 +-
airflow/jobs/triggerer_job_runner.py | 2 +-
...3_1_10_4_make_taskinstance_pool_not_nullable.py | 12 +-
.../versions/0054_1_10_10_add_dag_code_table.py | 6 +-
.../0060_2_0_0_remove_id_column_from_xcom.py | 2 +-
...89_2_2_0_make_xcom_pkey_columns_non_nullable.py | 4 +-
.../0102_2_3_0_switch_xcom_table_to_use_run_id.py | 6 +-
... 0125_2_6_2_add_onupdate_cascade_to_taskmap.py} | 40 +-
airflow/models/abstractoperator.py | 25 +-
airflow/models/dag.py | 6 +-
airflow/models/dagbag.py | 28 +-
airflow/models/skipmixin.py | 3 +
airflow/models/taskinstance.py | 10 +-
airflow/models/taskmap.py | 1 +
airflow/models/taskmixin.py | 15 +-
airflow/operators/python.py | 9 +-
airflow/operators/subdag.py | 7 +-
airflow/providers/amazon/aws/hooks/dms.py | 2 +-
.../providers/amazon/aws/log/s3_task_handler.py | 2 +-
airflow/providers/apache/beam/provider.yaml | 2 +-
airflow/providers/apache/hive/hooks/hive.py | 6 +-
airflow/providers/apache/hive/provider.yaml | 5 +
.../apache/hive/transfers/mssql_to_hive.py | 6 +-
.../apache/hive/transfers/mysql_to_hive.py | 41 +-
.../apache/hive/transfers/vertica_to_hive.py | 7 +-
airflow/providers/google/ads/hooks/ads.py | 22 +-
.../_internal_client/secret_manager_client.py | 2 +-
.../google/cloud/example_dags/example_compute.py | 107 -----
.../cloud/example_dags/example_compute_ssh.py | 90 ----
airflow/providers/google/cloud/hooks/automl.py | 9 +
airflow/providers/google/cloud/hooks/bigquery.py | 37 +-
airflow/providers/google/cloud/hooks/bigtable.py | 2 +
airflow/providers/google/cloud/hooks/cloud_sql.py | 15 +-
airflow/providers/google/cloud/hooks/compute.py | 38 +-
.../providers/google/cloud/hooks/compute_ssh.py | 17 +
airflow/providers/google/cloud/hooks/mlengine.py | 43 +-
.../google/cloud/hooks/natural_language.py | 29 +-
airflow/providers/google/cloud/hooks/spanner.py | 4 +-
.../providers/google/cloud/hooks/speech_to_text.py | 7 +-
.../providers/google/cloud/hooks/text_to_speech.py | 9 +-
.../google/cloud/hooks/video_intelligence.py | 28 +-
airflow/providers/google/cloud/hooks/vision.py | 66 ++-
.../google/cloud/log/stackdriver_task_handler.py | 4 +-
airflow/providers/google/cloud/operators/automl.py | 27 +-
.../providers/google/cloud/operators/bigquery.py | 23 +-
.../providers/google/cloud/operators/compute.py | 7 +-
.../google/cloud/operators/natural_language.py | 17 +-
.../google/cloud/operators/speech_to_text.py | 2 +-
.../google/cloud/operators/translate_speech.py | 2 +-
.../google/cloud/operators/video_intelligence.py | 21 +-
airflow/providers/google/cloud/operators/vision.py | 17 +-
.../providers/google/cloud/operators/workflows.py | 6 +-
.../google/cloud/transfers/bigquery_to_gcs.py | 8 +-
.../google/cloud/transfers/gcs_to_bigquery.py | 5 +-
.../providers/google/cloud/transfers/sql_to_gcs.py | 18 +-
.../google/cloud/utils/bigquery_get_data.py | 7 +-
.../google/cloud/utils/mlengine_operator_utils.py | 4 +-
airflow/providers/google/provider.yaml | 95 ++--
.../microsoft/azure/log/wasb_task_handler.py | 2 +-
.../azure/transfers/oracle_to_azure_data_lake.py | 10 +-
.../providers/mysql/transfers/vertica_to_mysql.py | 6 +-
airflow/providers/odbc/hooks/odbc.py | 30 ++
.../providers/yandex/.latest-doc-only-change.txt | 1 -
airflow/providers/yandex/CHANGELOG.rst | 200 --------
airflow/providers/yandex/__init__.py | 16 -
airflow/providers/yandex/hooks/__init__.py | 16 -
airflow/providers/yandex/hooks/yandex.py | 176 -------
.../providers/yandex/hooks/yandexcloud_dataproc.py | 35 --
airflow/providers/yandex/operators/__init__.py | 16 -
.../yandex/operators/yandexcloud_dataproc.py | 525 --------------------
airflow/providers/yandex/provider.yaml | 70 ---
airflow/sensors/base.py | 3 +-
airflow/sensors/bash.py | 2 +-
airflow/sensors/external_task.py | 17 +-
airflow/timetables/base.py | 22 +-
airflow/timetables/simple.py | 12 +-
airflow/triggers/external_task.py | 17 +-
airflow/utils/db.py | 1 +
airflow/utils/file.py | 2 +-
airflow/utils/log/file_task_handler.py | 14 +-
airflow/utils/task_group.py | 5 +
airflow/www/decorators.py | 2 +-
airflow/www/extensions/init_manifest_files.py | 2 +-
airflow/www/forms.py | 3 +-
airflow/www/security.py | 17 +
airflow/www/static/js/api/useTaskLog.ts | 1 -
airflow/www/static/js/connection_form.js | 6 +
airflow/www/static/js/dag/Main.tsx | 6 +-
.../js/dag/details/taskInstance/Logs/index.tsx | 19 +-
airflow/www/static/js/dag/grid/index.tsx | 38 +-
airflow/www/static/js/task_instances.js | 4 +-
airflow/www/static/js/types/api-generated.ts | 35 +-
airflow/www/templates/airflow/dag_audit_log.html | 1 -
airflow/www/templates/airflow/dags.html | 4 +-
airflow/www/templates/airflow/trigger.html | 4 +-
airflow/www/validators.py | 21 +
airflow/www/views.py | 54 ++-
dev/README_RELEASE_AIRFLOW.md | 17 +-
dev/README_RELEASE_HELM_CHART.md | 15 +
dev/README_RELEASE_PROVIDER_PACKAGES.md | 32 ++
dev/breeze/README.md | 2 +-
dev/breeze/SELECTIVE_CHECKS.md | 111 +++--
.../0002-implement-standalone-python-command.md | 6 +-
.../0007-using-database-volumes-for-backends.md | 2 +-
dev/breeze/setup.cfg | 5 +-
.../src/airflow_breeze/commands/ci_commands.py | 25 +-
.../airflow_breeze/commands/ci_commands_config.py | 10 +-
.../airflow_breeze/commands/ci_image_commands.py | 36 +-
.../commands/ci_image_commands_config.py | 27 +-
.../airflow_breeze/commands/developer_commands.py | 117 ++++-
.../commands/developer_commands_config.py | 24 +-
.../airflow_breeze/commands/kubernetes_commands.py | 7 +-
.../src/airflow_breeze/commands/main_command.py | 6 +-
.../commands/minor_release_command.py | 2 +-
.../commands/production_image_commands.py | 34 +-
.../commands/production_image_commands_config.py | 27 +-
.../commands/release_candidate_command.py | 19 +-
.../src/airflow_breeze/commands/release_command.py | 6 +-
.../commands/release_management_commands.py | 228 ++++++++-
.../commands/release_management_commands_config.py | 29 +-
.../airflow_breeze/commands/testing_commands.py | 74 ++-
.../commands/testing_commands_config.py | 5 +
dev/breeze/src/airflow_breeze/global_constants.py | 87 +++-
.../airflow_breeze/params/common_build_params.py | 1 -
.../src/airflow_breeze/params/shell_params.py | 14 +-
dev/breeze/src/airflow_breeze/pre_commit_ids.py | 1 -
dev/breeze/src/airflow_breeze/utils/coertions.py | 6 +
.../src/airflow_breeze/utils/common_options.py | 36 +-
dev/breeze/src/airflow_breeze/utils/confirm.py | 4 +-
.../airflow_breeze/utils/docker_command_utils.py | 85 ++--
dev/breeze/src/airflow_breeze/utils/image.py | 29 +-
.../src/airflow_breeze/utils/kubernetes_utils.py | 6 +-
dev/breeze/src/airflow_breeze/utils/parallel.py | 8 +-
dev/breeze/src/airflow_breeze/utils/path_utils.py | 1 +
.../airflow_breeze/utils/provider_dependencies.py | 52 ++
dev/breeze/src/airflow_breeze/utils/registry.py | 57 +--
dev/breeze/src/airflow_breeze/utils/run_tests.py | 12 +-
dev/breeze/src/airflow_breeze/utils/run_utils.py | 15 +-
.../src/airflow_breeze/utils/selective_checks.py | 250 +++++++---
dev/breeze/tests/test_cache.py | 4 +-
dev/breeze/tests/test_docker_command_utils.py | 8 +-
dev/breeze/tests/test_exclude_from_matrix.py | 10 +-
dev/breeze/tests/test_pr_info.py | 10 +-
dev/breeze/tests/test_provider_dependencies.py | 47 ++
dev/breeze/tests/test_selective_checks.py | 535 +++++++++++++++++----
dev/provider_packages/prepare_provider_packages.py | 2 +-
dev/retag_docker_images.py | 2 +-
.../test_examples_of_prod_image_building.py | 3 +-
.../operators/cloud/compute_ssh.rst | 12 +-
.../operators/cloud/dataproc.rst | 4 +-
.../connections/odbc.rst | 9 +
docs/apache-airflow-providers-yandex/commits.rst | 260 ----------
.../connections/yandexcloud.rst | 62 ---
docs/apache-airflow-providers-yandex/index.rst | 93 ----
.../installing-providers-from-sources.rst | 18 -
docs/apache-airflow-providers-yandex/operators.rst | 33 --
.../logging-monitoring/callbacks.rst | 9 +
.../logging-monitoring/logging-tasks.rst | 22 +-
.../logging-monitoring/metrics.rst | 1 -
.../authoring-and-scheduling/deferring.rst | 5 -
.../dynamic-task-mapping.rst | 5 +-
.../authoring-and-scheduling/serializers.rst | 2 +-
docs/apache-airflow/best-practices.rst | 4 +-
docs/apache-airflow/core-concepts/dags.rst | 4 +-
docs/apache-airflow/extra-packages-ref.rst | 2 -
docs/apache-airflow/howto/custom-operator.rst | 2 +-
docs/apache-airflow/howto/docker-compose/index.rst | 11 +-
docs/apache-airflow/howto/notifications.rst | 7 +-
docs/apache-airflow/howto/set-config.rst | 13 +
.../howto/upgrading-from-1-10/index.rst | 2 +-
docs/apache-airflow/img/airflow_erd.sha256 | 2 +-
docs/apache-airflow/img/airflow_erd.svg | 136 +++---
docs/apache-airflow/img/grid.png | Bin 347155 -> 249561 bytes
docs/apache-airflow/img/grid_instance_details.png | Bin 287820 -> 178129 bytes
docs/apache-airflow/img/grid_mapped_task.png | Bin 334409 -> 135753 bytes
docs/apache-airflow/img/grid_run_details.png | Bin 243661 -> 202106 bytes
.../installation/installing-from-pypi.rst | 9 +
docs/apache-airflow/installation/prerequisites.rst | 6 +-
.../installation/supported-versions.rst | 2 +-
docs/apache-airflow/migrations-ref.rst | 4 +-
docs/apache-airflow/release-process.rst | 3 +
docs/apache-airflow/ui.rst | 14 +-
docs/conf.py | 6 +-
docs/docker-stack/README.md | 10 +-
docs/docker-stack/build.rst | 68 ++-
.../customizing/own-requirements.sh | 1 +
.../extending/add-apt-packages/Dockerfile | 2 +-
.../add-build-essential-extend/Dockerfile | 4 +-
.../extending/add-providers/Dockerfile | 4 +-
.../extending/add-pypi-packages/Dockerfile | 4 +-
.../extending/add-requirement-packages/Dockerfile | 4 +-
.../extending/custom-providers/Dockerfile | 4 +-
.../extending/embedding-dags/Dockerfile | 2 +-
.../extending/writable-directory/Dockerfile | 2 +-
docs/docker-stack/entrypoint.rst | 14 +-
docs/integration-logos/yandex/Yandex-Cloud.png | Bin 12347 -> 0 bytes
docs/spelling_wordlist.txt | 7 +
generated/provider_dependencies.json | 320 +++++++-----
images/breeze/output-commands-hash.txt | 89 ++--
images/breeze/output-commands.svg | 92 ++--
images/breeze/output_build-docs.svg | 4 +-
images/breeze/output_ci-image_build.svg | 98 ++--
images/breeze/output_ci-image_pull.svg | 66 +--
images/breeze/output_ci-image_verify.svg | 42 +-
images/breeze/output_ci.svg | 30 +-
images/breeze/output_ci_selective-check.svg | 44 +-
images/breeze/output_down.svg | 115 +++++
images/breeze/output_k8s_build-k8s-image.svg | 52 +-
images/breeze/output_k8s_configure-cluster.svg | 60 +--
images/breeze/output_k8s_create-cluster.svg | 10 +-
images/breeze/output_k8s_delete-cluster.svg | 34 +-
images/breeze/output_k8s_deploy-airflow.svg | 70 +--
images/breeze/output_k8s_k9s.svg | 32 +-
images/breeze/output_k8s_logs.svg | 34 +-
images/breeze/output_k8s_run-complete-tests.svg | 68 +--
images/breeze/output_k8s_shell.svg | 40 +-
images/breeze/output_k8s_status.svg | 36 +-
images/breeze/output_k8s_tests.svg | 68 +--
images/breeze/output_k8s_upload-k8s-image.svg | 58 +--
images/breeze/output_prod-image_build.svg | 140 +++---
images/breeze/output_prod-image_pull.svg | 66 +--
images/breeze/output_prod-image_verify.svg | 44 +-
images/breeze/output_release-management.svg | 22 +-
...put_release-management_generate-constraints.svg | 4 +-
...management_generate-issue-content-providers.svg | 34 +-
...elease-management_install-provider-packages.svg | 216 +++++++++
...e-management_prepare-provider-documentation.svg | 54 ++-
...elease-management_prepare-provider-packages.svg | 32 +-
...tput_release-management_release-prod-images.svg | 2 +-
...release-management_verify-provider-packages.svg | 52 +-
.../output_setup_check-all-params-in-groups.svg | 38 +-
images/breeze/output_setup_config.svg | 2 +-
.../output_setup_regenerate-command-images.svg | 20 +-
images/breeze/output_shell.svg | 116 +++--
images/breeze/output_start-airflow.svg | 42 +-
images/breeze/output_static-checks.svg | 154 +++---
images/breeze/output_stop.svg | 115 -----
.../breeze/output_testing_docker-compose-tests.svg | 54 ++-
images/breeze/output_testing_integration-tests.svg | 70 +--
images/breeze/output_testing_tests.svg | 114 +++--
licenses/LICENSE-unicodecsv.txt | 25 +
scripts/ci/docker-compose/_docker.env | 5 +-
scripts/ci/docker-compose/backend-mssql.yml | 1 -
scripts/ci/docker-compose/backend-mysql.yml | 1 -
scripts/ci/docker-compose/backend-postgres.yml | 1 -
scripts/ci/docker-compose/backend-sqlite.yml | 3 +-
scripts/ci/docker-compose/base.yml | 7 +-
scripts/ci/docker-compose/devcontainer.env | 9 +-
scripts/ci/docker-compose/devcontainer.yml | 2 +-
scripts/ci/docker-compose/forward-credentials.yml | 4 +-
scripts/ci/docker-compose/integration-celery.yml | 2 +
scripts/ci/docker-compose/integration-kafka.yml | 63 +++
.../ci/docker-compose/kafka/update_run.sh | 15 +-
.../ci_start_arm_instance_and_connect_to_docker.sh | 6 +-
...py => pre_commit_check_aiobotocore_optional.py} | 48 +-
scripts/ci/pre_commit/pre_commit_insert_extras.py | 4 +-
.../ci/pre_commit/pre_commit_local_yml_mounts.py | 6 +-
scripts/ci/pre_commit/pre_commit_mypy.py | 10 +-
.../ci/pre_commit/pre_commit_supported_versions.py | 2 +-
...elm_lint.py => pre_commit_unittest_testcase.py} | 39 +-
.../pre_commit_update_providers_dependencies.py | 18 +-
scripts/docker/entrypoint_ci.sh | 54 ++-
scripts/docker/install_os_dependencies.sh | 2 +-
scripts/in_container/bin/run_tmux | 11 +
scripts/in_container/check_environment.sh | 3 +
scripts/in_container/run_generate_constraints.sh | 22 +-
.../in_container/run_provider_yaml_files_check.py | 41 +-
scripts/in_container/verify_providers.py | 196 +++++---
scripts/tools/initialize_virtualenv.py | 2 +-
setup.cfg | 7 +-
setup.py | 32 +-
{airflow/api_connexion => tests/api}/__init__.py | 0
.../api_connexion => tests/api/common}/__init__.py | 0
tests/api/common/test_airflow_health.py | 100 ++++
.../endpoints/test_connection_endpoint.py | 14 +
.../api_connexion/endpoints/test_user_endpoint.py | 14 +
tests/cli/commands/test_connection_command.py | 10 +
tests/cli/commands/test_kubernetes_command.py | 4 +-
tests/core/test_configuration.py | 52 +-
tests/dag_processing/test_processor.py | 115 +++--
tests/decorators/test_python.py | 5 +-
tests/decorators/test_task_group.py | 74 +++
tests/executors/test_kubernetes_executor.py | 2 +
tests/jobs/test_scheduler_job.py | 40 +-
tests/models/test_dagbag.py | 3 +-
tests/models/test_dagrun.py | 26 +-
tests/models/test_param.py | 39 +-
tests/models/test_taskinstance.py | 95 +++-
tests/operators/test_python.py | 48 ++
tests/operators/test_subdag_operator.py | 11 +-
tests/providers/apache/hive/hooks/test_hive.py | 11 +-
.../apache/hive/transfers/test_mssql_to_hive.py | 5 +-
.../apache/hive/transfers/test_mysql_to_hive.py | 5 +-
.../_internal_client/test_secret_manager_client.py | 10 +-
.../providers/google/cloud/hooks/test_bigtable.py | 4 +-
.../providers/google/cloud/hooks/test_cloud_sql.py | 62 +--
.../hooks/test_cloud_storage_transfer_service.py | 10 +-
tests/providers/google/cloud/hooks/test_compute.py | 74 ++-
.../google/cloud/hooks/test_natural_language.py | 2 +-
tests/providers/google/cloud/hooks/test_pubsub.py | 9 +-
.../google/cloud/hooks/test_secret_manager.py | 6 +-
.../google/cloud/hooks/test_speech_to_text.py | 14 +-
.../google/cloud/hooks/test_text_to_speech.py | 11 +-
.../google/cloud/hooks/test_video_intelligence.py | 32 +-
tests/providers/google/cloud/hooks/test_vision.py | 45 +-
.../google/cloud/operators/test_automl.py | 2 +
.../google/cloud/operators/test_functions.py | 5 +-
.../cloud/operators/test_natural_language.py | 2 +-
.../google/cloud/operators/test_speech_to_text.py | 5 +-
.../cloud/operators/test_translate_speech.py | 2 +-
.../cloud/operators/test_video_intelligence.py | 9 +-
.../google/cloud/operators/test_vision.py | 2 +-
.../google/cloud/operators/test_workflows.py | 9 +-
.../google/cloud/transfers/test_sql_to_gcs.py | 124 ++---
.../google/cloud/triggers/test_cloud_build.py | 3 +
.../cloud/utils/test_mlengine_operator_utils.py | 16 +-
.../transfers/test_oracle_to_azure_data_lake.py | 7 +-
tests/providers/odbc/hooks/test_odbc.py | 26 +-
tests/sensors/test_base.py | 35 ++
tests/sensors/test_external_task_sensor.py | 57 +++
tests/serialization/test_dag_serialization.py | 4 +-
.../google/cloud/automl/example_automl_dataset.py | 4 +-
.../google/cloud/automl/example_automl_model.py | 18 +-
.../automl/example_automl_nl_text_extraction.py | 79 ++-
.../automl/example_automl_vision_classification.py | 66 ++-
.../cloud/bigquery/example_bigquery_queries.py | 1 +
.../bigquery/example_bigquery_queries_async.py | 2 +-
.../google/cloud/cloud_sql/example_cloud_sql.py | 5 -
.../cloud/composer/example_cloud_composer.py | 2 +-
.../composer/example_cloud_composer_deferrable.py | 2 +-
.../google/cloud/compute/example_compute.py | 12 +-
.../google/cloud/compute/example_compute_igm.py | 10 -
.../google/cloud/compute/example_compute_ssh.py | 32 +-
...tive_python_async.py => example_dataflow_go.py} | 119 ++---
.../cloud/dataflow/example_dataflow_native_java.py | 8 +
.../dataflow/example_dataflow_native_python.py | 6 +-
.../example_dataflow_native_python_async.py | 2 +-
.../google/cloud/dataflow/resources/wordcount.go | 230 +++++++++
.../google/cloud/dataprep/example_dataprep.py | 4 +-
.../cloud/dataproc/example_dataproc_batch.py | 4 +-
.../dataproc/example_dataproc_batch_persistent.py | 2 +-
.../example_dataproc_cluster_deferrable.py | 6 +-
.../dataproc/example_dataproc_cluster_generator.py | 2 +-
...pdate.py => example_dataproc_cluster_update.py} | 2 +-
.../google/cloud/dataproc/example_dataproc_gke.py | 28 +-
.../cloud/dataproc/example_dataproc_hadoop.py | 2 +-
.../google/cloud/dataproc/example_dataproc_hive.py | 2 +-
.../google/cloud/dataproc/example_dataproc_pig.py | 2 +-
.../cloud/dataproc/example_dataproc_pyspark.py | 2 +-
.../cloud/dataproc/example_dataproc_spark.py | 2 +-
.../cloud/dataproc/example_dataproc_spark_async.py | 2 +-
.../dataproc/example_dataproc_spark_deferrable.py | 2 +-
.../cloud/dataproc/example_dataproc_spark_sql.py | 2 +-
.../cloud/dataproc/example_dataproc_sparkr.py | 2 +-
.../example_dataproc_metastore.py | 7 +-
.../google/cloud/gcs/example_firestore.py | 28 +-
.../google/cloud/gcs/example_gcs_sensor.py | 4 -
.../cloud/gcs/example_gcs_to_bigquery_async.py | 2 +-
.../google/cloud/gcs/example_gcs_to_gcs.py | 2 +-
.../google/cloud/gcs/example_s3_to_gcs.py | 6 +-
.../google/cloud/gcs/example_trino_to_gcs.py | 59 ++-
.../{bigquery => gcs}/resources/us-states.csv | 0
.../google/cloud/ml_engine/example_mlengine.py | 16 +-
.../cloud/ml_engine/example_mlengine_async.py | 22 +-
.../natural_language/example_natural_language.py | 2 +-
.../cloud/sql_to_sheets/example_sql_to_sheets.py | 61 ++-
.../cloud/stackdriver/example_stackdriver.py | 12 +-
.../providers/google/cloud/tasks/example_queue.py | 5 +-
.../providers/google/cloud/tasks/example_tasks.py | 6 +-
.../example_vertex_ai_custom_container.py | 9 +-
.../vertex_ai/example_vertex_ai_custom_job.py | 3 +-
.../example_vertex_ai_custom_job_python_package.py | 10 +-
.../cloud/vertex_ai/example_vertex_ai_dataset.py | 4 +-
.../cloud/vertex_ai/example_vertex_ai_endpoint.py | 11 +-
.../vertex_ai/example_vertex_ai_model_service.py | 2 +-
...housing.zip => California-housing-ai-model.zip} | Bin
...zip => California-housing-custom-container.zip} | Bin
...using.zip => California-housing-custom-job.zip} | Bin
...g.zip => California-housing-python-package.zip} | Bin
.../cloud/vision/example_vision_annotate_image.py | 6 +-
.../cloud/vision/example_vision_autogenerated.py | 6 +-
.../google/cloud/vision/example_vision_explicit.py | 4 +-
tests/utils/test_db.py | 3 +
tests/utils/test_task_group.py | 80 +++
tests/www/test_validators.py | 35 ++
tests/www/views/test_views.py | 2 +-
tests/www/views/test_views_acl.py | 6 +-
tests/www/views/test_views_configuration.py | 8 +-
tests/www/views/test_views_connection.py | 11 +
tests/www/views/test_views_home.py | 54 ++-
439 files changed, 7712 insertions(+), 5292 deletions(-)
copy airflow/{api_connexion/endpoints/health_endpoint.py =>
api/common/airflow_health.py} (72%)
create mode 100644
airflow/example_dags/example_dynamic_task_mapping_with_no_taskflow_operators.py
copy
airflow/migrations/versions/{0042_1_10_3_task_reschedule_fk_on_cascade_delete.py
=> 0125_2_6_2_add_onupdate_cascade_to_taskmap.py} (53%)
delete mode 100644
airflow/providers/google/cloud/example_dags/example_compute.py
delete mode 100644
airflow/providers/google/cloud/example_dags/example_compute_ssh.py
delete mode 100644 airflow/providers/yandex/.latest-doc-only-change.txt
delete mode 100644 airflow/providers/yandex/CHANGELOG.rst
delete mode 100644 airflow/providers/yandex/__init__.py
delete mode 100644 airflow/providers/yandex/hooks/__init__.py
delete mode 100644 airflow/providers/yandex/hooks/yandex.py
delete mode 100644 airflow/providers/yandex/hooks/yandexcloud_dataproc.py
delete mode 100644 airflow/providers/yandex/operators/__init__.py
delete mode 100644 airflow/providers/yandex/operators/yandexcloud_dataproc.py
delete mode 100644 airflow/providers/yandex/provider.yaml
create mode 100644 dev/breeze/src/airflow_breeze/utils/provider_dependencies.py
create mode 100644 dev/breeze/tests/test_provider_dependencies.py
delete mode 100644 docs/apache-airflow-providers-yandex/commits.rst
delete mode 100644
docs/apache-airflow-providers-yandex/connections/yandexcloud.rst
delete mode 100644 docs/apache-airflow-providers-yandex/index.rst
delete mode 100644
docs/apache-airflow-providers-yandex/installing-providers-from-sources.rst
delete mode 100644 docs/apache-airflow-providers-yandex/operators.rst
delete mode 100644 docs/integration-logos/yandex/Yandex-Cloud.png
create mode 100644 images/breeze/output_down.svg
create mode 100644
images/breeze/output_release-management_install-provider-packages.svg
delete mode 100644 images/breeze/output_stop.svg
create mode 100644 licenses/LICENSE-unicodecsv.txt
create mode 100644 scripts/ci/docker-compose/integration-kafka.yml
copy airflow/api_connexion/endpoints/request_dict.py =>
scripts/ci/docker-compose/kafka/update_run.sh (61%)
mode change 100644 => 100755
copy scripts/ci/pre_commit/{pre_commit_docstring_param_type.py =>
pre_commit_check_aiobotocore_optional.py} (54%)
copy scripts/ci/pre_commit/{pre_commit_helm_lint.py =>
pre_commit_unittest_testcase.py} (51%)
copy {airflow/api_connexion => tests/api}/__init__.py (100%)
copy {airflow/api_connexion => tests/api/common}/__init__.py (100%)
create mode 100644 tests/api/common/test_airflow_health.py
copy
tests/system/providers/google/cloud/dataflow/{example_dataflow_native_python_async.py
=> example_dataflow_go.py} (50%)
create mode 100644
tests/system/providers/google/cloud/dataflow/resources/wordcount.go
rename
tests/system/providers/google/cloud/dataproc/{example_dataproc_update.py =>
example_dataproc_cluster_update.py} (98%)
copy tests/system/providers/google/cloud/{bigquery =>
gcs}/resources/us-states.csv (100%)
copy
tests/system/providers/google/cloud/vertex_ai/resources/{California-housing.zip
=> California-housing-ai-model.zip} (100%)
copy
tests/system/providers/google/cloud/vertex_ai/resources/{California-housing.zip
=> California-housing-custom-container.zip} (100%)
copy
tests/system/providers/google/cloud/vertex_ai/resources/{California-housing.zip
=> California-housing-custom-job.zip} (100%)
rename
tests/system/providers/google/cloud/vertex_ai/resources/{California-housing.zip
=> California-housing-python-package.zip} (100%)
