This is an automated email from the ASF dual-hosted git repository.
ephraimbuddy pushed a change to branch dag-processor-activation
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit 74ccd2d3ded Preserve Dag processor work across retries and reparses
omit 571a018f65e Run Dag processors without metadata database access
omit 2c27096f6d7 Publish Dag processor results through the Execution API
omit 4d725d18c81 Ban new httpx imports in airflow-core
omit 203d3f4b394 Parse Execution API JSON errors that carry media type
parameters
omit 104a65d412a Align Dag processor claim migrations with main
omit 2653b51131c Test legacy Dag deactivation after bundle-name backfill
omit ca3b8731e97 Fix Dag processor work retries and metadata cleanup
omit 6135503fc3e Retain Dag processor work until delivery is acknowledged
omit 354bcfb8ebd Align API-mode Dag publication with DB-mode cleanup
omit 50720c5b704 Allow Dag processors to publish metadata through the
Execution API
omit 1f4ec6ab009 Make the bundle refresh regression test independent of
uptime
omit 5bd97cc8124 Avoid parsing stalls while a bundle refresh waits for
imports
omit 326bc78321e Keep Dag parsing results aligned with refreshed bundle
sources
omit 1bd0a764d34 Distinguish invalid Dag access control from FAB backend
failures (#74788)
omit 359d361de93 Base CI and PROD images on hardened Python images (#73038)
omit 12df3270b84 Clean up unused mypy suppressions in standard provider
tests (#75210)
omit b0cfa1e1b1b Stop the EKS executor concurrency test from starving its
own builders (#75197)
omit 1d63c9cbedb Exclude minimal and ECS Amazon Linux AMIs in
get_latest_ami_id (#75171)
omit b7b69d70389 Skip or fail ECS tasks correctly when their CloudWatch log
stream is missing (#74730)
omit 31bb7edc69c Add emit_message_history to record an agent transcript
without seeding it (#74811)
omit 347dd398d62 Require uv 0.13.1 (#75201)
omit 249e5cbcbc1 Fix Modal contract test importing a private class from its
old location (#75188)
omit eaefdf9c160 Fix duplicate pyvespa block in uv.lock (#75200)
omit 1c4af038874 Java SDK: Reject IDs that collide with task group join
nodes (#74781)
omit 7126d336e94 Log the retry reason in airflowctl API client retries
(#75059)
add 403d678b979 Distinguish invalid Dag access control from FAB backend
failures
add c4f0c55d037 Keep Dag parsing results aligned with refreshed bundle
sources
add 39c95d7c3f9 Avoid parsing stalls while a bundle refresh waits for
imports
add cfb451e00ed Make the bundle refresh regression test independent of
uptime
add e86e383ab4b Allow Dag processors to publish metadata through the
Execution API
add 021b1e60cb0 Align API-mode Dag publication with DB-mode cleanup
add 7e3fca18111 Retain Dag processor work until delivery is acknowledged
add 14473174ecb Fix Dag processor work retries and metadata cleanup
add 72a3d21b5a5 Test legacy Dag deactivation after bundle-name backfill
add 6a8a0ee06d6 Align Dag processor claim migrations with main
add 4d61fc0f38b Parse Execution API JSON errors that carry media type
parameters
add b5c085ffcd0 Ban new httpx imports in airflow-core
add dbb8f29fad5 Publish Dag processor results through the Execution API
add f8222be2452 Back off inventory retries and report results the API
would refuse
add 29907838141 Run Dag processors without metadata database access
add 573c296f06a Preserve Dag processor work across retries and reparses
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (74ccd2d3ded)
\
N -- N -- N refs/heads/dag-processor-activation (573c296f06a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../actions/build_ci_image_with_cache/action.yml | 10 +-
.github/workflows/ci-image-build.yml | 19 +-
.github/workflows/mirror-hardened-base-images.yml | 80 ----
.github/workflows/prod-image-build.yml | 9 +-
.github/workflows/push-image-cache.yml | 4 +-
.../scheduled-legacy-prod-image-canary.yml | 262 ------------
Dockerfile | 443 +++++----------------
Dockerfile.ci | 435 +++++---------------
airflow-core/newsfragments/73038.significant.rst | 31 --
airflow-core/pyproject.toml | 2 +-
.../src/airflow/dag_processing/bundles/base.py | 5 +-
.../src/airflow/dag_processing/bundles/manager.py | 32 +-
airflow-core/src/airflow/dag_processing/manager.py | 194 ++++++---
.../execution_api/test_dag_processor_client.py | 8 +-
.../tests/unit/dag_processing/bundles/test_base.py | 2 +-
.../bundles/test_dag_bundle_manager.py | 21 -
.../tests/unit/dag_processing/test_manager.py | 112 +++++-
airflow-ctl/pyproject.toml | 2 +-
airflow-ctl/src/airflowctl/api/client.py | 4 +-
airflow-ctl/tests/airflow_ctl/api/test_client.py | 31 --
airflow-e2e-tests/docker/Dockerfile.java | 23 +-
dev/breeze/doc/09_release_management_tasks.rst | 28 --
dev/breeze/doc/ci/02_images.md | 20 +-
dev/breeze/doc/images/output_ci-image_build.svg | 94 +++--
dev/breeze/doc/images/output_ci-image_build.txt | 2 +-
dev/breeze/doc/images/output_prod-image_build.svg | 164 ++++----
dev/breeze/doc/images/output_prod-image_build.txt | 2 +-
.../doc/images/output_release-management.svg | 38 +-
.../doc/images/output_release-management.txt | 2 +-
...utput_release-management_mirror-base-images.svg | 116 ------
...utput_release-management_mirror-base-images.txt | 1 -
.../output_setup_check-all-params-in-groups.svg | 20 +-
.../output_setup_check-all-params-in-groups.txt | 2 +-
.../output_setup_regenerate-command-images.svg | 2 +-
.../output_setup_regenerate-command-images.txt | 2 +-
.../airflow_breeze/commands/ci_image_commands.py | 4 +
.../commands/ci_image_commands_config.py | 1 +
.../commands/common_image_options.py | 10 -
.../airflow_breeze/commands/kubernetes_commands.py | 12 +-
.../commands/production_image_commands.py | 4 -
.../commands/production_image_commands_config.py | 1 -
.../commands/release_management_commands.py | 92 +----
.../commands/release_management_commands_config.py | 9 -
dev/breeze/src/airflow_breeze/global_constants.py | 32 +-
.../src/airflow_breeze/params/build_prod_params.py | 11 -
.../airflow_breeze/params/common_build_params.py | 6 +-
.../airflow_breeze/utils/check_release_files.py | 2 +-
dev/breeze/tests/test_build_prod_params.py | 82 ----
dev/breeze/tests/test_environment_check.py | 2 +-
.../tests/test_kubernetes_lang_sdk_commands.py | 45 ---
dev/breeze/tests/test_mirror_base_images.py | 98 -----
dev/mypy/pyproject.toml | 2 +-
docker-stack-docs/build-arg-ref.rst | 30 +-
docker-stack-docs/build.rst | 128 +-----
docker-stack-docs/changelog.rst | 43 --
.../customizing/add-build-essential-custom.sh | 2 +-
.../docker-examples/customizing/custom-sources.sh | 2 +-
.../customizing/pypi-dev-runtime-deps.sh | 2 +-
.../customizing/pypi-extras-and-deps.sh | 2 +-
.../customizing/pypi-selected-version.sh | 2 +-
.../extending/add-providers/Dockerfile | 4 +-
.../restricted/restricted_environments.sh | 2 +-
docs/spelling_wordlist.txt | 3 -
.../org/apache/airflow/sdk/AirflowProcessor.kt | 15 +-
.../org/apache/airflow/sdk/AirflowProcessorTest.kt | 123 ------
.../main/kotlin/org/apache/airflow/sdk/DagDef.kt | 38 +-
.../kotlin/org/apache/airflow/sdk/TaskGroupRef.kt | 13 +-
.../kotlin/org/apache/airflow/sdk/internal/Ids.kt | 13 -
.../kotlin/org/apache/airflow/sdk/TaskGroupTest.kt | 98 -----
kubernetes-tests/lang_sdk/Dockerfile.runtimes | 23 +-
.../providers/amazon/aws/utils/task_log_fetcher.py | 24 +-
.../amazon/tests/system/amazon/aws/utils/ec2.py | 29 +-
.../aws/executors/eks/test__client_factory.py | 3 -
.../tests/unit/amazon/aws/operators/test_ecs.py | 22 -
.../unit/amazon/aws/utils/test_task_log_fetcher.py | 8 -
providers/common/ai/docs/message_history.rst | 10 +-
providers/common/ai/docs/operators/agent.rst | 3 -
providers/common/ai/docs/stability.rst | 4 -
providers/common/ai/docs/troubleshooting.rst | 2 +-
.../airflow/providers/common/ai/operators/agent.py | 14 +-
.../tests/unit/common/ai/operators/test_agent.py | 45 +--
.../ai/tests/unit/common/ai/sandbox/test_modal.py | 3 +-
.../tests/unit/standard/decorators/test_bash.py | 2 +-
.../tests/unit/standard/decorators/test_python.py | 12 +-
.../unit/standard/decorators/test_short_circuit.py | 2 +-
.../unit/standard/operators/test_latest_only.py | 2 +-
.../unit/standard/sensors/test_external_task.py | 12 +-
.../tests/unit/standard/sensors/test_time_delta.py | 4 +-
.../tests/unit/standard/sensors/test_weekday.py | 2 +-
pyproject.toml | 2 +-
scripts/ci/prek/update_docker_gpg_keys.py | 2 -
scripts/ci/prek/upgrade_important_versions.py | 82 ++--
scripts/docker/common.sh | 16 +-
scripts/docker/entrypoint_ci.sh | 5 +-
scripts/docker/install_mssql.sh | 4 +-
scripts/docker/install_mysql.sh | 17 +-
scripts/docker/install_os_dependencies.sh | 353 ++++------------
scripts/docker/install_postgres.sh | 2 +-
scripts/docker/keys/microsoft-2025.asc | 29 --
scripts/in_container/run_ci_tests.sh | 11 +-
.../ci/prek/test_upgrade_important_versions.py | 62 ---
uv.lock | 21 +-
102 files changed, 921 insertions(+), 3066 deletions(-)
delete mode 100644 .github/workflows/mirror-hardened-base-images.yml
delete mode 100644 .github/workflows/scheduled-legacy-prod-image-canary.yml
delete mode 100644 airflow-core/newsfragments/73038.significant.rst
delete mode 100644
dev/breeze/doc/images/output_release-management_mirror-base-images.svg
delete mode 100644
dev/breeze/doc/images/output_release-management_mirror-base-images.txt
delete mode 100644 dev/breeze/tests/test_build_prod_params.py
delete mode 100644 dev/breeze/tests/test_mirror_base_images.py
delete mode 100644 scripts/docker/keys/microsoft-2025.asc
