This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a change to branch ci-upgrade-main
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit 5bc0b907679 [main] CI: Upgrade important CI environment
add 3647e3c7e17 i18n(ko): add missing translaction in common.json (Nov 11)
(#76920)
add b4d1f1c22be Support durable execution in SageMakerTrainingOperator
(#76036)
add 54970db82f1 Reject reserved loop keys when deleting an XCom (#76892)
add b2d58334144 Regenerate skill-eval hash after AGENTS.md link update
(#76960)
add 426357062fb Prevent Databricks task failure after successful repair
(#76904)
add 9a02819b4e4 Fix AwsEksExecutor dying with a 401 after its credentials
rotate (#76332)
add 2448be40082 Make common.ai tool_prefix and output_type signatures
consistent (#74695)
add c43ff0fc1dd Make durable flag reach default_args in
PowerBIDatasetRefreshOperator (#76793)
add 509124275f4 Clamp negative offset in mapped XCom slice queries (#75015)
add 020428773e7 Remove duplicate GroupCommandParser type from airflowctl
CLI (#76345)
add 30f51ff2d4b Apply the configured XCom backend to language SDK task
XComs (#75184)
add e024d4c15a5 Fix TimeSensor test failing when host time zone is ahead
of UTC (#76901)
add 29331a83256 Add Innogando to INTHEWILD.md (#76906)
add 992321d1cd2 Fix durable agent retries replaying the same failing tool
call on every attempt (#76862)
add 36230cf39e9 Reduce task state store calls made by durable agent tasks
(#76863)
add 02fb985e3c1 Let durable agent tasks run chosen tools on every attempt
instead of replaying them (#76864)
add 0a65e225207 Let durable agent tasks use toolsets without an id (#76877)
add e17cecaabbb Block merging on fork CI once a PR is moved to project CI
(#76672)
add 1eb0dcc87c1 Bump the core-ui-package-updates group across 1 directory
with 5 updates (#76945)
add 2becf0f4aa0 Bump the oxc group across 1 directory with 2 updates
(#76941)
add 38c11037bc1 Take the Postgres migration lock without a bound parameter
(#76976)
add 6c2e7b36460 Fail the attempt when `SQLToolset` cannot reach its
database (#76783)
add e99f9af45a8 Add iter_paginated_run to KiotaRequestAdapterHook (#75649)
add f9dca510f9f Fix global env vars shadowing team secrets in tasks and
triggers (#75998)
add 7dfc57945e8 Let plugin tabs show their icon beside the label (#76842)
add ed918278f2f Use READ COMMITTED for async MySQL metadata sessions
(#76694)
add a67d43f2baf Fix SSHHookAsync deadlock on the extra and compress like
SSHHook (#76528)
add 1c870b3478d Fix XComs of task-end trigger events being JSON-encoded
twice (#76596)
add c6a4acf8cde Fix multi-line secrets leaking in logs formatted with
pprint (#75651)
add c03cb844cb6 Update providers metadata 2026-11-12 (#76988)
add 0bc9e82ed25 Delete the CloudWatch log group left by
example_mwaa_serverless (#76810)
add 5821e0b67bc [main] CI: Upgrade important CI environment
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (5bc0b907679)
\
N -- N -- N refs/heads/ci-upgrade-main (5821e0b67bc)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/additional-prod-image-tests.yml | 19 +
.github/workflows/airflow-e2e-tests.yml | 28 +-
.github/workflows/ci-amd.yml | 2 +
.github/workflows/ci-arm.yml | 2 +
.../scheduled-legacy-prod-image-canary.yml | 2 +
INTHEWILD.md | 1 +
.../docs/administration-and-deployment/plugins.rst | 12 +
.../language-sdks/index.rst | 8 +-
airflow-core/docs/core-concepts/xcoms.rst | 2 +
.../security/secrets/mask-sensitive-values.rst | 11 +
.../src/airflow/api_fastapi/common}/xcom.py | 27 +-
.../api_fastapi/core_api/datamodels/plugins.py | 3 +
.../core_api/openapi/v2-rest-api-generated.yaml | 12 +
.../api_fastapi/core_api/routes/public/xcom.py | 13 +-
.../execution_api/routes/task_instances.py | 8 +-
.../api_fastapi/execution_api/routes/xcoms.py | 17 +-
.../src/airflow/jobs/triggerer_job_runner.py | 3 +-
airflow-core/src/airflow/models/trigger.py | 10 +-
airflow-core/src/airflow/settings.py | 24 +-
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 24 +
.../airflow/ui/openapi-gen/requests/types.gen.ts | 2 +
airflow-core/src/airflow/ui/package.json | 14 +-
airflow-core/src/airflow/ui/pnpm-lock.yaml | 544 ++++++++-------
.../airflow/ui/public/i18n/locales/ko/common.json | 2 +
.../airflow/ui/src/hooks/usePluginTabs.test.tsx | 39 +-
.../src/airflow/ui/src/hooks/usePluginTabs.tsx | 3 +
.../ui/src/layouts/Details/NavTabs.test.tsx | 32 +-
.../src/airflow/ui/src/layouts/Details/NavTabs.tsx | 43 +-
airflow-core/src/airflow/utils/db.py | 16 +-
.../core_api/routes/public/test_plugins.py | 3 +
.../execution_api/versions/head/test_xcoms.py | 25 +
airflow-core/tests/unit/core/test_settings.py | 12 +
airflow-core/tests/unit/jobs/test_triggerer_job.py | 15 +
airflow-core/tests/unit/models/test_trigger.py | 9 +-
airflow-core/tests/unit/utils/test_db.py | 22 +-
.../src/airflowctl/api/datamodels/generated.py | 2 +
airflow-ctl/src/airflowctl/ctl/cli_config.py | 25 +-
airflow-ctl/src/airflowctl/ctl/cli_parser.py | 18 +-
...ml-dag.yml => lang-sdk-xcom-object-storage.yml} | 24 +-
.../tests/airflow_e2e_tests/conftest.py | 178 +++--
.../tests/airflow_e2e_tests/constants.py | 5 +
.../__init__.py | 0
.../test_lang_sdk_xcom_backend.py | 291 ++++++++
contributing-docs/25_maintainer_pr_triage.md | 8 +-
contributing-docs/30_new_language_sdk.rst | 5 +
dev/breeze/doc/ci/04_selective_checks.md | 11 +
.../doc/ci/08_fork_ci_for_external_contributors.md | 73 +-
.../images/output_testing_airflow-e2e-tests.svg | 32 +-
.../images/output_testing_airflow-e2e-tests.txt | 2 +-
.../airflow_breeze/commands/testing_commands.py | 1 +
.../src/airflow_breeze/utils/fork_ci_messages.py | 3 +
.../src/airflow_breeze/utils/fork_ci_reconcile.py | 363 +++++++++-
.../src/airflow_breeze/utils/selective_checks.py | 17 +
.../src/airflow_breeze/utils/verification_plan.py | 1 +
dev/breeze/tests/test_fork_ci_reconcile.py | 763 +++++++++++++++++++++
dev/breeze/tests/test_selective_checks.py | 64 ++
dev/skill-evals/last-eval-hash.txt | 2 +-
generated/provider_metadata.json | 4 +
go-sdk/README.md | 1 +
go-sdk/capabilities.yaml | 5 +
go-sdk/pkg/execution/genmodels/models.gen.go | 9 +-
go-sdk/schema/supervisor-schema.json | 3 +-
java-sdk/README.md | 3 +-
java-sdk/capabilities.yaml | 9 +-
java-sdk/gradle.properties | 2 +-
java-sdk/sdk/module.md | 3 +-
java-sdk/sdk/schema/schema.json | 168 ++++-
providers/amazon/docs/changelog.rst | 9 +
providers/amazon/docs/executors/eks-executor.rst | 5 +
.../amazon/aws/executors/eks/_client_factory.py | 94 ++-
.../providers/amazon/aws/hooks/sagemaker.py | 36 +-
.../providers/amazon/aws/operators/sagemaker.py | 188 +++--
.../system/amazon/aws/example_mwaa_serverless.py | 19 +-
.../aws/executors/eks/test__client_factory.py | 232 ++++++-
.../aws/operators/test_sagemaker_training.py | 228 +++++-
providers/common/ai/README.rst | 12 +-
providers/common/ai/docs/capabilities.rst | 8 +-
providers/common/ai/docs/changelog.rst | 36 +
providers/common/ai/docs/code_mode.rst | 3 +
providers/common/ai/docs/concepts.rst | 2 +-
providers/common/ai/docs/decision_models.rst | 7 +-
providers/common/ai/docs/durable_execution.rst | 182 ++++-
providers/common/ai/docs/features.rst | 3 +-
.../common/ai/docs/frameworks/openai_agents.rst | 3 +
providers/common/ai/docs/index.rst | 12 +-
providers/common/ai/docs/operators/agent.rst | 22 +-
providers/common/ai/docs/operators/llm.rst | 1 +
.../common/ai/docs/operators/llm_file_analysis.rst | 13 +-
providers/common/ai/docs/stability.rst | 11 +-
providers/common/ai/docs/structured_output.rst | 48 +-
providers/common/ai/docs/toolsets/datafusion.rst | 3 +
providers/common/ai/docs/toolsets/index.rst | 11 +-
providers/common/ai/docs/toolsets/mcp.rst | 4 +-
providers/common/ai/docs/toolsets/sql.rst | 49 +-
providers/common/ai/docs/troubleshooting.rst | 32 +-
providers/common/ai/pyproject.toml | 21 +-
.../airflow/providers/common/ai/decorators/llm.py | 3 +-
.../airflow/providers/common/ai/durable/base.py | 3 +
.../providers/common/ai/durable/capability.py | 211 +++++-
.../providers/common/ai/durable/fingerprint.py | 25 +-
.../airflow/providers/common/ai/durable/journal.py | 297 ++++++--
.../providers/common/ai/durable/openai_agents.py | 11 +-
.../ai/example_dags/example_agent_durable.py | 75 +-
.../src/airflow/providers/common/ai/exceptions.py | 23 +
.../src/airflow/providers/common/ai/hooks/mcp.py | 4 +-
.../airflow/providers/common/ai/operators/agent.py | 172 +++--
.../airflow/providers/common/ai/operators/llm.py | 37 +-
.../common/ai/operators/llm_file_analysis.py | 10 +-
.../providers/common/ai/toolsets/datafusion.py | 33 +-
.../airflow/providers/common/ai/toolsets/hook.py | 6 +-
.../airflow/providers/common/ai/toolsets/mcp.py | 6 +-
.../providers/common/ai/toolsets/object_storage.py | 17 +-
.../providers/common/ai/toolsets/sandbox.py | 24 +-
.../airflow/providers/common/ai/toolsets/sql.py | 134 +++-
.../providers/common/ai/utils/output_type.py | 81 ++-
.../providers/common/ai/utils/query_results.py | 20 +-
.../providers/common/ai/utils/tool_prefix.py | 56 ++
.../tests/unit/common/ai/durable/memory_storage.py | 24 +-
.../unit/common/ai/durable/test_capability.py | 575 +++++++++++++++-
.../unit/common/ai/durable/test_fingerprint.py | 18 +-
.../tests/unit/common/ai/durable/test_journal.py | 458 ++++++++++++-
.../ai/tests/unit/common/ai/hooks/test_mcp.py | 18 +
.../tests/unit/common/ai/operators/test_agent.py | 226 +++++-
.../ai/tests/unit/common/ai/operators/test_llm.py | 78 ++-
.../common/ai/operators/test_llm_file_analysis.py | 28 +
.../ai/tests/unit/common/ai/sqlite_connection.py | 85 +++
.../unit/common/ai/tools/test__from_toolset.py | 18 +-
.../unit/common/ai/toolsets/test_datafusion.py | 90 +++
.../ai/tests/unit/common/ai/toolsets/test_mcp.py | 9 +-
.../unit/common/ai/toolsets/test_object_storage.py | 13 +-
.../tests/unit/common/ai/toolsets/test_sandbox.py | 3 +
.../ai/tests/unit/common/ai/toolsets/test_sql.py | 304 +++++++-
.../tests/unit/common/ai/utils/test_output_type.py | 343 ++++++++-
.../unit/common/ai/utils/test_query_results.py | 27 +
.../tests/unit/common/ai/utils/test_tool_prefix.py | 78 +++
.../providers/databricks/operators/databricks.py | 1 +
.../unit/databricks/operators/test_databricks.py | 16 +
.../edge3/example_dags/integration_test.py | 2 +-
.../providers/edge3/example_dags/win_test.py | 2 +-
.../providers/microsoft/azure/hooks/msgraph.py | 120 ++--
.../providers/microsoft/azure/operators/powerbi.py | 5 +
.../unit/microsoft/azure/hooks/test_msgraph.py | 148 ++++
.../unit/microsoft/azure/operators/test_powerbi.py | 34 +-
providers/ssh/README.rst | 2 +-
providers/ssh/docs/connections/ssh.rst | 1 +
providers/ssh/docs/index.rst | 2 +-
providers/ssh/docs/operators/ssh_remote_job.rst | 5 +
providers/ssh/pyproject.toml | 4 +-
.../ssh/src/airflow/providers/ssh/hooks/ssh.py | 22 +-
.../ssh/tests/unit/ssh/hooks/test_ssh_async.py | 117 ++--
.../tests/unit/standard/sensors/test_time.py | 2 +-
scripts/ci/prek/lang_sdk_compat_matrix.py | 1 +
.../plugins_manager/plugins_manager.py | 1 +
.../secrets_masker/secrets_masker.py | 6 +
.../tests/secrets_masker/test_secrets_masker.py | 51 ++
task-sdk/src/airflow/sdk/bases/xcom.py | 50 +-
.../src/airflow/sdk/coordinators/_subprocess.py | 351 +++++++++-
.../src/airflow/sdk/coordinators/_xcom_backend.py | 357 ++++++++++
task-sdk/src/airflow/sdk/exceptions.py | 1 +
task-sdk/src/airflow/sdk/execution_time/comms.py | 19 +
task-sdk/src/airflow/sdk/execution_time/context.py | 80 ++-
.../airflow/sdk/execution_time/schema/schema.json | 3 +-
.../sdk/execution_time/schema/versions/__init__.py | 2 +
.../execution_time/schema/versions/v2027_02_28.py | 27 +-
.../src/airflow/sdk/execution_time/supervisor.py | 26 +-
.../src/airflow/sdk/execution_time/task_runner.py | 29 +-
task-sdk/tests/task_sdk/bases/test_xcom.py | 17 +-
.../tests/task_sdk/coordinators/test_subprocess.py | 710 ++++++++++++++++++-
.../task_sdk/coordinators/test_xcom_backend.py | 464 +++++++++++++
.../tests/task_sdk/definitions/test_connection.py | 2 +-
.../tests/task_sdk/definitions/test_variables.py | 2 +-
.../execution_time/schema/test_migrator.py | 26 +
.../tests/task_sdk/execution_time/test_comms.py | 13 +
.../tests/task_sdk/execution_time/test_context.py | 58 ++
.../task_sdk/execution_time/test_context_cache.py | 16 +-
.../task_sdk/execution_time/test_supervisor.py | 8 +-
.../task_sdk/execution_time/test_task_runner.py | 43 +-
ts-sdk/README.md | 1 +
ts-sdk/capabilities.yaml | 5 +
ts-sdk/schema/supervisor-schema.json | 3 +-
ts-sdk/src/generated/supervisor.ts | 6 +-
uv.lock | 542 +++++++--------
182 files changed, 9915 insertions(+), 1529 deletions(-)
copy {task-sdk/src/airflow/sdk/execution_time =>
airflow-core/src/airflow/api_fastapi/common}/xcom.py (59%)
copy airflow-e2e-tests/docker/{yaml-dag.yml =>
lang-sdk-xcom-object-storage.yml} (52%)
copy {airflow-core/src/airflow/_shared =>
airflow-e2e-tests/tests/airflow_e2e_tests/lang_sdk_xcom_object_storage_tests}/__init__.py
(100%)
create mode 100644
airflow-e2e-tests/tests/airflow_e2e_tests/lang_sdk_xcom_object_storage_tests/test_lang_sdk_xcom_backend.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/utils/tool_prefix.py
create mode 100644
providers/common/ai/tests/unit/common/ai/sqlite_connection.py
create mode 100644
providers/common/ai/tests/unit/common/ai/utils/test_tool_prefix.py
create mode 100644 task-sdk/src/airflow/sdk/coordinators/_xcom_backend.py
create mode 100644 task-sdk/tests/task_sdk/coordinators/test_xcom_backend.py
