This is an automated email from the ASF dual-hosted git repository.
ashb pushed a change to branch task-loops-stack-13
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard b1225b2917e Show loop iterations and outcomes throughout the UI
discard a6839047682 Show current task executions in the UI
discard b8944de444c Let the HITL review plugin address one loop pass or mapped
slot
discard 32bb9daf949 Read current task executions by exact region and try
discard 6f38bb74665 Clear selected loop work and regenerate only what was
replaced
discard cc4f2a3fac2 Run loop gates and advance iterations atomically
discard 0e415aeb7f6 Attribute log, asset-event and asset-state writes to the
task attempt
discard 63cb1549606 Give every mapped expansion a region and store its index
as region_index
discard 7774d1b4daf Pin loop definitions and carry region coordinates through
task execution
discard 8d326bee56f Resolve live producers by region and read their XCom by
attempt
discard 049690db847 Introduce the (internal/db level) concept of "dynamic
regions" for TIs
discard c89e71507fa Add user-facing docs for the new Task Loops feature.
add 96582adf66f Respect run on latest version when clearing a running Dag
run (#71425)
add ccd96d21597 Regenerate the stale TS SDK supervisor types (#74412)
add 11cbf4da098 Allow a custom Kubernetes client factory for
KubernetesExecutor (#73014)
add 57183056ca7 Fix AsyncToSync error in Databricks deferrable operators
on Airflow 3.0 (#74406)
add 0b4039e250c Upgrade UI major dependencies: msw, vitest, react-table
and mermaid (#74369)
add 8b115c0be65 Verify SSH host keys by default in SSH and SFTP hooks
(#73419)
add 77e0906dc31 Scope plugin views on any entity field, not a fixed
criteria set (#74278)
add 4da2ad1e2b3 Document HTTP statuses that route helpers raise (#74027)
add 05dec276c72 Let the task endpoint describe a task at a given Dag
version (#74365)
add 19d20a091dc UI: Add 'Mark as Skipped' option to Task Instance actions
(#72425)
add d38e9ceb749 Fix date range filter inputs committing on every keystroke
(#74011)
add 37032a47d49 Bump the fab-ui-package-updates group across 1 directory
with 2 updates (#74418)
add 37dad4c1df2 Fix codespell failure on main in DateRangeFilter test
(#74430)
add fe4481cc8e5 Fix Databricks async connection test on Airflow before 3.1
(#74427)
add 8043c703de1 Java SDK: Group a native Dag's tasks with task groups
(#74230)
add 1921597eeff Go SDK: embed each native Dag's source file in the bundle
(#74342)
add 01b977568e2 Java SDK: Serialize native Dags to DagSerialization v3
(#71190)
add 46d3c59dfec Parse native Go Dags and show each Dag's own source
(#74341)
add ddb905f0ee0 Fix airflowctl dags get-tags crashing whenever a Dag has
tags (#73743)
add 9c9346c8065 Reconnect SMTP hooks after leaving a context manager
(#74416)
add 08205150190 i18n(ko): Align particles after Dag with the agreed
pronunciation (#73799)
add 22afa85809d Remove most accordions from forms (#74422)
add 950fc37bf6c Fix Data Fusion polling after transient 404 responses
(#74237)
add 9068fba9cb7 Fix remote processor injection happening before dictConfig
runs in configure_logging (#66633)
add 599fcb41ac0 Run the XCom column prek hook on Python 3.11 or newer
(#74405)
add a0e9d68b98a Fix common.ai managed agent failover when an identity
cannot resolve (#72786)
add bf3d4022db3 Remove slow-on-MySQL and now needless CHECK CONSTRAINT
(#74409)
add 57ad5d30539 Prepare providers release 2026-10-06 (#74429)
add 3f552272f9f Sync the TS SDK supervisor schema from its Task SDK source
(#74415)
add ef9feefd83f Fix Grid and Graph 500 errors for cyclic TaskGroup
dependencies (#73724)
add 3384f87b55d Add user-facing docs for the new Task Loops feature.
add 2e454937336 fixup! Add user-facing docs for the new Task Loops feature.
add 8dd908bbc3c Introduce the (internal/db level) concept of "dynamic
regions" for TIs
add 1b3d613d21b Resolve live producers by region and read their XCom by
attempt
add 07542b14316 Pin loop definitions and carry region coordinates through
task execution
add ea42c23144f Give every mapped expansion a region and store its index
as region_index
add 6c6f184c988 fixup! Give every mapped expansion a region and store its
index as region_index
add c3b6ed75865 Attribute log, asset-event and asset-state writes to the
task attempt
add 08d8d0cc299 Run loop gates and advance iterations atomically
add 094803ea908 Clear selected loop work and regenerate only what was
replaced
add f34b417982b Read current task executions by exact region and try
add db74ddfa0af Let the HITL review plugin address one loop pass or mapped
slot
add 14f600a122e Show current task executions in the UI
add b2d1aecc82a Show loop iterations and outcomes throughout the UI
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b1225b2917e)
\
N -- N -- N refs/heads/task-loops-stack-13 (b2d1aecc82a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.agents/skills/airflow-translations/locales/ko.md | 18 +
.pre-commit-config.yaml | 35 +-
.../0007-taskflow-across-language-boundary.md | 5 +
.../0015-per-dag-source-in-bundle-artifact.md | 13 +-
.../docs/administration-and-deployment/plugins.rst | 172 ++++--
.../authoring-and-scheduling/language-sdks/go.rst | 64 ++-
.../language-sdks/java.rst | 76 ++-
.../docs/authoring-and-scheduling/loops.rst | 32 +-
.../core-concepts/task-and-asset-state-store.rst | 2 +-
airflow-core/docs/core-concepts/tasks.rst | 23 +-
airflow-core/newsfragments/66633.bugfix.rst | 1 +
airflow-core/newsfragments/69148.feature.rst | 1 -
airflow-core/newsfragments/74278.feature.rst | 1 +
.../api_fastapi/core_api/datamodels/plugins.py | 31 +-
.../api_fastapi/core_api/openapi/_private_ui.yaml | 6 +
.../core_api/openapi/v2-rest-api-generated.yaml | 80 +--
.../api_fastapi/core_api/routes/public/tasks.py | 29 +-
.../api_fastapi/core_api/routes/ui/calendar.py | 4 +-
.../api_fastapi/execution_api/routes/hitl.py | 6 +-
.../execution_api/routes/task_instances.py | 10 +-
.../api_fastapi/execution_api/routes/xcoms.py | 30 +-
..._3_4_0_fold_task_map_into_xcom_mapped_length.py | 2 -
airflow-core/src/airflow/models/dagrun.py | 6 +-
airflow-core/src/airflow/models/taskinstance.py | 107 ++--
airflow-core/src/airflow/models/xcom.py | 2 -
airflow-core/src/airflow/plugins_manager.py | 216 +++++++-
.../airflow/serialization/definitions/taskgroup.py | 42 +-
.../src/airflow/ui/openapi-gen/queries/common.ts | 5 +-
.../ui/openapi-gen/queries/ensureQueryData.ts | 10 +-
.../src/airflow/ui/openapi-gen/queries/prefetch.ts | 10 +-
.../src/airflow/ui/openapi-gen/queries/queries.ts | 10 +-
.../src/airflow/ui/openapi-gen/queries/suspense.ts | 10 +-
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 84 +--
.../ui/openapi-gen/requests/services.gen.ts | 9 +
.../airflow/ui/openapi-gen/requests/types.gen.ts | 20 +-
airflow-core/src/airflow/ui/package.json | 10 +-
airflow-core/src/airflow/ui/pnpm-lock.yaml | 599 ++++++++++-----------
.../ui/public/i18n/locales/ar/components.json | 1 -
.../ui/public/i18n/locales/ca/components.json | 1 -
.../ui/public/i18n/locales/de/components.json | 1 -
.../ui/public/i18n/locales/el/components.json | 1 -
.../ui/public/i18n/locales/en/components.json | 1 -
.../airflow/ui/public/i18n/locales/en/dags.json | 1 +
.../ui/public/i18n/locales/es/components.json | 1 -
.../ui/public/i18n/locales/fr/components.json | 1 -
.../ui/public/i18n/locales/he/components.json | 1 -
.../ui/public/i18n/locales/hi/components.json | 1 -
.../ui/public/i18n/locales/hu/components.json | 1 -
.../ui/public/i18n/locales/it/components.json | 1 -
.../ui/public/i18n/locales/ja/components.json | 1 -
.../airflow/ui/public/i18n/locales/ko/assets.json | 4 +-
.../ui/public/i18n/locales/ko/components.json | 5 +-
.../src/airflow/ui/public/i18n/locales/ko/dag.json | 2 +-
.../airflow/ui/public/i18n/locales/ko/dags.json | 4 +-
.../ui/public/i18n/locales/nl/components.json | 1 -
.../ui/public/i18n/locales/pl/components.json | 1 -
.../ui/public/i18n/locales/pt/components.json | 1 -
.../ui/public/i18n/locales/ru/components.json | 1 -
.../ui/public/i18n/locales/th/components.json | 1 -
.../ui/public/i18n/locales/tr/components.json | 1 -
.../ui/public/i18n/locales/zh-CN/components.json | 1 -
.../ui/public/i18n/locales/zh-TW/components.json | 1 -
.../components/ActionAccordion/ActionAccordion.tsx | 140 ++---
.../ui/src/components/ActionAccordion/columns.tsx | 9 +-
.../ui/src/components/Assets/AssetEvent.test.tsx | 2 +-
.../src/components/Assets/TriggeredRuns.test.tsx | 2 +-
.../src/components/Banner/BackfillBanner.test.tsx | 2 +-
.../src/components/Banner/DrainingBanner.test.tsx | 2 +-
.../TaskInstance/ClearTaskInstanceDialog.test.tsx | 4 +-
.../src/airflow/ui/src/components/ConfigForm.tsx | 71 ++-
.../DagActions/PartitionPreviewTable.tsx | 3 +-
.../components/DagActions/RunBackfillForm.test.tsx | 2 +-
.../components/DagActions/inlineMessage.test.tsx | 2 +-
.../src/components/DagDeactivatedBanner.test.tsx | 2 +-
.../ui/src/components/DataTable/CardList.tsx | 14 +-
.../DataTable/DataTable.columnVisibility.test.tsx | 5 +-
.../DataTable/DataTable.stateChange.test.tsx | 101 ++++
.../ui/src/components/DataTable/DataTable.test.tsx | 40 +-
.../ui/src/components/DataTable/DataTable.tsx | 75 ++-
.../src/components/DataTable/FilterMenuButton.tsx | 12 +-
.../ui/src/components/DataTable/TableList.test.tsx | 11 +-
.../ui/src/components/DataTable/TableList.tsx | 18 +-
.../features.ts} | 27 +-
.../ui/src/components/DataTable/skeleton.tsx | 3 +-
.../airflow/ui/src/components/DataTable/types.ts | 18 +-
.../ui/src/components/FilterBar/FilterBar.test.tsx | 2 +-
.../src/components/FilterBar/filters/DateInput.tsx | 10 +-
.../FilterBar/filters/DateRangeFilter.test.tsx | 164 +++++-
.../FilterBar/filters/DateRangeFilter.tsx | 4 +
.../FilterBar/filters/DateRangeInputs.tsx | 8 +-
.../components/FlexibleForm/FieldObject.test.tsx | 2 +-
.../src/components/FlexibleForm/FlexibleForm.tsx | 163 ++----
.../ui/src/components/Graph/TaskLink.test.tsx | 2 +-
.../ui/src/components/ImportErrorCount.test.tsx | 2 +-
.../airflow/ui/src/components/JsonEditor.test.tsx | 2 +-
.../TaskInstance/MarkTaskInstanceAsButton.tsx | 17 +-
.../TaskInstance/MarkTaskInstanceAsDialog.test.tsx | 76 ++-
.../TaskInstance/MarkTaskInstanceAsDialog.tsx | 26 +-
.../airflow/ui/src/components/MarkAs/utils.test.ts | 46 +-
.../src/airflow/ui/src/components/MarkAs/utils.ts | 12 +-
.../ui/src/components/MarkdownModal.test.tsx | 2 +-
.../airflow/ui/src/components/NotePreview.test.tsx | 2 +-
.../src/airflow/ui/src/components/PoolBar.test.tsx | 2 +-
.../ui/src/components/PresetFiltersMenu.test.tsx | 2 +-
.../ui/src/components/RenderedJsonField.test.tsx | 2 +-
.../airflow/ui/src/components/StatsCard.test.tsx | 2 +-
.../ui/src/components/TaskInstanceTooltip.test.tsx | 2 +-
.../airflow/ui/src/components/TeamName.test.tsx | 2 +-
.../airflow/ui/src/components/TogglePause.test.tsx | 2 +-
.../TriggerDag/PausedDagOptions.test.tsx | 2 +-
.../TriggerDag/TriggerDAGAdvancedOptions.tsx | 12 +-
.../TriggerDag/TriggerDAGButton.test.tsx | 2 +-
.../components/TriggerDag/TriggerDAGForm.test.tsx | 16 +-
.../components/TriggerDag/TriggerDAGModal.test.tsx | 2 +-
.../src/components/VersionCompareSelect.test.tsx | 2 +-
.../ui/src/components/renderStructuredLog.test.tsx | 2 +-
.../src/airflow/ui/src/hooks/useDateRangeFilter.ts | 117 ++--
.../ui/src/hooks/usePluginAppliesToContext.test.ts | 12 +
.../ui/src/hooks/usePluginAppliesToContext.ts | 12 +-
.../airflow/ui/src/hooks/usePluginTabs.test.tsx | 10 +-
.../ui/src/layouts/Details/DagBreadcrumb.test.tsx | 2 +-
.../ui/src/layouts/Details/Grid/Grid.test.tsx | 2 +-
.../src/layouts/Details/Grid/GridButton.test.tsx | 2 +-
.../ui/src/layouts/Details/Grid/GridTI.test.tsx | 2 +-
.../ui/src/layouts/Details/PanelButtons.test.tsx | 2 +-
.../airflow/ui/src/layouts/Nav/DocsButton.test.tsx | 2 +-
.../ui/src/layouts/Nav/LanguageSelector.test.tsx | 2 +-
.../src/airflow/ui/src/layouts/Nav/Nav.test.tsx | 2 +-
.../airflow/ui/src/layouts/Nav/NavButton.test.tsx | 2 +-
.../ui/src/layouts/Nav/PluginMenus.test.tsx | 2 +-
.../ui/src/layouts/Nav/UserSettingsButton.test.tsx | 2 +-
.../Asset/AssetStateStore/AssetStateStore.tsx | 6 +-
.../src/pages/Asset/CreateAssetEventModal.test.tsx | 2 +-
.../src/airflow/ui/src/pages/Asset/Header.test.tsx | 2 +-
.../ui/src/pages/AssetsList/AssetsList.test.tsx | 4 +-
.../airflow/ui/src/pages/AssetsList/AssetsList.tsx | 3 +-
.../pages/AssetsList/DependencyPopover.test.tsx | 2 +-
.../src/airflow/ui/src/pages/Configs/Configs.tsx | 3 +-
.../src/pages/Connections/ConnectionForm.test.tsx | 18 +-
.../ui/src/pages/Connections/ConnectionForm.tsx | 84 ++-
.../pages/Connections/ConnectionStandardFields.tsx | 2 +-
.../ui/src/pages/Connections/Connections.tsx | 5 +-
.../pages/Connections/NothingFoundInfo.test.tsx | 2 +-
.../pages/Dag/Backfills/BackfillDagRunsModal.tsx | 3 +-
.../ui/src/pages/Dag/Backfills/Backfills.test.tsx | 2 +-
.../ui/src/pages/Dag/Backfills/Backfills.tsx | 3 +-
.../Dag/Backfills/BackfillsDateFilter.test.tsx | 2 +-
.../ui/src/pages/Dag/Code/FileLocation.test.tsx | 2 +-
.../airflow/ui/src/pages/Dag/DagHeader.test.tsx | 4 +-
.../ui/src/pages/Dag/DeadlineAlertsBadge.test.tsx | 2 +-
.../src/airflow/ui/src/pages/Dag/Header.test.tsx | 2 +-
.../ui/src/pages/Dag/Overview/Overview.test.tsx | 4 +-
.../src/airflow/ui/src/pages/Dag/Tasks/Tasks.tsx | 3 +-
.../pages/Dag/Versions/VersionDiff.i18n.test.tsx | 2 +-
.../ui/src/pages/Dag/Versions/VersionDiff.test.tsx | 2 +-
.../ui/src/pages/Dag/Versions/Versions.test.tsx | 2 +-
.../airflow/ui/src/pages/DagBundle/BundleFiles.tsx | 3 +-
.../ui/src/pages/DagBundles/DagBundles.test.tsx | 4 +-
.../airflow/ui/src/pages/DagBundles/DagBundles.tsx | 3 +-
.../src/pages/DagRuns/BulkDeleteDagRunsButton.tsx | 3 +-
.../airflow/ui/src/pages/DagRuns/DagRuns.test.tsx | 2 +-
.../src/airflow/ui/src/pages/DagRuns/DagRuns.tsx | 8 +-
.../DagsList/DagFolderTree/DagFolderTree.test.tsx | 2 +-
.../DagsList/DagsFilters/DagsFilters.test.tsx | 2 +-
.../ui/src/pages/DagsList/DagsList.test.tsx | 4 +-
.../ui/src/pages/DagsList/DagsListColumns.tsx | 3 +-
.../src/pages/DagsList/PartitionScheduleModal.tsx | 3 +-
.../pages/DagsList/RecentTaskStateCountsColumn.tsx | 4 +-
.../ui/src/pages/Dashboard/AlertContent.test.tsx | 2 +-
.../ui/src/pages/Dashboard/Dashboard.test.tsx | 2 +-
.../ui/src/pages/Dashboard/Health/Health.test.tsx | 2 +-
.../Dashboard/PoolSummary/PoolSummary.test.tsx | 2 +-
.../ui/src/pages/Dashboard/Stats/Stats.test.tsx | 2 +-
.../src/airflow/ui/src/pages/Deadlines/index.tsx | 6 +-
.../src/airflow/ui/src/pages/Events/Events.tsx | 3 +-
.../pages/HITLTaskInstances/HITLResponseForm.tsx | 1 -
.../pages/HITLTaskInstances/HITLTaskInstances.tsx | 3 +-
airflow-core/src/airflow/ui/src/pages/Jobs.tsx | 8 +-
.../src/airflow/ui/src/pages/LandingPage.test.tsx | 2 +-
.../src/airflow/ui/src/pages/Providers.tsx | 3 +-
.../src/airflow/ui/src/pages/ReactPlugin.test.tsx | 2 +-
.../src/airflow/ui/src/pages/Run/Header.test.tsx | 2 +-
.../ui/src/pages/Task/Overview/Overview.test.tsx | 4 +-
.../ui/src/pages/TaskInstance/DefaultTab.test.tsx | 2 +-
.../ui/src/pages/TaskInstance/Details.test.tsx | 2 +-
.../ui/src/pages/TaskInstance/ExtraLinks.test.tsx | 2 +-
.../ui/src/pages/TaskInstance/Header.test.tsx | 2 +-
.../TaskInstance/Logs/LogSearchInput.test.tsx | 2 +-
.../ui/src/pages/TaskInstance/Logs/Logs.test.tsx | 2 +-
.../pages/TaskInstance/Logs/LogsSelection.test.tsx | 2 +-
.../pages/TaskInstance/RenderedTemplates.test.tsx | 2 +-
.../ui/src/pages/TaskInstances/TaskInstances.tsx | 3 +-
.../ui/src/pages/TaskStateStore/TaskStateStore.tsx | 3 +-
.../airflow/ui/src/pages/Variables/Variables.tsx | 7 +-
.../src/airflow/ui/src/pages/XCom/XCom.tsx | 3 +-
.../airflow/ui/src/pages/XCom/XComEntry.test.tsx | 4 +-
.../airflow/ui/src/queries/useTogglePause.test.tsx | 2 +-
.../airflow/ui/src/utils/pluginAppliesTo.test.ts | 303 +++++++----
.../src/airflow/ui/src/utils/pluginAppliesTo.ts | 202 ++++---
.../renderMermaid.test.ts} | 31 +-
.../src/airflow/ui/src/utils/renderMermaid.ts | 10 +-
airflow-core/src/airflow/ui/testsSetup.ts | 2 +-
.../core_api/routes/public/test_plugins.py | 30 +-
.../core_api/routes/public/test_tasks.py | 35 ++
.../api_fastapi/core_api/routes/ui/test_grid.py | 26 +
.../core_api/routes/ui/test_structure.py | 22 +
.../unit/cli/commands/test_plugins_command.py | 4 +-
airflow-core/tests/unit/models/test_cleartasks.py | 239 +++++++-
airflow-core/tests/unit/plugins/test_plugin.py | 4 +-
.../tests/unit/plugins/test_plugins_manager.py | 220 +++++++-
airflow-core/tests/unit/utils/test_task_group.py | 134 +++++
.../airflowctl_tests/test_airflowctl_commands.py | 1 +
.../src/airflowctl/api/datamodels/generated.py | 20 +-
airflow-ctl/src/airflowctl/ctl/cli_config.py | 12 +-
.../tests/airflow_ctl/ctl/test_cli_config.py | 96 ++++
dev/breeze/doc/ci/04_selective_checks.md | 13 +-
.../src/airflow_breeze/utils/selective_checks.py | 22 +-
dev/breeze/tests/test_selective_checks.py | 88 ++-
generated/known_airflow_exceptions.txt | 2 +-
go-sdk/README.md | 4 +-
go-sdk/airflow/bundle.go | 53 +-
go-sdk/airflow/bundle_test.go | 27 +
go-sdk/airflow/dag.go | 11 +-
.../level.go => airflow/dag_factory_test.go} | 11 +-
go-sdk/airflow/dag_source_test.go | 65 +++
go-sdk/airflow/serve.go | 6 +-
go-sdk/cmd/airflow-go-pack/inspect.go | 59 +-
go-sdk/cmd/airflow-go-pack/inspect_test.go | 89 ++-
go-sdk/cmd/airflow-go-pack/main.go | 9 +-
go-sdk/cmd/airflow-go-pack/pack.go | 123 +++--
.../cmd/airflow-go-pack/pack_integration_test.go | 21 +-
go-sdk/cmd/airflow-go-pack/pack_test.go | 102 +++-
go-sdk/cmd/airflow-go-pack/sources.go | 256 +++++++++
go-sdk/cmd/airflow-go-pack/sources_test.go | 430 +++++++++++++++
.../airflow-go-pack/testdata/handlersonly/main.go} | 23 +-
.../testdata/multidag/factory/factory.go} | 18 +-
.../airflow-go-pack/testdata/multidag}/main.go | 38 +-
.../testdata/multidag/reports/reports.go} | 18 +-
go-sdk/example/bundle/Justfile | 4 +-
go-sdk/internal/airflowmetadata/airflowmetadata.go | 3 +
go-sdk/internal/bundle/doc.go | 9 +-
go-sdk/internal/bundle/task.go | 5 +
go-sdk/pkg/execution/metadata.go | 5 +
go-sdk/pkg/execution/metadata_test.go | 36 ++
java-sdk/README.md | 22 +-
java-sdk/capabilities.yaml | 17 +-
.../example/nativedag/AnnotationExample.java | 17 +-
.../org/apache/airflow/sdk/BuilderProcessor.kt | 324 +++++++++--
.../kotlin/org/apache/airflow/sdk/BuilderTest.kt | 511 +++++++++++++++++-
.../ci/prek/check_serialization_conformance.py | 30 +-
java-sdk/sdk/build.gradle.kts | 52 +-
java-sdk/sdk/module.md | 22 +-
.../main/kotlin/org/apache/airflow/sdk/Bundle.kt | 4 +-
.../main/kotlin/org/apache/airflow/sdk/DagDef.kt | 150 +++++-
.../src/main/kotlin/org/apache/airflow/sdk/Deps.kt | 99 +++-
.../main/kotlin/org/apache/airflow/sdk/Endpoint.kt | 15 +-
.../main/kotlin/org/apache/airflow/sdk/Server.kt | 11 +
.../kotlin/org/apache/airflow/sdk/TaskGroupRef.kt | 111 ++++
.../org/apache/airflow/sdk/execution/Serde.kt | 490 +++++++++++++++++
.../org/apache/airflow/sdk/internal/ArgValues.kt | 24 +-
.../kotlin/org/apache/airflow/sdk/internal/Ids.kt | 15 +-
.../kotlin/org/apache/airflow/sdk/internal/Refs.kt | 67 ++-
.../org/apache/airflow/sdk/internal/TaskArgs.kt | 2 +-
.../org/apache/airflow/sdk/ArgTestSupport.kt | 2 +-
.../kotlin/org/apache/airflow/sdk/ArgValuesTest.kt | 11 +
.../kotlin/org/apache/airflow/sdk/ServerTest.kt | 63 +++
.../kotlin/org/apache/airflow/sdk/TaskGroupTest.kt | 286 ++++++++++
.../airflow/sdk/conformance/SerializeJava.kt | 140 +++++
.../org/apache/airflow/sdk/execution/SerdeTest.kt | 428 +++++++++++++++
.../apache/airflow/sdk/internal/ArgValuesTest.kt | 2 +-
.../org/apache/airflow/sdk/internal/RefsTest.kt | 82 ++-
providers/.last_release_date.txt | 2 +-
providers/airbyte/README.rst | 6 +-
providers/airbyte/docs/changelog.rst | 22 +
providers/airbyte/docs/index.rst | 6 +-
providers/airbyte/provider.yaml | 3 +-
providers/airbyte/pyproject.toml | 6 +-
.../src/airflow/providers/airbyte/__init__.py | 2 +-
providers/akeyless/README.rst | 6 +-
providers/akeyless/docs/changelog.rst | 20 +
providers/akeyless/docs/index.rst | 6 +-
providers/akeyless/provider.yaml | 3 +-
providers/akeyless/pyproject.toml | 6 +-
.../src/airflow/providers/akeyless/__init__.py | 2 +-
providers/alibaba/README.rst | 6 +-
providers/alibaba/docs/changelog.rst | 17 +
providers/alibaba/docs/index.rst | 6 +-
providers/alibaba/provider.yaml | 3 +-
providers/alibaba/pyproject.toml | 6 +-
.../src/airflow/providers/alibaba/__init__.py | 2 +-
providers/amazon/README.rst | 12 +-
providers/amazon/docs/changelog.rst | 55 ++
providers/amazon/docs/index.rst | 8 +-
providers/amazon/provider.yaml | 3 +-
providers/amazon/pyproject.toml | 8 +-
.../src/airflow/providers/amazon/__init__.py | 2 +-
.../unit/amazon/aws/transfers/test_s3_to_sftp.py | 12 +
.../unit/amazon/aws/transfers/test_sftp_to_s3.py | 12 +
providers/anthropic/README.rst | 8 +-
providers/anthropic/docs/changelog.rst | 22 +
providers/anthropic/docs/commits.rst | 5 +-
providers/anthropic/docs/index.rst | 8 +-
providers/anthropic/provider.yaml | 3 +-
providers/anthropic/pyproject.toml | 8 +-
.../src/airflow/providers/anthropic/__init__.py | 2 +-
providers/apache/arrow/README.rst | 6 +-
providers/apache/arrow/docs/changelog.rst | 19 +
providers/apache/arrow/docs/commits.rst | 8 +-
providers/apache/arrow/docs/index.rst | 6 +-
providers/apache/arrow/provider.yaml | 3 +-
providers/apache/arrow/pyproject.toml | 6 +-
.../src/airflow/providers/apache/arrow/__init__.py | 2 +-
providers/apache/beam/README.rst | 6 +-
providers/apache/beam/docs/changelog.rst | 17 +
providers/apache/beam/docs/index.rst | 6 +-
providers/apache/beam/provider.yaml | 3 +-
providers/apache/beam/pyproject.toml | 6 +-
.../src/airflow/providers/apache/beam/__init__.py | 2 +-
providers/apache/cassandra/README.rst | 6 +-
providers/apache/cassandra/docs/changelog.rst | 17 +
providers/apache/cassandra/docs/index.rst | 6 +-
providers/apache/cassandra/provider.yaml | 3 +-
providers/apache/cassandra/pyproject.toml | 6 +-
.../airflow/providers/apache/cassandra/__init__.py | 2 +-
providers/apache/drill/README.rst | 6 +-
providers/apache/drill/docs/changelog.rst | 21 +
providers/apache/drill/docs/index.rst | 6 +-
providers/apache/drill/provider.yaml | 3 +-
providers/apache/drill/pyproject.toml | 6 +-
.../src/airflow/providers/apache/drill/__init__.py | 2 +-
providers/apache/druid/README.rst | 6 +-
providers/apache/druid/docs/changelog.rst | 17 +
providers/apache/druid/docs/index.rst | 6 +-
providers/apache/druid/provider.yaml | 3 +-
providers/apache/druid/pyproject.toml | 6 +-
.../src/airflow/providers/apache/druid/__init__.py | 2 +-
providers/apache/flink/README.rst | 6 +-
providers/apache/flink/docs/changelog.rst | 17 +
providers/apache/flink/docs/index.rst | 6 +-
providers/apache/flink/provider.yaml | 3 +-
providers/apache/flink/pyproject.toml | 6 +-
.../src/airflow/providers/apache/flink/__init__.py | 2 +-
providers/apache/hdfs/README.rst | 6 +-
providers/apache/hdfs/docs/changelog.rst | 17 +
providers/apache/hdfs/docs/index.rst | 6 +-
providers/apache/hdfs/provider.yaml | 3 +-
providers/apache/hdfs/pyproject.toml | 6 +-
.../src/airflow/providers/apache/hdfs/__init__.py | 2 +-
providers/apache/hive/README.rst | 8 +-
providers/apache/hive/docs/changelog.rst | 19 +
providers/apache/hive/docs/index.rst | 6 +-
providers/apache/hive/provider.yaml | 3 +-
providers/apache/hive/pyproject.toml | 6 +-
.../src/airflow/providers/apache/hive/__init__.py | 2 +-
providers/apache/iceberg/README.rst | 6 +-
providers/apache/iceberg/docs/changelog.rst | 20 +
providers/apache/iceberg/docs/index.rst | 6 +-
providers/apache/iceberg/provider.yaml | 3 +-
providers/apache/iceberg/pyproject.toml | 6 +-
.../airflow/providers/apache/iceberg/__init__.py | 2 +-
providers/apache/impala/README.rst | 14 +-
providers/apache/impala/docs/changelog.rst | 24 +
providers/apache/impala/docs/index.rst | 6 +-
providers/apache/impala/provider.yaml | 3 +-
providers/apache/impala/pyproject.toml | 6 +-
.../airflow/providers/apache/impala/__init__.py | 2 +-
providers/apache/kafka/README.rst | 6 +-
providers/apache/kafka/docs/changelog.rst | 29 +
providers/apache/kafka/docs/index.rst | 6 +-
providers/apache/kafka/provider.yaml | 3 +-
providers/apache/kafka/pyproject.toml | 6 +-
.../src/airflow/providers/apache/kafka/__init__.py | 2 +-
providers/apache/kylin/README.rst | 6 +-
providers/apache/kylin/docs/changelog.rst | 17 +
providers/apache/kylin/docs/index.rst | 6 +-
providers/apache/kylin/provider.yaml | 3 +-
providers/apache/kylin/pyproject.toml | 6 +-
.../src/airflow/providers/apache/kylin/__init__.py | 2 +-
providers/apache/livy/README.rst | 6 +-
providers/apache/livy/docs/changelog.rst | 19 +
providers/apache/livy/docs/index.rst | 6 +-
providers/apache/livy/provider.yaml | 3 +-
providers/apache/livy/pyproject.toml | 6 +-
.../src/airflow/providers/apache/livy/__init__.py | 2 +-
providers/apache/pig/README.rst | 6 +-
providers/apache/pig/docs/changelog.rst | 17 +
providers/apache/pig/docs/index.rst | 6 +-
providers/apache/pig/provider.yaml | 3 +-
providers/apache/pig/pyproject.toml | 6 +-
.../src/airflow/providers/apache/pig/__init__.py | 2 +-
providers/apache/pinot/README.rst | 6 +-
providers/apache/pinot/docs/changelog.rst | 21 +
providers/apache/pinot/docs/index.rst | 6 +-
providers/apache/pinot/provider.yaml | 3 +-
providers/apache/pinot/pyproject.toml | 6 +-
.../src/airflow/providers/apache/pinot/__init__.py | 2 +-
providers/apache/spark/README.rst | 6 +-
providers/apache/spark/docs/changelog.rst | 27 +
providers/apache/spark/docs/index.rst | 6 +-
providers/apache/spark/provider.yaml | 3 +-
providers/apache/spark/pyproject.toml | 6 +-
.../src/airflow/providers/apache/spark/__init__.py | 2 +-
providers/apache/tinkerpop/README.rst | 6 +-
providers/apache/tinkerpop/docs/changelog.rst | 17 +
providers/apache/tinkerpop/docs/index.rst | 6 +-
providers/apache/tinkerpop/provider.yaml | 3 +-
providers/apache/tinkerpop/pyproject.toml | 6 +-
.../airflow/providers/apache/tinkerpop/__init__.py | 2 +-
providers/apprise/README.rst | 6 +-
providers/apprise/docs/changelog.rst | 17 +
providers/apprise/docs/index.rst | 6 +-
providers/apprise/provider.yaml | 3 +-
providers/apprise/pyproject.toml | 6 +-
.../src/airflow/providers/apprise/__init__.py | 2 +-
providers/arangodb/README.rst | 6 +-
providers/arangodb/docs/changelog.rst | 17 +
providers/arangodb/docs/index.rst | 6 +-
providers/arangodb/provider.yaml | 3 +-
providers/arangodb/pyproject.toml | 6 +-
.../src/airflow/providers/arangodb/__init__.py | 2 +-
providers/asana/README.rst | 6 +-
providers/asana/docs/changelog.rst | 17 +
providers/asana/docs/index.rst | 6 +-
providers/asana/provider.yaml | 3 +-
providers/asana/pyproject.toml | 6 +-
.../asana/src/airflow/providers/asana/__init__.py | 2 +-
providers/atlassian/jira/README.rst | 6 +-
providers/atlassian/jira/docs/changelog.rst | 17 +
providers/atlassian/jira/docs/index.rst | 6 +-
providers/atlassian/jira/provider.yaml | 3 +-
providers/atlassian/jira/pyproject.toml | 6 +-
.../airflow/providers/atlassian/jira/__init__.py | 2 +-
providers/celery/README.rst | 6 +-
providers/celery/docs/changelog.rst | 28 +
providers/celery/docs/index.rst | 6 +-
providers/celery/provider.yaml | 3 +-
providers/celery/pyproject.toml | 6 +-
.../src/airflow/providers/celery/__init__.py | 2 +-
providers/clickhousedb/README.rst | 6 +-
providers/clickhousedb/docs/changelog.rst | 28 +
providers/clickhousedb/docs/index.rst | 6 +-
providers/clickhousedb/provider.yaml | 3 +-
providers/clickhousedb/pyproject.toml | 6 +-
.../src/airflow/providers/clickhousedb/__init__.py | 2 +-
providers/cloudant/README.rst | 6 +-
providers/cloudant/docs/changelog.rst | 17 +
providers/cloudant/docs/index.rst | 6 +-
providers/cloudant/provider.yaml | 3 +-
providers/cloudant/pyproject.toml | 6 +-
.../src/airflow/providers/cloudant/__init__.py | 2 +-
providers/cncf/kubernetes/README.rst | 6 +-
providers/cncf/kubernetes/docs/changelog.rst | 30 ++
providers/cncf/kubernetes/docs/index.rst | 6 +-
providers/cncf/kubernetes/provider.yaml | 27 +-
providers/cncf/kubernetes/pyproject.toml | 6 +-
.../airflow/providers/cncf/kubernetes/__init__.py | 2 +-
.../kubernetes/executors/kubernetes_executor.py | 19 +-
.../executors/kubernetes_executor_utils.py | 17 +-
.../providers/cncf/kubernetes/get_provider_info.py | 14 +
.../providers/cncf/kubernetes/kube_client.py | 43 ++
.../executors/test_kubernetes_executor.py | 175 +++++-
.../tests/unit/cncf/kubernetes/test_kube_client.py | 96 +++-
providers/cohere/README.rst | 6 +-
providers/cohere/docs/changelog.rst | 17 +
providers/cohere/docs/index.rst | 6 +-
providers/cohere/provider.yaml | 3 +-
providers/cohere/pyproject.toml | 6 +-
.../src/airflow/providers/cohere/__init__.py | 2 +-
providers/common/ai/README.rst | 13 +-
providers/common/ai/docs/changelog.rst | 82 +++
providers/common/ai/docs/index.rst | 8 +-
.../common/ai/docs/toolsets/managed_agent.rst | 8 +
providers/common/ai/provider.yaml | 3 +-
providers/common/ai/pyproject.toml | 8 +-
.../ai/src/airflow/providers/common/ai/__init__.py | 2 +-
providers/common/compat/README.rst | 6 +-
providers/common/compat/docs/changelog.rst | 27 +
providers/common/compat/docs/index.rst | 6 +-
providers/common/compat/provider.yaml | 3 +-
providers/common/compat/pyproject.toml | 6 +-
.../airflow/providers/common/compat/__init__.py | 2 +-
providers/common/io/README.rst | 6 +-
providers/common/io/docs/changelog.rst | 14 +
providers/common/io/docs/index.rst | 6 +-
providers/common/io/docs/state_store_backend.rst | 4 +-
providers/common/io/provider.yaml | 5 +-
providers/common/io/pyproject.toml | 6 +-
.../io/src/airflow/providers/common/io/__init__.py | 2 +-
.../providers/common/io/get_provider_info.py | 2 +-
providers/common/messaging/README.rst | 6 +-
providers/common/messaging/docs/changelog.rst | 17 +
providers/common/messaging/docs/index.rst | 6 +-
providers/common/messaging/provider.yaml | 3 +-
providers/common/messaging/pyproject.toml | 6 +-
.../airflow/providers/common/messaging/__init__.py | 2 +-
providers/common/sql/README.rst | 8 +-
providers/common/sql/docs/changelog.rst | 27 +
providers/common/sql/docs/index.rst | 6 +-
providers/common/sql/provider.yaml | 3 +-
providers/common/sql/pyproject.toml | 6 +-
.../src/airflow/providers/common/sql/__init__.py | 2 +-
providers/databricks/README.rst | 10 +-
providers/databricks/docs/changelog.rst | 25 +
providers/databricks/docs/index.rst | 8 +-
providers/databricks/provider.yaml | 3 +-
providers/databricks/pyproject.toml | 8 +-
.../src/airflow/providers/databricks/__init__.py | 2 +-
.../providers/databricks/hooks/databricks_base.py | 8 +
.../tests/unit/databricks/hooks/test_databricks.py | 27 +
providers/datadog/README.rst | 6 +-
providers/datadog/docs/changelog.rst | 17 +
providers/datadog/docs/index.rst | 6 +-
providers/datadog/provider.yaml | 3 +-
providers/datadog/pyproject.toml | 6 +-
.../src/airflow/providers/datadog/__init__.py | 2 +-
providers/dbt/cloud/README.rst | 6 +-
providers/dbt/cloud/docs/changelog.rst | 27 +
providers/dbt/cloud/docs/index.rst | 6 +-
providers/dbt/cloud/provider.yaml | 3 +-
providers/dbt/cloud/pyproject.toml | 6 +-
.../src/airflow/providers/dbt/cloud/__init__.py | 2 +-
providers/dingding/README.rst | 6 +-
providers/dingding/docs/changelog.rst | 17 +
providers/dingding/docs/index.rst | 6 +-
providers/dingding/provider.yaml | 3 +-
providers/dingding/pyproject.toml | 6 +-
.../src/airflow/providers/dingding/__init__.py | 2 +-
providers/discord/README.rst | 6 +-
providers/discord/docs/changelog.rst | 26 +
providers/discord/docs/index.rst | 6 +-
providers/discord/provider.yaml | 3 +-
providers/discord/pyproject.toml | 6 +-
.../src/airflow/providers/discord/__init__.py | 2 +-
providers/docker/README.rst | 6 +-
providers/docker/docs/changelog.rst | 17 +
providers/docker/docs/index.rst | 6 +-
providers/docker/provider.yaml | 3 +-
providers/docker/pyproject.toml | 6 +-
.../src/airflow/providers/docker/__init__.py | 2 +-
providers/duckdb/README.rst | 6 +-
providers/duckdb/docs/changelog.rst | 17 +
providers/duckdb/docs/commits.rst | 4 +-
providers/duckdb/docs/index.rst | 6 +-
providers/duckdb/provider.yaml | 3 +-
providers/duckdb/pyproject.toml | 6 +-
.../src/airflow/providers/duckdb/__init__.py | 2 +-
providers/edge3/docs/changelog.rst | 17 +-
providers/edge3/provider.yaml | 2 +-
providers/elasticsearch/README.rst | 6 +-
providers/elasticsearch/docs/changelog.rst | 25 +
providers/elasticsearch/docs/index.rst | 6 +-
providers/elasticsearch/provider.yaml | 3 +-
providers/elasticsearch/pyproject.toml | 6 +-
.../airflow/providers/elasticsearch/__init__.py | 2 +-
providers/exasol/README.rst | 14 +-
providers/exasol/docs/changelog.rst | 22 +
providers/exasol/docs/index.rst | 6 +-
providers/exasol/provider.yaml | 3 +-
providers/exasol/pyproject.toml | 6 +-
.../src/airflow/providers/exasol/__init__.py | 2 +-
providers/fab/README.rst | 6 +-
providers/fab/docs/changelog.rst | 30 ++
providers/fab/docs/index.rst | 6 +-
providers/fab/provider.yaml | 3 +-
providers/fab/pyproject.toml | 6 +-
.../providers/3rd-party-licenses/LICENSES-ui.txt | 2 +-
.../fab/src/airflow/providers/fab/__init__.py | 2 +-
.../fab/src/airflow/providers/fab/www/package.json | 4 +-
.../src/airflow/providers/fab/www/pnpm-lock.yaml | 30 +-
providers/facebook/README.rst | 6 +-
providers/facebook/docs/changelog.rst | 21 +
providers/facebook/docs/index.rst | 6 +-
providers/facebook/provider.yaml | 3 +-
providers/facebook/pyproject.toml | 6 +-
.../src/airflow/providers/facebook/__init__.py | 2 +-
providers/ftp/README.rst | 6 +-
providers/ftp/docs/changelog.rst | 22 +
providers/ftp/docs/index.rst | 6 +-
providers/ftp/provider.yaml | 3 +-
providers/ftp/pyproject.toml | 6 +-
.../ftp/src/airflow/providers/ftp/__init__.py | 2 +-
providers/git/README.rst | 6 +-
providers/git/docs/changelog.rst | 24 +
providers/git/docs/index.rst | 6 +-
providers/git/provider.yaml | 3 +-
providers/git/pyproject.toml | 6 +-
.../git/src/airflow/providers/git/__init__.py | 2 +-
providers/github/README.rst | 6 +-
providers/github/docs/changelog.rst | 17 +
providers/github/docs/index.rst | 6 +-
providers/github/provider.yaml | 3 +-
providers/github/pyproject.toml | 6 +-
.../src/airflow/providers/github/__init__.py | 2 +-
providers/google/README.rst | 16 +-
providers/google/docs/changelog.rst | 51 ++
providers/google/docs/index.rst | 8 +-
providers/google/provider.yaml | 3 +-
providers/google/pyproject.toml | 8 +-
.../src/airflow/providers/google/__init__.py | 2 +-
.../providers/google/cloud/hooks/datafusion.py | 7 +-
.../providers/google/cloud/sensors/datafusion.py | 11 +-
.../unit/google/cloud/hooks/test_datafusion.py | 38 ++
.../unit/google/cloud/sensors/test_datafusion.py | 35 +-
providers/grpc/README.rst | 6 +-
providers/grpc/docs/changelog.rst | 17 +
providers/grpc/docs/index.rst | 6 +-
providers/grpc/provider.yaml | 3 +-
providers/grpc/pyproject.toml | 6 +-
.../grpc/src/airflow/providers/grpc/__init__.py | 2 +-
providers/hashicorp/README.rst | 8 +-
providers/hashicorp/docs/changelog.rst | 22 +
providers/hashicorp/docs/index.rst | 6 +-
providers/hashicorp/provider.yaml | 3 +-
providers/hashicorp/pyproject.toml | 6 +-
.../src/airflow/providers/hashicorp/__init__.py | 2 +-
providers/http/README.rst | 6 +-
providers/http/docs/changelog.rst | 28 +
providers/http/docs/index.rst | 6 +-
providers/http/provider.yaml | 3 +-
providers/http/pyproject.toml | 6 +-
.../http/src/airflow/providers/http/__init__.py | 2 +-
providers/ibm/db2/README.rst | 121 ++---
providers/ibm/mq/README.rst | 43 +-
providers/imap/README.rst | 6 +-
providers/imap/docs/changelog.rst | 17 +
providers/imap/docs/index.rst | 6 +-
providers/imap/provider.yaml | 3 +-
providers/imap/pyproject.toml | 6 +-
.../imap/src/airflow/providers/imap/__init__.py | 2 +-
providers/influxdb/README.rst | 6 +-
providers/influxdb/docs/changelog.rst | 22 +
providers/influxdb/docs/index.rst | 6 +-
providers/influxdb/provider.yaml | 3 +-
providers/influxdb/pyproject.toml | 6 +-
.../src/airflow/providers/influxdb/__init__.py | 2 +-
providers/informatica/README.rst | 6 +-
providers/informatica/docs/changelog.rst | 19 +
providers/informatica/docs/index.rst | 6 +-
providers/informatica/provider.yaml | 3 +-
providers/informatica/pyproject.toml | 6 +-
.../src/airflow/providers/informatica/__init__.py | 2 +-
providers/jdbc/README.rst | 8 +-
providers/jdbc/docs/changelog.rst | 15 +
providers/jdbc/docs/index.rst | 6 +-
providers/jdbc/provider.yaml | 3 +-
providers/jdbc/pyproject.toml | 6 +-
.../jdbc/src/airflow/providers/jdbc/__init__.py | 2 +-
providers/jenkins/README.rst | 6 +-
providers/jenkins/docs/changelog.rst | 23 +
providers/jenkins/docs/index.rst | 6 +-
providers/jenkins/provider.yaml | 3 +-
providers/jenkins/pyproject.toml | 6 +-
.../src/airflow/providers/jenkins/__init__.py | 2 +-
providers/keycloak/README.rst | 6 +-
providers/keycloak/docs/changelog.rst | 23 +
providers/keycloak/docs/index.rst | 6 +-
providers/keycloak/provider.yaml | 3 +-
providers/keycloak/pyproject.toml | 6 +-
.../src/airflow/providers/keycloak/__init__.py | 2 +-
providers/microsoft/azure/README.rst | 8 +-
providers/microsoft/azure/docs/changelog.rst | 27 +
providers/microsoft/azure/docs/index.rst | 8 +-
providers/microsoft/azure/provider.yaml | 3 +-
providers/microsoft/azure/pyproject.toml | 8 +-
.../airflow/providers/microsoft/azure/__init__.py | 2 +-
providers/microsoft/mssql/README.rst | 6 +-
providers/microsoft/mssql/docs/changelog.rst | 21 +
providers/microsoft/mssql/docs/index.rst | 6 +-
providers/microsoft/mssql/provider.yaml | 3 +-
providers/microsoft/mssql/pyproject.toml | 6 +-
.../airflow/providers/microsoft/mssql/__init__.py | 2 +-
providers/microsoft/psrp/README.rst | 6 +-
providers/microsoft/psrp/docs/changelog.rst | 17 +
providers/microsoft/psrp/docs/index.rst | 6 +-
providers/microsoft/psrp/provider.yaml | 3 +-
providers/microsoft/psrp/pyproject.toml | 6 +-
.../airflow/providers/microsoft/psrp/__init__.py | 2 +-
providers/microsoft/winrm/README.rst | 6 +-
providers/microsoft/winrm/docs/changelog.rst | 17 +
providers/microsoft/winrm/docs/index.rst | 6 +-
providers/microsoft/winrm/provider.yaml | 3 +-
providers/microsoft/winrm/pyproject.toml | 6 +-
.../airflow/providers/microsoft/winrm/__init__.py | 2 +-
providers/modal/README.rst | 1 -
providers/mongo/README.rst | 6 +-
providers/mongo/docs/changelog.rst | 17 +
providers/mongo/docs/index.rst | 6 +-
providers/mongo/provider.yaml | 3 +-
providers/mongo/pyproject.toml | 6 +-
.../mongo/src/airflow/providers/mongo/__init__.py | 2 +-
providers/mysql/README.rst | 6 +-
providers/mysql/docs/changelog.rst | 17 +
providers/mysql/docs/index.rst | 6 +-
providers/mysql/provider.yaml | 3 +-
providers/mysql/pyproject.toml | 6 +-
.../mysql/src/airflow/providers/mysql/__init__.py | 2 +-
providers/neo4j/README.rst | 6 +-
providers/neo4j/docs/changelog.rst | 17 +
providers/neo4j/docs/index.rst | 6 +-
providers/neo4j/provider.yaml | 3 +-
providers/neo4j/pyproject.toml | 6 +-
.../neo4j/src/airflow/providers/neo4j/__init__.py | 2 +-
providers/odbc/README.rst | 6 +-
providers/odbc/docs/changelog.rst | 19 +
providers/odbc/docs/index.rst | 6 +-
providers/odbc/provider.yaml | 3 +-
providers/odbc/pyproject.toml | 6 +-
.../odbc/src/airflow/providers/odbc/__init__.py | 2 +-
providers/openai/README.rst | 6 +-
providers/openai/docs/changelog.rst | 27 +
providers/openai/docs/commits.rst | 2 +-
providers/openai/docs/index.rst | 6 +-
providers/openai/provider.yaml | 3 +-
providers/openai/pyproject.toml | 6 +-
.../src/airflow/providers/openai/__init__.py | 2 +-
providers/openfaas/README.rst | 6 +-
providers/openfaas/docs/changelog.rst | 21 +
providers/openfaas/docs/index.rst | 6 +-
providers/openfaas/provider.yaml | 3 +-
providers/openfaas/pyproject.toml | 6 +-
.../src/airflow/providers/openfaas/__init__.py | 2 +-
providers/openlineage/README.rst | 14 +-
providers/openlineage/docs/changelog.rst | 25 +
providers/openlineage/docs/index.rst | 6 +-
providers/openlineage/provider.yaml | 3 +-
providers/openlineage/pyproject.toml | 6 +-
.../src/airflow/providers/openlineage/__init__.py | 2 +-
providers/opensearch/README.rst | 6 +-
providers/opensearch/docs/changelog.rst | 30 ++
providers/opensearch/docs/index.rst | 6 +-
providers/opensearch/provider.yaml | 3 +-
providers/opensearch/pyproject.toml | 6 +-
.../src/airflow/providers/opensearch/__init__.py | 2 +-
providers/opsgenie/README.rst | 6 +-
providers/opsgenie/docs/changelog.rst | 18 +
providers/opsgenie/docs/index.rst | 6 +-
providers/opsgenie/provider.yaml | 3 +-
providers/opsgenie/pyproject.toml | 6 +-
.../src/airflow/providers/opsgenie/__init__.py | 2 +-
providers/oracle/README.rst | 12 +-
providers/oracle/docs/changelog.rst | 17 +
providers/oracle/docs/index.rst | 6 +-
providers/oracle/provider.yaml | 3 +-
providers/oracle/pyproject.toml | 6 +-
.../src/airflow/providers/oracle/__init__.py | 2 +-
providers/pagerduty/README.rst | 6 +-
providers/pagerduty/docs/changelog.rst | 17 +
providers/pagerduty/docs/index.rst | 6 +-
providers/pagerduty/provider.yaml | 3 +-
providers/pagerduty/pyproject.toml | 6 +-
.../src/airflow/providers/pagerduty/__init__.py | 2 +-
providers/papermill/README.rst | 6 +-
providers/papermill/docs/changelog.rst | 20 +
providers/papermill/docs/index.rst | 6 +-
providers/papermill/provider.yaml | 3 +-
providers/papermill/pyproject.toml | 6 +-
.../src/airflow/providers/papermill/__init__.py | 2 +-
providers/pgvector/README.rst | 6 +-
providers/pgvector/docs/changelog.rst | 20 +
providers/pgvector/docs/index.rst | 6 +-
providers/pgvector/provider.yaml | 3 +-
providers/pgvector/pyproject.toml | 6 +-
.../src/airflow/providers/pgvector/__init__.py | 2 +-
providers/pinecone/README.rst | 6 +-
providers/pinecone/docs/changelog.rst | 17 +
providers/pinecone/docs/index.rst | 6 +-
providers/pinecone/provider.yaml | 3 +-
providers/pinecone/pyproject.toml | 6 +-
.../src/airflow/providers/pinecone/__init__.py | 2 +-
providers/postgres/README.rst | 8 +-
providers/postgres/docs/changelog.rst | 23 +
providers/postgres/docs/index.rst | 6 +-
providers/postgres/provider.yaml | 3 +-
providers/postgres/pyproject.toml | 6 +-
.../src/airflow/providers/postgres/__init__.py | 2 +-
providers/presto/README.rst | 8 +-
providers/presto/docs/changelog.rst | 22 +
providers/presto/docs/index.rst | 6 +-
providers/presto/provider.yaml | 3 +-
providers/presto/pyproject.toml | 6 +-
.../src/airflow/providers/presto/__init__.py | 2 +-
providers/qdrant/README.rst | 6 +-
providers/qdrant/docs/changelog.rst | 17 +
providers/qdrant/docs/index.rst | 6 +-
providers/qdrant/provider.yaml | 3 +-
providers/qdrant/pyproject.toml | 6 +-
.../src/airflow/providers/qdrant/__init__.py | 2 +-
providers/redis/README.rst | 6 +-
providers/redis/docs/changelog.rst | 17 +
providers/redis/docs/index.rst | 6 +-
providers/redis/provider.yaml | 3 +-
providers/redis/pyproject.toml | 6 +-
.../redis/src/airflow/providers/redis/__init__.py | 2 +-
providers/salesforce/README.rst | 6 +-
providers/salesforce/docs/changelog.rst | 18 +
providers/salesforce/docs/index.rst | 6 +-
providers/salesforce/provider.yaml | 3 +-
providers/salesforce/pyproject.toml | 6 +-
.../src/airflow/providers/salesforce/__init__.py | 2 +-
providers/samba/README.rst | 6 +-
providers/samba/docs/changelog.rst | 17 +
providers/samba/docs/index.rst | 6 +-
providers/samba/provider.yaml | 3 +-
providers/samba/pyproject.toml | 6 +-
.../samba/src/airflow/providers/samba/__init__.py | 2 +-
providers/segment/README.rst | 6 +-
providers/segment/docs/changelog.rst | 17 +
providers/segment/docs/index.rst | 6 +-
providers/segment/provider.yaml | 3 +-
providers/segment/pyproject.toml | 6 +-
.../src/airflow/providers/segment/__init__.py | 2 +-
providers/sendgrid/README.rst | 6 +-
providers/sendgrid/docs/changelog.rst | 23 +
providers/sendgrid/docs/index.rst | 6 +-
providers/sendgrid/provider.yaml | 3 +-
providers/sendgrid/pyproject.toml | 6 +-
.../src/airflow/providers/sendgrid/__init__.py | 2 +-
providers/sftp/README.rst | 8 +-
providers/sftp/docs/changelog.rst | 44 ++
providers/sftp/docs/connections/sftp.rst | 2 +-
providers/sftp/docs/index.rst | 8 +-
providers/sftp/provider.yaml | 3 +-
providers/sftp/pyproject.toml | 8 +-
.../sftp/src/airflow/providers/sftp/__init__.py | 2 +-
.../sftp/src/airflow/providers/sftp/hooks/sftp.py | 38 +-
.../src/airflow/providers/sftp/operators/sftp.py | 1 +
.../src/airflow/providers/sftp/triggers/sftp.py | 22 +-
providers/sftp/tests/unit/sftp/hooks/test_sftp.py | 205 ++++++-
.../sftp/tests/unit/sftp/operators/test_sftp.py | 31 +-
providers/singularity/README.rst | 6 +-
providers/singularity/docs/changelog.rst | 21 +
providers/singularity/docs/index.rst | 6 +-
providers/singularity/provider.yaml | 3 +-
providers/singularity/pyproject.toml | 6 +-
.../src/airflow/providers/singularity/__init__.py | 2 +-
providers/slack/README.rst | 6 +-
providers/slack/docs/changelog.rst | 23 +
providers/slack/docs/index.rst | 6 +-
providers/slack/provider.yaml | 3 +-
providers/slack/pyproject.toml | 6 +-
.../slack/src/airflow/providers/slack/__init__.py | 2 +-
providers/smtp/README.rst | 6 +-
providers/smtp/docs/changelog.rst | 23 +
providers/smtp/docs/index.rst | 6 +-
providers/smtp/provider.yaml | 3 +-
providers/smtp/pyproject.toml | 6 +-
.../smtp/src/airflow/providers/smtp/__init__.py | 2 +-
.../smtp/src/airflow/providers/smtp/hooks/smtp.py | 10 +-
providers/smtp/tests/unit/smtp/hooks/test_smtp.py | 42 ++
.../tests/unit/smtp/notifications/test_smtp.py | 27 +
providers/snowflake/README.rst | 16 +-
providers/snowflake/docs/changelog.rst | 31 ++
providers/snowflake/docs/index.rst | 6 +-
providers/snowflake/provider.yaml | 3 +-
providers/snowflake/pyproject.toml | 6 +-
.../src/airflow/providers/snowflake/__init__.py | 2 +-
providers/sqlite/README.rst | 6 +-
providers/sqlite/docs/changelog.rst | 23 +
providers/sqlite/docs/index.rst | 6 +-
providers/sqlite/provider.yaml | 3 +-
providers/sqlite/pyproject.toml | 6 +-
.../src/airflow/providers/sqlite/__init__.py | 2 +-
providers/ssh/README.rst | 6 +-
providers/ssh/docs/changelog.rst | 42 ++
providers/ssh/docs/connections/ssh.rst | 2 +-
providers/ssh/docs/index.rst | 6 +-
providers/ssh/provider.yaml | 3 +-
providers/ssh/pyproject.toml | 6 +-
.../ssh/src/airflow/providers/ssh/__init__.py | 2 +-
.../ssh/src/airflow/providers/ssh/hooks/ssh.py | 71 ++-
providers/ssh/tests/unit/ssh/hooks/test_ssh.py | 67 ++-
.../ssh/tests/unit/ssh/hooks/test_ssh_async.py | 64 +++
providers/standard/README.rst | 6 +-
providers/standard/docs/changelog.rst | 25 +
providers/standard/docs/index.rst | 6 +-
providers/standard/provider.yaml | 3 +-
providers/standard/pyproject.toml | 6 +-
.../src/airflow/providers/standard/__init__.py | 2 +-
providers/tableau/README.rst | 6 +-
providers/tableau/docs/changelog.rst | 17 +
providers/tableau/docs/index.rst | 6 +-
providers/tableau/provider.yaml | 3 +-
providers/tableau/pyproject.toml | 6 +-
.../src/airflow/providers/tableau/__init__.py | 2 +-
providers/telegram/README.rst | 6 +-
providers/telegram/docs/changelog.rst | 18 +
providers/telegram/docs/index.rst | 6 +-
providers/telegram/provider.yaml | 3 +-
providers/telegram/pyproject.toml | 6 +-
.../src/airflow/providers/telegram/__init__.py | 2 +-
providers/teradata/README.rst | 8 +-
providers/teradata/docs/changelog.rst | 19 +
providers/teradata/docs/index.rst | 6 +-
providers/teradata/provider.yaml | 3 +-
providers/teradata/pyproject.toml | 6 +-
.../src/airflow/providers/teradata/__init__.py | 2 +-
providers/trino/README.rst | 6 +-
providers/trino/docs/changelog.rst | 17 +
providers/trino/docs/index.rst | 6 +-
providers/trino/provider.yaml | 3 +-
providers/trino/pyproject.toml | 6 +-
.../trino/src/airflow/providers/trino/__init__.py | 2 +-
providers/vertica/README.rst | 14 +-
providers/vertica/docs/changelog.rst | 28 +
providers/vertica/docs/index.rst | 6 +-
providers/vertica/provider.yaml | 3 +-
providers/vertica/pyproject.toml | 6 +-
.../src/airflow/providers/vertica/__init__.py | 2 +-
providers/vespa/README.rst | 6 +-
providers/vespa/docs/changelog.rst | 21 +
providers/vespa/docs/index.rst | 6 +-
providers/vespa/provider.yaml | 3 +-
providers/vespa/pyproject.toml | 6 +-
.../vespa/src/airflow/providers/vespa/__init__.py | 2 +-
providers/weaviate/README.rst | 6 +-
providers/weaviate/docs/changelog.rst | 17 +
providers/weaviate/docs/index.rst | 6 +-
providers/weaviate/provider.yaml | 3 +-
providers/weaviate/pyproject.toml | 6 +-
.../src/airflow/providers/weaviate/__init__.py | 2 +-
providers/yandex/README.rst | 6 +-
providers/yandex/docs/changelog.rst | 17 +
providers/yandex/docs/index.rst | 6 +-
providers/yandex/provider.yaml | 3 +-
providers/yandex/pyproject.toml | 6 +-
.../src/airflow/providers/yandex/__init__.py | 2 +-
providers/ydb/README.rst | 14 +-
providers/ydb/docs/changelog.rst | 16 +
providers/ydb/docs/index.rst | 6 +-
providers/ydb/provider.yaml | 3 +-
providers/ydb/pyproject.toml | 6 +-
.../ydb/src/airflow/providers/ydb/__init__.py | 2 +-
providers/zendesk/README.rst | 6 +-
providers/zendesk/docs/changelog.rst | 21 +
providers/zendesk/docs/index.rst | 6 +-
providers/zendesk/provider.yaml | 3 +-
providers/zendesk/pyproject.toml | 6 +-
.../src/airflow/providers/zendesk/__init__.py | 2 +-
scripts/ci/lang_sdk_serialization/compare.py | 17 +-
scripts/ci/lang_sdk_serialization/test_dags.yaml | 4 +-
.../ci/prek/check_openapi_exception_doc_in_sync.py | 201 ++++++-
scripts/ci/prek/check_xcom_model_columns.py | 2 +-
...nc_go_sdk_schemas.py => sync_ts_sdk_schemas.py} | 66 +--
.../ci/lang_sdk_serialization/test_compare.py | 11 +
.../test_check_openapi_exception_doc_in_sync.py | 114 +++-
..._sdk_schemas.py => test_sync_ts_sdk_schemas.py} | 39 +-
shared/dagnode/src/airflow_shared/dagnode/node.py | 113 +++-
shared/dagnode/tests/dagnode/test_node.py | 36 +-
.../observability/metrics/metrics_template.yaml | 6 +-
.../plugins_manager/plugins_manager.py | 12 +-
task-sdk/docs/airflow-metadata.schema.json | 49 +-
task-sdk/docs/executable-bundle-spec.rst | 86 ++-
.../airflow/sdk/coordinators/_bundle_metadata.py | 16 +
.../src/airflow/sdk/coordinators/_dag_importer.py | 1 +
.../sdk/coordinators/executable/_bundle_reader.py | 195 +++++++
.../sdk/coordinators/executable/_dag_importer.py | 105 ++++
.../sdk/coordinators/executable/coordinator.py | 108 +++-
task-sdk/src/airflow/sdk/definitions/taskgroup.py | 45 +-
task-sdk/src/airflow/sdk/execution_time/loop.py | 6 +-
task-sdk/src/airflow/sdk/log.py | 15 +-
.../coordinators/executable/_bundle_test_utils.py | 98 ++++
.../coordinators/executable/test_bundle_reader.py | 217 ++++++++
.../coordinators/executable/test_coordinator.py | 168 +++++-
.../coordinators/executable/test_dag_importer.py | 283 ++++++++++
task-sdk/tests/task_sdk/definitions/test_dag.py | 12 +
.../tests/task_sdk/execution_time/test_loop.py | 11 +
task-sdk/tests/task_sdk/test_log.py | 89 +++
ts-sdk/.pre-commit-config.yaml | 1 +
{go-sdk => ts-sdk}/schema/supervisor-schema.json | 0
ts-sdk/scripts/ci/prek/check_supervisor_schema.py | 4 +-
ts-sdk/scripts/ci/prek/sync_dag_schema.py | 65 ---
ts-sdk/scripts/generate-dag-schema.mjs | 2 +-
ts-sdk/scripts/generate-supervisor.mjs | 7 +-
ts-sdk/src/generated/supervisor.ts | 104 ++--
uv.lock | 211 ++++----
975 files changed, 14316 insertions(+), 3788 deletions(-)
create mode 100644 airflow-core/newsfragments/66633.bugfix.rst
delete mode 100644 airflow-core/newsfragments/69148.feature.rst
create mode 100644 airflow-core/newsfragments/74278.feature.rst
create mode 100644
airflow-core/src/airflow/ui/src/components/DataTable/DataTable.stateChange.test.tsx
copy airflow-core/src/airflow/ui/src/components/{DAGWarningsModal.test.tsx =>
DataTable/features.ts} (62%)
copy
providers/edge3/src/airflow/providers/edge3/plugins/www/src/components/StateBadge.tsx
=> airflow-core/src/airflow/ui/src/components/MarkAs/utils.test.ts (57%)
copy airflow-core/src/airflow/ui/src/{components/StatsCard.test.tsx =>
utils/renderMermaid.test.ts} (58%)
copy go-sdk/{pkg/logging/level.go => airflow/dag_factory_test.go} (79%)
create mode 100644 go-sdk/airflow/dag_source_test.go
create mode 100644 go-sdk/cmd/airflow-go-pack/sources.go
create mode 100644 go-sdk/cmd/airflow-go-pack/sources_test.go
copy go-sdk/{pkg/logging/attrs.go =>
cmd/airflow-go-pack/testdata/handlersonly/main.go} (65%)
copy go-sdk/{pkg/logging/attrs.go =>
cmd/airflow-go-pack/testdata/multidag/factory/factory.go} (74%)
copy go-sdk/{airflow/testdata/foreignnode =>
cmd/airflow-go-pack/testdata/multidag}/main.go (56%)
copy go-sdk/{pkg/logging/attrs.go =>
cmd/airflow-go-pack/testdata/multidag/reports/reports.go} (73%)
copy {ts-sdk => java-sdk}/scripts/ci/prek/check_serialization_conformance.py
(62%)
copy airflow-core/src/airflow/ui/src/hooks/useShowTeam.ts =>
java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/Endpoint.kt (69%)
create mode 100644
java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/TaskGroupRef.kt
create mode 100644
java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/execution/Serde.kt
copy airflow-core/src/airflow/ui/src/hooks/useShowTeam.ts =>
java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/internal/Ids.kt (71%)
create mode 100644
java-sdk/sdk/src/test/kotlin/org/apache/airflow/sdk/TaskGroupTest.kt
create mode 100644
java-sdk/sdk/src/test/kotlin/org/apache/airflow/sdk/conformance/SerializeJava.kt
create mode 100644
java-sdk/sdk/src/test/kotlin/org/apache/airflow/sdk/execution/SerdeTest.kt
copy scripts/ci/prek/{sync_go_sdk_schemas.py => sync_ts_sdk_schemas.py} (56%)
copy scripts/tests/ci/prek/{test_sync_go_sdk_schemas.py =>
test_sync_ts_sdk_schemas.py} (75%)
create mode 100644
task-sdk/src/airflow/sdk/coordinators/executable/_bundle_reader.py
create mode 100644
task-sdk/src/airflow/sdk/coordinators/executable/_dag_importer.py
create mode 100644
task-sdk/tests/task_sdk/coordinators/executable/_bundle_test_utils.py
create mode 100644
task-sdk/tests/task_sdk/coordinators/executable/test_bundle_reader.py
create mode 100644
task-sdk/tests/task_sdk/coordinators/executable/test_dag_importer.py
create mode 100644 task-sdk/tests/task_sdk/test_log.py
copy {go-sdk => ts-sdk}/schema/supervisor-schema.json (100%)
delete mode 100755 ts-sdk/scripts/ci/prek/sync_dag_schema.py
