This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch jason/lang-sdk-e2e/08-native-e2e
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit 9b90cd28cbe Build the k8s TypeScript artifacts only for the native Dag
test
omit bb9571b2854 Share the Dag run lookup and the native Java queue name
omit 235e1068f67 Run the Lang-SDK e2e jobs when native Dag parsing changes
omit cd0c15cf1fa Add a native Java Dag to the compose e2e
omit dd27975f491 Keep the containerized Java build's Maven repository on
the host
omit 966f757221b Check the native TypeScript Dag's source and triggered run
omit 2b956f97ab4 Parse the native TypeScript Dag in the compose e2e
omit 7c4c206de69 TS SDK: cover native TypeScript Dags end to end
omit a943bc2f31c Java SDK: Serialize native Dags to DagSerialization v3 on
parse requests
omit 542927f5829 Java SDK: Declare a Dag's task graph with @Builder.Deps
omit 7c2f6063c60 Java SDK: Resolve a task's arguments from the Dag's own
wiring
omit 079161f0e0b Java SDK: Generate Dag and task configuration from the
serialization schema
omit 5593c8f92fa Java SDK: Describe native Java Dags without reference to
Python
omit 888dc778166 Complete the license header of the TS SDK conformance check
omit d86e6693175 TS SDK: add triggerDagRun to the native Dag surface
omit 91cdfefcbba TS SDK: add multi-way branching with dag.switch
omit ca1414a4f4e TS SDK: add conditional branching with dag.if
omit f7d858a8a77 TS SDK: answer the Dag-parsing request from bundle.serve
omit c354e508aec TS SDK: serialize a native Dag to Dag JSON
omit 948ab339189 TS SDK: reject a Dag whose order-only edges form a cycle
omit 5ba7e3552cf TS SDK: add task groups to the native Dag surface
omit 234687c4f78 TS SDK: draw order-only edges with before and after
omit 12eec52d5de TS SDK: default a task id to the handler's name
omit b441444a504 Share the bundle section mutation helper between Node tests
omit 51181dbaaef Document parsing native TypeScript Dags
omit 703a846c474 Parse native TypeScript Dags with the Node coordinator
omit 4619343691a Name the JAR manifest attributes the Java coordinator reads
omit 059fb010913 Test the Java Dag source placeholder for a JAR without a
manifest
omit 0345380d615 Treat an unreadable JAR like one without a manifest
omit 38870368149 Document native Java Dag parsing
omit fc7bdb31bbc Parse native Java Dags with their coordinator
omit 95656dcaab9 Embed the Dag source in Java bundle JARs
omit 9e5288476d4 Build the JVM command that parses a Dag JAR
omit 27324eafa27 Read JAR manifests per the JAR specification
omit 6242f832450 Start a coordinator's runtime from a fresh interpreter on
macOS
omit 4ac3ea65674 Keep a bundle's importers when the coordinator config is
broken
omit 223e57a3dcc Store only native Dags that a Dag bag can rebuild
omit c35d7350951 Start a Lang-SDK parse runtime with default signal handling
omit 5e21873e033 Keep the first parse result from a Lang-SDK runtime
omit b1b72ce9a84 Run the Lang-SDK runtime in its own process group
omit 6e8bc728a57 Report an invalid frame from a Lang-SDK runtime as an
import error
omit 9b6b895a81b Name the missing task when a native Dag's edge points
nowhere
omit 4db8595aef0 Keep a native Dag's timezone when rebuilding it
omit 7e47a2fe824 Reject a native Dag whose task graph has a cycle
omit ed3d1c829ff Signal a Lang-SDK runtime's whole process tree
omit 61476ac72e9 Kill the Lang-SDK runtime when its parse is interrupted
omit c23c13fd14a Parse coordinator-claimed Dag files with their runtime
omit d1429705f42 Import native Dags through their coordinator
omit 8c8aaa84a20 Parse a coordinator's Dag file outside the Dag processor
omit 8a5d3042f9d Parse a Dag file with its coordinator's runtime
omit bf86b2e69be Rebuild a Lang-SDK Dag as an airflow.sdk.DAG
omit 806e80644ed Keep a stub task's preset arg bindings when serializing it
omit c7c9d35d31d Find the coordinators that parse a Dag bundle
omit 82ee9ae0918 Harden claimed-file discovery errors and archive importers
omit 9eed9ec1b9f Tidy the importer routing helpers and their tests
omit 1a42d02c217 Keep claimed files queued when their importer cannot list
them
omit 3e2e43eff5e Claim Dag files by extension and keep .py, .pyc and .zip
legacy
omit dc2ce958aab Read a claimed file's Dag source through its importer
omit 1f60b56820f Queue and track files that a Task SDK importer claims
omit 99205148612 Parse files that a Task SDK importer claims
omit dd58cd30c38 Materialise each configured importer spec once during
discovery
omit 1ea433b4ff5 Discover a bundle's Dag definitions through the importer
registry
omit bc31a7a4970 Reject empty subprocess coordinator artifact roots
omit 002e75919aa Reject concurrent subprocess coordinator execution
consistently
omit ec7447aa0c8 Make task bundle access dependencies explicit
omit ef75e1e84f0 Preserve custom subprocess coordinator compatibility
omit db652eb3b8e Read the Dag bundle configuration through a single entry
point
omit 5beb821edf1 Resolve coordinator Dag bundles without importing the task
runner
omit 869d197fc8b Make JavaCoordinator main_class optional in co-located mode
omit 7203ab14efa Harden subprocess coordinator artifact bundle resolution
omit fb3929e9ffd Document dag_bundle_name option for language-SDK
coordinators
omit a5b1c8731f8 Add dag_bundle_name option to subprocess coordinators
add 22ebb0943f7 [main] Upgrade important CI environment (#73629)
add 3107af07cd1 Use POSIX-compliant tail to extract token line in EKS
command (#73690)
add 23c6e77067f Fix provider.yaml doc URL check with several suspended
providers (#73782)
add f2fae8c737a Fix flaky test_trigger_log in triggerer job tests (#72919)
add a55f0a1af04 Add GitSync sidecar upgrade warning to Helm chart docs
(#73814)
add 5bc738efafe Add tests for the Google Cloud Tasks links (#72846)
add ebe3bddae79 Keep README stable Airflow version pins in sync with
releases (#73786)
add 46823e47aee Correct the meaning of _arg_bindings in the serialization
schema (#73644)
add f49bce025a8 Optimize UI and Playwright dependency caching (#73183)
add acaa1cba930 Republish repaired prek caches and keep unrelated host
caches (#73182)
add dff97f0908a Added Louisa AI to the list of companies using Apache
Airflow (#73665)
add 28e7c89c363 fix k8s download_schemas_for_version for hpa (#73821)
add 23bcbe6c99a Regenerate provider requirement tables in breeze ci
upgrade (#73802)
add 2827a13dc4f Check ui-field-behaviour drift between hooks and
provider.yaml (#73801)
add 30d0d7bac4a Adjust warnings, import errors, and Dag source models for
sdk/importers (#72474)
add f242c8e0513 Skip CI duration alerts when the newest run is stale
(#73831)
add 3634bc89d04 Discover a bundle's Dag definitions through the importer
registry (#73728)
add 7eac9ac7c1b Add connections and agent-security entries to the
common.ai sidebar (#73837)
add 58af2f521bb Close the ssh askpass helper in GitHook before ssh
executes it (#73427)
add b86fa49a15d Optimize CI disk cleanup (#73184)
add 5fbe8dbb433 Clarify DateTimeSensorAsync changelog entry (#73833)
add 873bd644b51 Let a person approve an agent's tool calls before they run
(#73586)
add 66607ac5483 AIP-82: Add KinesisMessageQueueProvider (#73509)
add b357ae24709 Bump the auth-ui-package-updates group across 1 directory
with 8 updates (#73740)
add 455d9581367 UI: Complete Polish translations and unify slot/deferred
wording (#72623)
add 5428b3b995b Secrets backends order updated (#70681)
add 232255e7fa7 Go SDK: add airflow.Dag and dag.Task, the native Dag
constructors (#73767)
add 68693340f67 Add dag_bundle_name option to subprocess coordinators
add b87d2907a24 Document dag_bundle_name option for language-SDK
coordinators
add da37c21686b Harden subprocess coordinator artifact bundle resolution
add 5fa4098e409 Make JavaCoordinator main_class optional in co-located mode
add ed24a7cdcdf Resolve coordinator Dag bundles without importing the task
runner
add 14d5448c276 Read the Dag bundle configuration through a single entry
point
add a59132f973f Preserve custom subprocess coordinator compatibility
add ce617c8474b Make task bundle access dependencies explicit
add 0aec44e8889 Reject concurrent subprocess coordinator execution
consistently
add a481344223b Reject empty subprocess coordinator artifact roots
add 12a840b2d22 Parse files that a Task SDK importer claims
add cf95650ca17 Queue and track files that a Task SDK importer claims
add b051d590807 Read a claimed file's Dag source through its importer
add a7a81df9f58 Claim Dag files by extension and keep .py, .pyc and .zip
legacy
add 1b150318965 Keep claimed files queued when their importer cannot list
them
add 9ff9d0a6da5 Tidy the importer routing helpers and their tests
add 40bf9b36a7c Harden claimed-file discovery errors and archive importers
add 396263ed7f0 Find the coordinators that parse a Dag bundle
add ae498dcc9ec Keep a stub task's preset arg bindings when serializing it
add 7fb00390d88 Rebuild a Lang-SDK Dag as an airflow.sdk.DAG
add 23903ec049f Parse a Dag file with its coordinator's runtime
add ecc19efc4b6 Parse a coordinator's Dag file outside the Dag processor
add 1fe5309883a Import native Dags through their coordinator
add 2eee8648c55 Parse coordinator-claimed Dag files with their runtime
add c39a58eaa2f Kill the Lang-SDK runtime when its parse is interrupted
add a0dfcd23390 Signal a Lang-SDK runtime's whole process tree
add aeb390767e1 Reject a native Dag whose task graph has a cycle
add cd7689b2fdf Keep a native Dag's timezone when rebuilding it
add 63af72aaa72 Name the missing task when a native Dag's edge points
nowhere
add c7007e70d66 Report an invalid frame from a Lang-SDK runtime as an
import error
add f743cc4c9e4 Run the Lang-SDK runtime in its own process group
add 2db7f8f64ad Keep the first parse result from a Lang-SDK runtime
add 0359eab0e99 Start a Lang-SDK parse runtime with default signal handling
add ac74a126a27 Store only native Dags that a Dag bag can rebuild
add 75761970850 Keep a bundle's importers when the coordinator config is
broken
add 8e8c9820d22 Start a coordinator's runtime from a fresh interpreter on
macOS
add 8159ac365b0 Read JAR manifests per the JAR specification
add b5b72f5ae4d Build the JVM command that parses a Dag JAR
add 52b381cc1c2 Embed the Dag source in Java bundle JARs
add ca7dc2a707d Parse native Java Dags with their coordinator
add 12fa0f239a5 Document native Java Dag parsing
add 6ff2da485c9 Treat an unreadable JAR like one without a manifest
add 178e4681803 Test the Java Dag source placeholder for a JAR without a
manifest
add a80facc76d2 Name the JAR manifest attributes the Java coordinator reads
add 91e43c013ae Parse native TypeScript Dags with the Node coordinator
add bf7586d8ff9 Document parsing native TypeScript Dags
add 3ccfffcd4a7 Share the bundle section mutation helper between Node tests
add dd99f04d8c8 TS SDK: default a task id to the handler's name
add cbdbf025ee7 TS SDK: draw order-only edges with before and after
add 1b17839258f TS SDK: add task groups to the native Dag surface
add 828b7963823 TS SDK: reject a Dag whose order-only edges form a cycle
add 54d07512b98 TS SDK: serialize a native Dag to Dag JSON
add b249b3fc2ec TS SDK: answer the Dag-parsing request from bundle.serve
add d8214901e71 TS SDK: add conditional branching with dag.if
add 65bd2e1d6de TS SDK: add multi-way branching with dag.switch
add 36db4e267d5 TS SDK: add triggerDagRun to the native Dag surface
add 4a7d46e61e0 Complete the license header of the TS SDK conformance check
add f7e6dea0a02 Java SDK: Describe native Java Dags without reference to
Python
add 2ae17e0f9ff Java SDK: Generate Dag and task configuration from the
serialization schema
add 715775b0e6f Java SDK: Resolve a task's arguments from the Dag's own
wiring
add 62271005134 Java SDK: Declare a Dag's task graph with @Builder.Deps
add daaccdf3532 Java SDK: Serialize native Dags to DagSerialization v3 on
parse requests
add 1200a1a23f9 TS SDK: cover native TypeScript Dags end to end
add 3d900fd9fcc Parse the native TypeScript Dag in the compose e2e
add 6e7cd30b929 Check the native TypeScript Dag's source and triggered run
add 58bb9345988 Keep the containerized Java build's Maven repository on
the host
add b70d2a0e8d6 Add a native Java Dag to the compose e2e
add 5869acb7432 Run the Lang-SDK e2e jobs when native Dag parsing changes
add 9f55d94af6a Share the Dag run lookup and the native Java queue name
add f5e50472249 Build the k8s TypeScript artifacts only for the native Dag
test
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (9b90cd28cbe)
\
N -- N -- N refs/heads/jason/lang-sdk-e2e/08-native-e2e
(f5e50472249)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/actions/install-prek/action.yml | 159 +-
.github/workflows/basic-tests.yml | 41 +-
.github/workflows/ui-e2e-tests.yml | 21 +-
.pre-commit-config.yaml | 4 +-
Dockerfile | 2 +-
Dockerfile.ci | 4 +-
INTHEWILD.md | 1 +
.../0007-taskflow-across-language-boundary.md | 3 +
.../adr/lang-sdk/0008-control-flow-constructs.md | 2 +-
.../0009-provider-operators-as-generated-dsl.md | 4 +-
airflow-core/docs/best-practices.rst | 2 +-
airflow-core/docs/migrations-ref.rst | 6 +-
.../security/secrets/secrets-backend/index.rst | 23 +-
airflow-core/newsfragments/70681.significant.rst | 7 +
airflow-core/pyproject.toml | 4 +-
airflow-core/src/airflow/api/common/delete_dag.py | 7 +-
.../auth/managers/simple/ui/package.json | 16 +-
.../auth/managers/simple/ui/pnpm-lock.yaml | 356 +-
.../api_fastapi/core_api/datamodels/dag_sources.py | 1 +
.../api_fastapi/core_api/datamodels/dag_warning.py | 4 +-
.../core_api/datamodels/import_error.py | 1 +
.../api_fastapi/core_api/openapi/_private_ui.yaml | 98 +
.../core_api/openapi/v2-rest-api-generated.yaml | 25 +-
.../core_api/routes/public/dag_sources.py | 1 +
.../core_api/routes/public/dag_warning.py | 6 +-
.../core_api/routes/public/import_error.py | 2 +
.../api_fastapi/core_api/routes/ui/config.py | 45 +-
.../src/airflow/config_templates/config.yml | 34 +
airflow-core/src/airflow/configuration.py | 7 +-
.../src/airflow/dag_processing/collection.py | 2 +
..._3_4_0_add_source_reference_to_import_error.py} | 24 +-
...g.py => 0140_3_4_0_add_language_to_dag_code.py} | 26 +-
airflow-core/src/airflow/models/dagcode.py | 3 +
airflow-core/src/airflow/models/dagwarning.py | 31 +-
airflow-core/src/airflow/models/errors.py | 12 +-
airflow-core/src/airflow/secrets/__init__.py | 4 +-
airflow-core/src/airflow/secrets/base_secrets.py | 5 +-
airflow-core/src/airflow/serialization/schema.json | 2 +-
.../src/airflow/ui/openapi-gen/queries/common.ts | 10 +-
.../ui/openapi-gen/queries/ensureQueryData.ts | 16 +-
.../src/airflow/ui/openapi-gen/queries/prefetch.ts | 16 +-
.../src/airflow/ui/openapi-gen/queries/queries.ts | 16 +-
.../src/airflow/ui/openapi-gen/queries/suspense.ts | 16 +-
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 36 +-
.../ui/openapi-gen/requests/services.gen.ts | 26 +-
.../airflow/ui/openapi-gen/requests/types.gen.ts | 37 +-
.../airflow/ui/public/i18n/locales/en/admin.json | 2 +
.../airflow/ui/public/i18n/locales/pl/admin.json | 30 +-
.../airflow/ui/public/i18n/locales/pl/assets.json | 21 +-
.../airflow/ui/public/i18n/locales/pl/browse.json | 38 +
.../airflow/ui/public/i18n/locales/pl/common.json | 103 +-
.../ui/public/i18n/locales/pl/components.json | 29 +-
.../src/airflow/ui/public/i18n/locales/pl/dag.json | 13 +-
.../airflow/ui/public/i18n/locales/pl/dags.json | 30 +-
.../ui/public/i18n/locales/pl/dashboard.json | 6 +-
.../airflow/ui/public/i18n/locales/pl/hitl.json | 2 +-
.../src/airflow/ui/src/components/StatsCard.tsx | 2 +-
.../src/airflow/ui/src/pages/Dag/Code/Code.tsx | 10 +-
.../BackendsOrderCard.tsx} | 33 +-
.../ui/src/pages/Variables/BackendsOrderModal.tsx | 64 +
.../airflow/ui/src/pages/Variables/Variables.tsx | 2 +
airflow-core/src/airflow/utils/db.py | 2 +-
.../tests/unit/always/test_project_structure.py | 1 -
airflow-core/tests/unit/always/test_secrets.py | 121 +
.../tests/unit/api/common/test_delete_dag.py | 49 +
.../core_api/routes/public/test_dag_sources.py | 4 +
.../core_api/routes/public/test_dag_warning.py | 19 +
.../core_api/routes/public/test_import_error.py | 76 +-
.../api_fastapi/core_api/routes/ui/test_config.py | 62 +
airflow-core/tests/unit/jobs/test_triggerer_job.py | 22 +-
airflow-core/tests/unit/models/test_dagcode.py | 8 +
airflow-core/tests/unit/models/test_dagwarning.py | 38 +-
airflow-core/tests/unit/models/test_errors.py | 86 +
airflow-ctl-tests/pyproject.toml | 4 +-
airflow-ctl/pyproject.toml | 4 +-
.../airflowctl/api/datamodels/auth_generated.py | 2 +-
.../src/airflowctl/api/datamodels/generated.py | 12 +-
.../tests/airflow_ctl/api/test_operations.py | 1 +
airflow-e2e-tests/pyproject.toml | 6 +-
chart/docs/manage-dag-files.rst | 4 +
chart/pyproject.toml | 4 +-
clients/python/pyproject.toml | 4 +-
dev/README_RELEASE_AIRFLOW.md | 6 +-
dev/breeze/doc/08_ci_tasks.rst | 4 +-
dev/breeze/doc/ci/02_images.md | 4 +-
dev/breeze/pyproject.toml | 10 +-
.../src/airflow_breeze/commands/ci_commands.py | 9 +
.../airflow_breeze/commands/kubernetes_commands.py | 2 +-
.../commands/release_management_commands.py | 6 +-
dev/breeze/src/airflow_breeze/global_constants.py | 2 +-
.../templates/pyproject_TEMPLATE.toml.jinja2 | 2 +-
dev/breeze/tests/test_ci_upgrade.py | 16 +-
dev/breeze/uv.lock | 160 +-
dev/mypy/pyproject.toml | 4 +-
dev/pyproject.toml | 4 +-
dev/registry/pyproject.toml | 4 +-
devel-common/pyproject.toml | 4 +-
docker-stack-docs/pyproject.toml | 4 +-
docker-tests/pyproject.toml | 4 +-
go-sdk/adr/0008-native-dag-interface.md | 18 +-
go-sdk/airflow/bundle.go | 37 +-
go-sdk/airflow/bundle_test.go | 44 +-
go-sdk/airflow/dag.go | 234 +
go-sdk/airflow/dag_test.go | 461 ++
go-sdk/airflow/serve.go | 2 +-
go-sdk/{pkg/logging/attrs.go => airflow/spec.go} | 17 +-
go-sdk/airflow/task_handler.go | 23 +-
.../foreignitem/main.go => task_option.go} | 25 +-
.../{foreignitem => foreigntaskoption}/main.go | 14 +-
java-sdk/sdk/schema/dag-schema.json | 2 +-
kubernetes-tests/pyproject.toml | 4 +-
.../test_kubernetes_pod_operator.py | 2 +-
performance/pyproject.toml | 2 +-
providers-summary-docs/pyproject.toml | 4 +-
providers/airbyte/pyproject.toml | 2 +-
providers/akeyless/pyproject.toml | 2 +-
providers/alibaba/pyproject.toml | 2 +-
providers/amazon/README.rst | 4 +-
providers/amazon/docs/index.rst | 4 +-
providers/amazon/docs/message-queues/index.rst | 55 +
providers/amazon/provider.yaml | 1 +
providers/amazon/pyproject.toml | 6 +-
.../src/airflow/providers/amazon/aws/hooks/eks.py | 6 +-
.../amazon/aws/queues/{sqs.py => kinesis.py} | 32 +-
.../airflow/providers/amazon/get_provider_info.py | 5 +-
.../amazon/aws/example_kinesis_message_queue.py | 98 +
.../amazon/tests/unit/amazon/aws/hooks/test_eks.py | 40 +-
.../tests/unit/amazon/aws/queues/test_kinesis.py | 185 +
providers/anthropic/pyproject.toml | 2 +-
providers/apache/arrow/pyproject.toml | 2 +-
providers/apache/beam/pyproject.toml | 2 +-
providers/apache/cassandra/pyproject.toml | 2 +-
providers/apache/drill/pyproject.toml | 2 +-
providers/apache/druid/pyproject.toml | 2 +-
providers/apache/flink/pyproject.toml | 2 +-
providers/apache/hdfs/pyproject.toml | 2 +-
providers/apache/hive/pyproject.toml | 2 +-
providers/apache/iceberg/pyproject.toml | 2 +-
providers/apache/impala/pyproject.toml | 2 +-
providers/apache/kafka/pyproject.toml | 2 +-
providers/apache/kylin/pyproject.toml | 2 +-
providers/apache/livy/pyproject.toml | 2 +-
providers/apache/pig/pyproject.toml | 2 +-
providers/apache/pinot/pyproject.toml | 2 +-
providers/apache/spark/pyproject.toml | 2 +-
providers/apache/tinkerpop/pyproject.toml | 2 +-
providers/apprise/pyproject.toml | 2 +-
providers/arangodb/pyproject.toml | 2 +-
providers/asana/pyproject.toml | 2 +-
providers/atlassian/jira/pyproject.toml | 2 +-
providers/celery/pyproject.toml | 2 +-
providers/clickhousedb/pyproject.toml | 2 +-
providers/cloudant/pyproject.toml | 2 +-
providers/cncf/kubernetes/pyproject.toml | 2 +-
.../cncf/kubernetes/utils/xcom_sidecar.py | 2 +-
.../system/cncf/kubernetes/example_kubernetes.py | 2 +-
.../cncf/kubernetes/example_kubernetes_async.py | 2 +-
.../unit/cncf/kubernetes/operators/test_pod.py | 6 +-
providers/cohere/pyproject.toml | 2 +-
providers/common/ai/docs/agent_security.rst | 22 +-
providers/common/ai/docs/connections/index.rst | 62 +
providers/common/ai/docs/examples.rst | 3 +
providers/common/ai/docs/features.rst | 9 +-
providers/common/ai/docs/index.rst | 4 +-
providers/common/ai/docs/observability.rst | 4 +-
providers/common/ai/docs/operations.rst | 1 -
providers/common/ai/docs/operators/agent.rst | 4 +-
providers/common/ai/docs/operators/index.rst | 4 +-
providers/common/ai/docs/tool_approval.rst | 144 +
providers/common/ai/pyproject.toml | 4 +-
.../ai/example_dags/example_agent_tool_approval.py | 58 +
.../src/airflow/providers/common/ai/exceptions.py | 32 +-
.../airflow/providers/common/ai/mixins/approval.py | 30 +
.../airflow/providers/common/ai/operators/agent.py | 334 +-
.../airflow/providers/common/ai/operators/llm.py | 24 +-
.../providers/common/ai/toolsets/logging.py | 6 +
.../tests/unit/common/ai/operators/test_agent.py | 6 +-
.../ai/operators/test_agent_tool_approval.py | 556 +++
.../tests/unit/common/ai/toolsets/test_logging.py | 18 +
providers/common/compat/pyproject.toml | 2 +-
providers/common/dataquality/pyproject.toml | 2 +-
providers/common/io/pyproject.toml | 2 +-
providers/common/messaging/pyproject.toml | 2 +-
providers/common/sql/pyproject.toml | 2 +-
providers/databricks/docs/index.rst | 2 +-
providers/databricks/pyproject.toml | 4 +-
providers/datadog/pyproject.toml | 2 +-
providers/dbt/cloud/pyproject.toml | 2 +-
providers/dingding/pyproject.toml | 2 +-
providers/discord/pyproject.toml | 2 +-
providers/docker/pyproject.toml | 2 +-
providers/duckdb/pyproject.toml | 2 +-
providers/edge3/pyproject.toml | 4 +-
providers/elasticsearch/pyproject.toml | 2 +-
providers/exasol/pyproject.toml | 2 +-
providers/fab/pyproject.toml | 4 +-
providers/facebook/pyproject.toml | 2 +-
providers/ftp/pyproject.toml | 2 +-
providers/git/pyproject.toml | 2 +-
.../git/src/airflow/providers/git/hooks/git.py | 31 +-
providers/git/tests/unit/git/hooks/test_git.py | 19 +
providers/github/pyproject.toml | 2 +-
providers/google/README.rst | 80 +-
providers/google/docs/index.rst | 80 +-
providers/google/pyproject.toml | 82 +-
.../unit/google/cloud/links/test_cloud_tasks.py | 155 +
providers/grpc/pyproject.toml | 2 +-
providers/hashicorp/docs/index.rst | 2 +-
providers/hashicorp/pyproject.toml | 4 +-
providers/http/pyproject.toml | 2 +-
providers/ibm/db2/pyproject.toml | 2 +-
providers/ibm/mq/pyproject.toml | 2 +-
providers/imap/pyproject.toml | 2 +-
providers/influxdb/pyproject.toml | 2 +-
providers/informatica/pyproject.toml | 2 +-
providers/jdbc/pyproject.toml | 2 +-
providers/jenkins/pyproject.toml | 2 +-
providers/keycloak/pyproject.toml | 2 +-
providers/microsoft/azure/README.rst | 30 +-
providers/microsoft/azure/docs/index.rst | 30 +-
providers/microsoft/azure/pyproject.toml | 34 +-
providers/microsoft/mssql/pyproject.toml | 2 +-
providers/microsoft/psrp/pyproject.toml | 2 +-
providers/microsoft/winrm/pyproject.toml | 2 +-
providers/mongo/pyproject.toml | 2 +-
providers/mysql/pyproject.toml | 2 +-
providers/neo4j/pyproject.toml | 2 +-
providers/odbc/pyproject.toml | 2 +-
providers/openai/pyproject.toml | 2 +-
providers/openfaas/pyproject.toml | 2 +-
providers/openlineage/pyproject.toml | 2 +-
providers/opensearch/pyproject.toml | 2 +-
providers/opsgenie/pyproject.toml | 2 +-
providers/oracle/pyproject.toml | 2 +-
providers/pagerduty/pyproject.toml | 2 +-
providers/papermill/pyproject.toml | 2 +-
providers/pgvector/pyproject.toml | 2 +-
providers/pinecone/pyproject.toml | 2 +-
providers/postgres/pyproject.toml | 2 +-
providers/presto/pyproject.toml | 2 +-
providers/qdrant/pyproject.toml | 2 +-
providers/redis/pyproject.toml | 2 +-
providers/salesforce/pyproject.toml | 2 +-
providers/samba/pyproject.toml | 2 +-
providers/segment/pyproject.toml | 2 +-
providers/sendgrid/pyproject.toml | 2 +-
providers/sftp/pyproject.toml | 2 +-
providers/singularity/pyproject.toml | 2 +-
providers/slack/pyproject.toml | 2 +-
providers/smtp/pyproject.toml | 2 +-
providers/snowflake/pyproject.toml | 2 +-
providers/sqlite/pyproject.toml | 2 +-
providers/ssh/pyproject.toml | 2 +-
providers/standard/docs/changelog.rst | 2 +-
providers/standard/pyproject.toml | 2 +-
providers/tableau/pyproject.toml | 2 +-
providers/telegram/pyproject.toml | 2 +-
providers/teradata/pyproject.toml | 2 +-
providers/trino/pyproject.toml | 2 +-
providers/vertica/pyproject.toml | 2 +-
providers/vespa/pyproject.toml | 2 +-
providers/weaviate/pyproject.toml | 2 +-
providers/yandex/pyproject.toml | 2 +-
providers/ydb/pyproject.toml | 2 +-
providers/zendesk/pyproject.toml | 2 +-
pyproject.toml | 7 +-
scripts/ci/analyze_ci_job_durations.py | 24 +-
scripts/ci/prek/check_extra_packages_ref.py | 2 +-
scripts/ci/prek/check_imports_in_providers.py | 2 +-
scripts/ci/prek/check_provider_conn_fields.py | 123 +-
scripts/ci/prek/download_k8s_schemas.py | 7 +-
scripts/ci/prek/ruff_format.py | 2 +-
scripts/ci/prek/supported_versions.py | 34 +
scripts/ci/prek_cache_markers.py | 60 +
.../in_container/provider_suspension.py | 40 +-
.../in_container/run_provider_yaml_files_check.py | 89 +-
scripts/pyproject.toml | 4 +-
.../test_check_ui_field_behaviour_matches_hook.py | 130 +
scripts/tests/ci/prek/test_supported_versions.py | 71 +
scripts/tests/ci/test_analyze_ci_job_durations.py | 46 +
scripts/tests/ci/test_prek_cache_optimizations.py | 439 ++
.../tests/in_container/test_provider_suspension.py | 64 +
.../_shared => scripts/tests/tools}/__init__.py | 0
scripts/tests/tools/test_free_up_disk_space.py | 98 +
scripts/tools/free_up_disk_space.sh | 25 +-
scripts/tools/setup_breeze | 2 +-
shared/configuration/pyproject.toml | 4 +-
.../configuration/secrets_backends.py | 88 +
shared/dagnode/pyproject.toml | 4 +-
shared/listeners/pyproject.toml | 4 +-
shared/logging/pyproject.toml | 4 +-
shared/module_loading/pyproject.toml | 4 +-
shared/observability/pyproject.toml | 4 +-
shared/plugins_manager/pyproject.toml | 4 +-
shared/providers_discovery/pyproject.toml | 4 +-
shared/secrets_backend/pyproject.toml | 4 +-
shared/secrets_masker/pyproject.toml | 4 +-
shared/serialization/pyproject.toml | 4 +-
shared/state/pyproject.toml | 4 +-
shared/template_rendering/pyproject.toml | 4 +-
shared/timezones/pyproject.toml | 4 +-
task-sdk-integration-tests/pyproject.toml | 4 +-
task-sdk/pyproject.toml | 4 +-
.../src/airflow/sdk/api/datamodels/_generated.py | 2 +-
task-sdk/src/airflow/sdk/configuration.py | 10 +-
.../airflow/sdk/execution_time/secrets/__init__.py | 9 +-
.../src/airflow/sdk/execution_time/supervisor.py | 3 +-
.../tests/task_sdk/execution_time/test_context.py | 13 +-
.../tests/task_sdk/execution_time/test_secrets.py | 3 +-
ts-sdk/schema/dag-schema.json | 2 +-
uv.lock | 4742 ++++++++++----------
311 files changed, 8577 insertions(+), 3380 deletions(-)
create mode 100644 airflow-core/newsfragments/70681.significant.rst
copy
airflow-core/src/airflow/migrations/versions/{0132_3_4_0_add_team_name_to_log.py
=> 0139_3_4_0_add_source_reference_to_import_error.py} (63%)
copy
airflow-core/src/airflow/migrations/versions/{0132_3_4_0_add_team_name_to_log.py
=> 0140_3_4_0_add_language_to_dag_code.py} (63%)
copy
airflow-core/src/airflow/ui/src/pages/{Asset/AssetStateStore/AddAssetStateStoreButton.tsx
=> Variables/BackendsOrderCard.tsx} (58%)
create mode 100644
airflow-core/src/airflow/ui/src/pages/Variables/BackendsOrderModal.tsx
create mode 100644 airflow-core/tests/unit/models/test_errors.py
create mode 100644 go-sdk/airflow/dag.go
create mode 100644 go-sdk/airflow/dag_test.go
copy go-sdk/{pkg/logging/attrs.go => airflow/spec.go} (68%)
copy go-sdk/airflow/{testdata/foreignitem/main.go => task_option.go} (58%)
copy go-sdk/airflow/testdata/{foreignitem => foreigntaskoption}/main.go (68%)
copy providers/amazon/src/airflow/providers/amazon/aws/queues/{sqs.py =>
kinesis.py} (63%)
create mode 100644
providers/amazon/tests/system/amazon/aws/example_kinesis_message_queue.py
create mode 100644
providers/amazon/tests/unit/amazon/aws/queues/test_kinesis.py
create mode 100644 providers/common/ai/docs/connections/index.rst
create mode 100644 providers/common/ai/docs/tool_approval.rst
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/example_dags/example_agent_tool_approval.py
create mode 100644
providers/common/ai/tests/unit/common/ai/operators/test_agent_tool_approval.py
create mode 100644
providers/google/tests/unit/google/cloud/links/test_cloud_tasks.py
create mode 100644 scripts/ci/prek_cache_markers.py
copy dev/breeze/src/airflow_breeze/utils/click_utils.py =>
scripts/in_container/provider_suspension.py (50%)
create mode 100644
scripts/tests/ci/prek/test_check_ui_field_behaviour_matches_hook.py
create mode 100644 scripts/tests/ci/prek/test_supported_versions.py
create mode 100644 scripts/tests/ci/test_prek_cache_optimizations.py
create mode 100644 scripts/tests/in_container/test_provider_suspension.py
copy {airflow-core/src/airflow/_shared => scripts/tests/tools}/__init__.py
(100%)
create mode 100644 scripts/tests/tools/test_free_up_disk_space.py
create mode 100644
shared/configuration/src/airflow_shared/configuration/secrets_backends.py
