This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch
jason/lang-sdk-e2e/02b-shared-dag-cycle-detection
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit 5c2018283cf Share Dag cycle detection between the Task SDK and core
add e9c552cc827 Add a vendor-neutral managed-agent hook contract to Common
AI (#73532)
add a33fe27cace Add async client to `AnthropicHook` (#73967)
add e6a9de5d832 Go SDK: bring the coordinator-protocol models back to the
supervisor schema (#73963)
add 27cd88ca74c Cache repeated agent prompts by default in AgentOperator
(#73994)
add 529f429400a Go SDK: reject a task handler and a Dag that share a
dag_id (#73876)
add bc9ca5b1227 Correctly shutdown async sessions on exit. (#73838)
add f1b36db6093 Back off between GitDagBundle bare clone attempts so a
fresh GitHub App token can propagate (#73878)
add 6b847411de1 Stop rebuilding the FAB app on every users and roles API
call (#73104)
add 56817294cdd Go SDK: add airflow.TriggerDagRun, a task that triggers a
Dag run (#74003)
add b8475094703 Fix go-sdk generated drift (#74028)
add 8f3e8466c67 Support Airflow 2.11 in the Common AI provider (#73991)
add f0cde671827 Use "Airflow versions" in user-facing docs, not "cores"
(#74045)
add 0bb09cb6ba5 Close the providers testing issue right after announcing
the release (#73890)
add 43377466a06 Release the DuckDB provider as 0.2.0 (#73888)
add 53e43f29130 Speed up CI image restores with zstd-compressed stashes
(#73997)
add 541001b9ee1 Share Dag cycle detection between the Task SDK and core
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (5c2018283cf)
\
N -- N -- N
refs/heads/jason/lang-sdk-e2e/02b-shared-dag-cycle-detection (541001b9ee1)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/ci-image-build.yml | 8 +
.pre-commit-config.yaml | 12 +-
airflow-core/newsfragments/68496.significant.rst | 4 +-
airflow-core/src/airflow/api_fastapi/app.py | 2 +
.../src/airflow/api_fastapi/execution_api/app.py | 2 +
airflow-core/src/airflow/settings.py | 14 +-
.../api_fastapi/auth/managers/simple/conftest.py | 3 +-
airflow-core/tests/unit/api_fastapi/conftest.py | 27 +-
.../core_api/routes/public/test_auth.py | 28 +-
.../core_api/routes/public/test_backfills.py | 20 +-
.../core_api/routes/public/test_connections.py | 9 +-
.../core_api/routes/public/test_dag_bundles.py | 43 ++-
.../core_api/routes/public/test_dag_parsing.py | 18 +-
.../core_api/routes/public/test_dag_run.py | 78 ++--
.../core_api/routes/public/test_task_instances.py | 111 +++---
.../unit/api_fastapi/execution_api/test_app.py | 84 ++++-
airflow-core/tests/unit/api_fastapi/test_app.py | 108 +++++-
airflow-core/tests/unit/core/test_settings.py | 120 ++++++
airflow-core/tests/unit/state/test_metastore.py | 9 +
airflow-core/tests/unit/utils/test_session.py | 18 +-
.../airflow_e2e_tests/openlineage_tests/README.md | 2 +-
.../dags_extra/e2e_versioned_bundle.py | 4 +-
dev/README_RELEASE_PROVIDERS.md | 25 +-
.../airflow_breeze/commands/testing_commands.py | 2 +-
dev/breeze/src/airflow_breeze/global_constants.py | 2 +-
dev/breeze/tests/test_selective_checks.py | 16 +-
docs/spelling_wordlist.txt | 2 +
go-sdk/README.md | 7 +-
go-sdk/airflow/bundle.go | 46 ++-
go-sdk/airflow/bundle_test.go | 111 ++++++
go-sdk/airflow/dag.go | 79 +++-
go-sdk/airflow/dag_test.go | 6 +
go-sdk/airflow/inputs.go | 6 +
go-sdk/airflow/spec.gen.go | 3 +-
go-sdk/airflow/trigger_dag_run.go | 177 +++++++++
go-sdk/airflow/trigger_dag_run_test.go | 272 ++++++++++++++
go-sdk/internal/genspec/authoring.go | 2 +-
.../pkg/execution/genmodels/discriminators.gen.go | 4 +
go-sdk/pkg/execution/genmodels/models.gen.go | 171 +++++----
providers/amazon/docs/index.rst | 2 +
providers/amazon/docs/operators/bedrock.rst | 40 ++
providers/amazon/pyproject.toml | 4 +
.../airflow/providers/amazon/aws/hooks/bedrock.py | 229 +++++++++++-
.../tests/unit/amazon/aws/hooks/test_bedrock.py | 280 ++++++++++++++
providers/anthropic/pyproject.toml | 2 +-
.../airflow/providers/anthropic/hooks/anthropic.py | 99 ++++-
.../tests/unit/anthropic/hooks/test_anthropic.py | 141 ++++++-
providers/common/ai/README.rst | 3 +-
providers/common/ai/docs/approval_gates.rst | 2 +-
providers/common/ai/docs/index.rst | 5 +-
providers/common/ai/docs/installation.rst | 33 +-
providers/common/ai/docs/observability.rst | 3 +
providers/common/ai/docs/operators/agent.rst | 98 ++++-
providers/common/ai/docs/operators/llm_batch.rst | 4 +-
providers/common/ai/docs/operators/llm_branch.rst | 2 +-
providers/common/ai/docs/quickstart.rst | 3 +-
providers/common/ai/docs/self_hosted_models.rst | 3 +-
providers/common/ai/docs/toolsets/index.rst | 35 +-
.../common/ai/docs/toolsets/managed_agent.rst | 337 +++++++++--------
providers/common/ai/docs/troubleshooting.rst | 8 +-
providers/common/ai/pyproject.toml | 7 +-
.../ai/src/airflow/providers/common/ai/__init__.py | 4 +-
.../airflow/providers/common/ai/batch/anthropic.py | 5 +-
.../airflow/providers/common/ai/batch/openai.py | 5 +-
.../airflow/providers/common/ai/batch/results.py | 5 +-
.../providers/common/ai/decorators/agent.py | 2 +-
.../airflow/providers/common/ai/decorators/llm.py | 2 +-
.../providers/common/ai/decorators/llm_batch.py | 2 +-
.../providers/common/ai/decorators/llm_branch.py | 2 +-
.../common/ai/decorators/llm_file_analysis.py | 2 +-
.../common/ai/decorators/llm_schema_compare.py | 2 +-
.../providers/common/ai/decorators/llm_sql.py | 2 +-
.../providers/common/ai/durable/caching_model.py | 4 +-
.../providers/common/ai/durable/caching_toolset.py | 17 +-
.../providers/common/ai/durable/fingerprint.py | 12 +-
.../airflow/providers/common/ai/durable/storage.py | 7 +-
.../common/ai/durable/task_state_store.py | 4 +-
.../src/airflow/providers/common/ai/exceptions.py | 15 +-
.../providers/common/ai/hooks/pydantic_ai.py | 10 +
.../providers/common/ai/managed_agents/__init__.py | 49 +++
.../providers/common/ai/managed_agents/base.py | 251 +++++++++++++
.../providers/common/ai/managed_agents/failover.py | 145 ++++++++
.../airflow/providers/common/ai/mixins/approval.py | 6 +-
.../airflow/providers/common/ai/observability.py | 19 +-
.../airflow/providers/common/ai/operators/agent.py | 54 ++-
.../common/ai/operators/llamaindex_embedding.py | 2 +-
.../common/ai/operators/llamaindex_retrieval.py | 2 +-
.../airflow/providers/common/ai/operators/llm.py | 17 +-
.../providers/common/ai/toolsets/__init__.py | 4 +-
.../providers/common/ai/toolsets/managed_agent.py | 354 +++++++++---------
.../airflow/providers/common/ai/utils/logging.py | 18 +-
.../providers/common/ai/utils/prompt_cache.py | 140 +++++++
.../providers/common/ai/utils/task_logger.py | 67 ++++
.../providers/common/ai/utils/usage_budget.py | 11 +-
.../ai/tests/unit/common/ai/batch/test_dispatch.py | 2 +-
.../ai/tests/unit/common/ai/batch/test_results.py | 2 +-
.../ai/tests/unit/common/ai/batch/test_state.py | 2 +-
.../common/ai/tests/unit/common/ai/conftest.py | 11 +-
.../tests/unit/common/ai/decorators/test_agent.py | 5 +-
.../unit/common/ai/decorators/test_llm_batch.py | 3 +-
.../common/ai/decorators/test_llm_file_analysis.py | 5 +-
.../ai/tests/unit/common/ai/durable/test_base.py | 2 +-
.../unit/common/ai/durable/test_caching_toolset.py | 55 +++
.../unit/common/ai/durable/test_fingerprint.py | 20 +
.../unit/common/ai/durable/test_replay_cost.py | 2 +-
.../common/ai/durable/test_replay_verification.py | 2 +-
.../tests/unit/common/ai/durable/test_storage.py | 2 +-
.../common/ai/durable/test_task_state_store.py | 2 +-
.../tests/unit/common/ai/hooks/test_pydantic_ai.py | 15 +
.../unit/common/ai/managed_agents}/__init__.py | 0
.../unit/common/ai/managed_agents/test_base.py | 137 +++++++
.../unit/common/ai/managed_agents/test_failover.py | 205 +++++++++++
.../tests/unit/common/ai/mixins/test_approval.py | 6 +-
.../tests/unit/common/ai/operators/test_agent.py | 119 +++++-
.../common/ai/operators/test_document_loader.py | 12 +-
.../ai/operators/test_llamaindex_embedding.py | 2 +-
.../ai/operators/test_llamaindex_retrieval.py | 4 +-
.../ai/tests/unit/common/ai/operators/test_llm.py | 24 +-
.../unit/common/ai/operators/test_llm_batch.py | 3 +-
.../unit/common/ai/operators/test_llm_branch.py | 4 +-
.../common/ai/operators/test_llm_file_analysis.py | 11 +-
.../tests/unit/common/ai/operators/test_llm_sql.py | 4 +-
.../ai/tests/unit/common/ai/test_observability.py | 19 +
.../unit/common/ai/toolsets/test_managed_agent.py | 403 ++++++++-------------
.../unit/common/ai/toolsets/test_object_storage.py | 8 +-
.../ai/tests/unit/common/ai/utils/test_logging.py | 55 ++-
.../unit/common/ai/utils/test_prompt_cache.py | 209 +++++++++++
.../tests/unit/common/ai/utils/test_task_logger.py | 82 +++++
.../common/compat/_set_during_execution.py | 40 ++
.../src/airflow/providers/common/compat/sdk.py | 14 +-
.../common/compat/security/test_access_view.py | 8 +-
.../common/compat/test__set_during_execution.py} | 22 +-
.../compat/tests/unit/common/compat/test_sdk.py | 18 +-
providers/duckdb/README.rst | 6 +-
providers/duckdb/docs/changelog.rst | 9 +-
providers/duckdb/docs/index.rst | 6 +-
providers/duckdb/provider.yaml | 2 +-
providers/duckdb/pyproject.toml | 6 +-
.../src/airflow/providers/duckdb/__init__.py | 2 +-
providers/fab/pyproject.toml | 2 +-
.../fab/auth_manager/api_fastapi/routes/roles.py | 14 +-
.../fab/auth_manager/api_fastapi/routes/users.py | 12 +-
.../providers/fab/auth_manager/fab_auth_manager.py | 4 +-
.../unit/fab/auth_manager/api_fastapi/conftest.py | 21 ++
.../auth_manager/api_fastapi/routes/test_roles.py | 161 ++++----
.../auth_manager/api_fastapi/routes/test_users.py | 128 ++++---
.../git/src/airflow/providers/git/bundles/git.py | 31 +-
.../git/src/airflow/providers/git/hooks/git.py | 7 +-
providers/git/tests/unit/git/bundles/test_git.py | 44 +++
providers/git/tests/unit/git/hooks/test_git.py | 10 +
providers/google/docs/index.rst | 2 +
.../google/docs/operators/cloud/vertex_ai.rst | 24 ++
providers/google/pyproject.toml | 4 +
.../google/cloud/hooks/vertex_ai/agent_engine.py | 159 +++++++-
.../cloud/hooks/vertex_ai/test_agent_engine.py | 145 ++++++++
.../smtp/src/airflow/providers/smtp/hooks/smtp.py | 2 +-
.../airflow/providers/standard/operators/hitl.py | 6 +-
.../tests/unit/standard/operators/test_hitl.py | 4 +-
pyproject.toml | 4 +-
scripts/ci/prek/check_go_sdk_generated_drift.py | 187 ++++++++++
scripts/ci/prek/check_go_sdk_spec_drift.py | 140 -------
...ift.py => test_check_go_sdk_generated_drift.py} | 83 +++--
.../observability/metrics/metrics_template.yaml | 20 +-
task-sdk/src/airflow/sdk/serde/__init__.py | 7 +-
uv.lock | 20 +-
165 files changed, 5531 insertions(+), 1516 deletions(-)
create mode 100644 go-sdk/airflow/trigger_dag_run.go
create mode 100644 go-sdk/airflow/trigger_dag_run_test.go
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/managed_agents/__init__.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/managed_agents/base.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/managed_agents/failover.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/utils/prompt_cache.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/utils/task_logger.py
copy {airflow-core/src/airflow/_shared =>
providers/common/ai/tests/unit/common/ai/managed_agents}/__init__.py (100%)
create mode 100644
providers/common/ai/tests/unit/common/ai/managed_agents/test_base.py
create mode 100644
providers/common/ai/tests/unit/common/ai/managed_agents/test_failover.py
create mode 100644
providers/common/ai/tests/unit/common/ai/utils/test_prompt_cache.py
create mode 100644
providers/common/ai/tests/unit/common/ai/utils/test_task_logger.py
create mode 100644
providers/common/compat/src/airflow/providers/common/compat/_set_during_execution.py
copy providers/{amazon/tests/unit/amazon/aws/cli/test_definition.py =>
common/compat/tests/unit/common/compat/test__set_during_execution.py} (52%)
create mode 100755 scripts/ci/prek/check_go_sdk_generated_drift.py
delete mode 100755 scripts/ci/prek/check_go_sdk_spec_drift.py
rename scripts/tests/ci/prek/{test_check_go_sdk_spec_drift.py =>
test_check_go_sdk_generated_drift.py} (52%)
