This is an automated email from the ASF dual-hosted git repository.
ephraimbuddy pushed a change to branch dag-processor-activation
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard ff4ff454026 Allow independent priority parses to finish in either
order in recovery tests
discard 09c26e05e79 Preserve requested Dag processing work during bundle
failures
discard 8ad77a6194e Wait for fresh inventories before running priority parses
discard 573f161b139 Preserve Dag processor work across retries and reparses
discard 3d0e8d94a1c Run Dag processors without metadata database access
discard 3e78842a838 Back off inventory retries and report results the API
would refuse
discard 6602941a87d Publish Dag processor results through the Execution API
discard 0bf45ddc37d Ban new httpx imports in airflow-core
discard 65d6df3eadc Parse Execution API JSON errors that carry media type
parameters
discard a2f0fc7dc51 Adapt legacy Dag upgrade regression to bundle discovery
results
discard a6ace48856b Align Dag processor claim migrations with main
discard d2810a51cdf Test legacy Dag deactivation after bundle-name backfill
discard 6241aa62adf Fix Dag processor work retries and metadata cleanup
discard 0a3d2efb81f Retain Dag processor work until delivery is acknowledged
discard 2e380581283 Align API-mode Dag publication with DB-mode cleanup
discard 461dfc42d09 Allow Dag processors to publish metadata through the
Execution API
discard 562a3874cba Keep Dag parsing responsive during bundle refresh failures
discard 1462d071edd Make the bundle refresh regression test independent of
uptime
discard cdb757f9190 Avoid parsing stalls while a bundle refresh waits for
imports
discard 7848035d25c Keep Dag parsing results aligned with refreshed bundle
sources
add 3b6f02efd81 Migrate airflow-core to `httpx2` (#74244)
add c01a68cb1e3 Show secondary states of mapped tasks in Grid view (#74883)
add 40b5d3647df Revert "Make Java SDK context signal-aware (#74690)"
(#75421)
add 24a809533ba Fix lost transaction in SnowflakeHook when
`session_parameters` is set (#75425)
add f92b7c33851 Run full CI on release branches on a schedule instead of
every push (#75420)
add 5798f1f9ee0 Allow asking for human input from inside a Python task
(#75358)
add f1883a4829c Add missing Amazon provider unit tests (#73169)
add 0abd1616919 Run team tasks on the team executor when named by class or
path (#75362)
add 7ac92c98ce6 Add Databricks agent invocation operator and hook (#74386)
add a419633896d Java SDK: Rewrite the user docs around Dag definition
(#74701)
add 29bb1401c83 Show Dag version diff occurrence count on the path (#75433)
add 01907e047f5 fix stat timing (#72553)
add af76df38634 Allow TypeScript SDK releases from core release branches
(#75447)
add 40ade230912 Keep Dag parsing results aligned with refreshed bundle
sources
add 1aac6aff89c Avoid parsing stalls while a bundle refresh waits for
imports
add d389070a6e5 Make the bundle refresh regression test independent of
uptime
add 9bf2500e269 Keep Dag parsing responsive during bundle refresh failures
add a849e989ab8 Allow Dag processors to publish metadata through the
Execution API
add d070c9eb9b9 Align API-mode Dag publication with DB-mode cleanup
add f5a3d0d90d7 Retain Dag processor work until delivery is acknowledged
add c4b2eced2b3 Fix Dag processor work retries and metadata cleanup
add 82c3a97156b Test legacy Dag deactivation after bundle-name backfill
add 1f1e11878d2 Align Dag processor claim migrations with main
add 44a73caff8c Adapt legacy Dag upgrade regression to bundle discovery
results
add 5d8c67eec5c Parse Execution API JSON errors that carry media type
parameters
add 3a6ad327768 Publish Dag processor results through the Execution API
add f8cdf63e885 Back off inventory retries and report results the API
would refuse
add 42b036df6a8 Avoid database sessions when collecting API-mode Dag
results
add 7aea38e3eab Run Dag processors without metadata database access
add 0c2250c56ec Preserve Dag processor work across retries and reparses
add 2bb8a24b1d1 Wait for fresh inventories before running priority parses
add 743b19134ed Preserve requested Dag processing work during bundle
failures
add 828bc398938 Allow independent priority parses to finish in either
order in recovery tests
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ff4ff454026)
\
N -- N -- N refs/heads/dag-processor-activation (828bc398938)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.agents/skills/airflow-java-sdk/SKILL.md | 9 +-
.github/workflows/basic-tests.yml | 35 +-
.github/workflows/ci-amd.yml | 122 +-
.github/workflows/ci-arm.yml | 126 +-
.github/workflows/ci-image-checks.yml | 23 +-
.github/workflows/java-sdk-release-verify.yml | 2 +-
.github/workflows/refresh-image-registry-cache.yml | 11 +-
.github/workflows/release-branch-ci.yml | 161 ++
.github/workflows/ts-sdk-release.yml | 14 +-
.pre-commit-config.yaml | 14 +
.../language-sdks/java.rst | 1773 ++++++++------------
airflow-core/pyproject.toml | 25 +-
.../src/airflow/api/client/local_client.py | 6 +-
.../src/airflow/api_fastapi/auth/tokens.py | 4 +-
.../api_fastapi/execution_api/in_process.py | 1 -
airflow-core/src/airflow/dag_processing/manager.py | 38 +-
.../airflow/example_dags/example_dag_decorator.py | 4 +-
.../src/airflow/jobs/scheduler_job_runner.py | 48 +-
.../src/airflow/ui/public/i18n/locales/ca/dag.json | 1 -
.../src/airflow/ui/public/i18n/locales/de/dag.json | 1 -
.../src/airflow/ui/public/i18n/locales/el/dag.json | 1 -
.../src/airflow/ui/public/i18n/locales/en/dag.json | 5 +-
.../src/airflow/ui/public/i18n/locales/fr/dag.json | 1 -
.../src/airflow/ui/public/i18n/locales/he/dag.json | 1 -
.../src/airflow/ui/public/i18n/locales/hi/dag.json | 1 -
.../src/airflow/ui/public/i18n/locales/ko/dag.json | 1 -
.../src/airflow/ui/public/i18n/locales/pl/dag.json | 1 -
.../airflow/ui/public/i18n/locales/zh-TW/dag.json | 1 -
.../ui/src/layouts/Details/Grid/GridTI.test.tsx | 92 +-
.../airflow/ui/src/layouts/Details/Grid/GridTI.tsx | 46 +-
.../ui/src/layouts/Details/Grid/constants.ts | 2 +-
.../airflow/ui/src/pages/Dag/Versions/PathText.tsx | 33 +-
.../ui/src/pages/Dag/Versions/VersionDiff.test.tsx | 27 +-
.../ui/src/pages/Dag/Versions/VersionDiff.tsx | 19 +-
.../tests/unit/always/test_project_structure.py | 1 -
.../tests/unit/api_fastapi/auth/test_tokens.py | 10 +-
.../core_api/routes/public/test_dag_sources.py | 2 +-
.../execution_api/versions/head/test_hitl.py | 11 +-
.../tests/unit/dag_processing/test_manager.py | 29 +-
.../dag_processing/test_manager_api_results.py | 63 +
airflow-core/tests/unit/jobs/test_scheduler_job.py | 85 +-
.../org/apache/airflow/e2e/TestBundleBuilder.java | 33 +-
.../src/resources/dags/java_test_dags.py | 12 -
.../airflow_e2e_tests/e2e_test_utils/clients.py | 8 -
.../java_sdk_tests/test_java_sdk_dag.py | 56 -
contributing-docs/30_new_language_sdk.rst | 2 +-
dev/README_AIRFLOW3_DEV.md | 9 +-
dev/README_RELEASE_AIRFLOW.md | 59 +-
dev/README_RELEASE_AIRFLOWCTL.md | 12 +-
dev/breeze/doc/ci/04_selective_checks.md | 49 +-
dev/breeze/doc/ci/05_workflows.md | 66 +-
.../doc/images/output_ci_selective-check.svg | 54 +-
.../doc/images/output_ci_selective-check.txt | 2 +-
.../src/airflow_breeze/commands/ci_commands.py | 60 +-
.../airflow_breeze/commands/ci_commands_config.py | 2 +
.../src/airflow_breeze/utils/selective_checks.py | 19 +-
dev/breeze/tests/test_ci_commands.py | 114 ++
dev/breeze/tests/test_pr_info.py | 2 +-
dev/breeze/tests/test_selective_checks.py | 33 +
docs/spelling_wordlist.txt | 2 +
java-sdk/{README.md => DEVELOPMENT.md} | 139 +-
java-sdk/README.md | 774 +--------
java-sdk/example/README.md | 107 ++
java-sdk/scripts/ci/verify-source-release.sh | 2 +-
java-sdk/sdk/module.md | 81 +-
.../main/kotlin/org/apache/airflow/sdk/Context.kt | 50 -
.../main/kotlin/org/apache/airflow/sdk/Server.kt | 29 +-
.../apache/airflow/sdk/execution/Cancellation.kt | 134 --
.../org/apache/airflow/sdk/execution/Logger.kt | 4 -
.../kotlin/org/apache/airflow/sdk/ContextTest.kt | 14 -
.../kotlin/org/apache/airflow/sdk/ServerTest.kt | 292 ----
.../airflow/sdk/execution/CancellationTest.kt | 205 ---
.../apache/airflow/sdk/execution/LogSenderTest.kt | 94 --
.../aws/executors/batch/test_batch_executor.py | 16 +
.../aws/executors/utils/test_base_config_keys.py} | 24 +-
providers/databricks/README.rst | 1 +
providers/databricks/docs/index.rst | 5 +-
providers/databricks/docs/operators/agent.rst | 210 +++
providers/databricks/provider.yaml | 4 +
providers/databricks/pyproject.toml | 5 +
.../src/airflow/providers/databricks/exceptions.py | 8 +
.../providers/databricks/get_provider_info.py | 12 +-
.../airflow/providers/databricks/hooks/agent.py | 464 +++++
.../providers/databricks/operators/agent.py | 208 +++
.../airflow/providers/databricks/triggers/agent.py | 99 ++
.../system/databricks/example_databricks_agent.py | 112 ++
.../tests/system/databricks/resources}/__init__.py | 0
.../databricks/resources/agent_server/README.md | 79 +
.../databricks/resources/agent_server}/__init__.py | 0
.../databricks/resources/agent_server/app.yaml | 5 +-
.../resources/agent_server/hello_agent.py | 34 +-
.../resources/agent_server/requirements.txt | 5 +-
.../tests/unit/databricks/hooks/test_agent.py | 1113 ++++++++++++
.../tests/unit/databricks/operators/test_agent.py | 362 ++++
.../tests/unit/databricks/triggers/test_agent.py | 115 ++
.../airflow/providers/snowflake/hooks/snowflake.py | 6 +-
.../tests/unit/snowflake/hooks/test_snowflake.py | 35 +
providers/standard/docs/operators/hitl.rst | 71 +
.../example_dags/example_hitl_request_input.py | 41 +-
.../airflow/providers/standard/operators/hitl.py | 261 ++-
.../airflow/providers/standard/operators/python.py | 46 +
.../src/airflow/providers/standard/utils/hitl.py | 153 ++
.../airflow/providers/standard/version_compat.py | 2 +
.../standard/operators/test_hitl_request_input.py | 795 +++++++++
.../tests/unit/standard/utils/test_hitl.py | 97 ++
scripts/ci/prek/check_ci_workflows_in_sync.py | 27 +-
scripts/ci/prek/sync_java_sdk_docs.py | 148 ++
scripts/tests/ci/prek/test_sync_java_sdk_docs.py | 119 ++
ts-sdk/DEVELOPMENT.md | 5 +
uv.lock | 12 +-
110 files changed, 6830 insertions(+), 3238 deletions(-)
create mode 100644 .github/workflows/release-branch-ci.yml
create mode 100644
airflow-core/tests/unit/dag_processing/test_manager_api_results.py
create mode 100644 dev/breeze/tests/test_ci_commands.py
copy java-sdk/{README.md => DEVELOPMENT.md} (81%)
create mode 100644 java-sdk/example/README.md
delete mode 100644
java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/execution/Cancellation.kt
delete mode 100644
java-sdk/sdk/src/test/kotlin/org/apache/airflow/sdk/execution/CancellationTest.kt
delete mode 100644
java-sdk/sdk/src/test/kotlin/org/apache/airflow/sdk/execution/LogSenderTest.kt
copy providers/amazon/{src/airflow/providers/amazon/aws/executors/eks/utils.py
=> tests/unit/amazon/aws/executors/utils/test_base_config_keys.py} (66%)
create mode 100644 providers/databricks/docs/operators/agent.rst
create mode 100644
providers/databricks/src/airflow/providers/databricks/hooks/agent.py
create mode 100644
providers/databricks/src/airflow/providers/databricks/operators/agent.py
create mode 100644
providers/databricks/src/airflow/providers/databricks/triggers/agent.py
create mode 100644
providers/databricks/tests/system/databricks/example_databricks_agent.py
copy {airflow-core/src/airflow/_shared =>
providers/databricks/tests/system/databricks/resources}/__init__.py (100%)
create mode 100644
providers/databricks/tests/system/databricks/resources/agent_server/README.md
copy {airflow-core/src/airflow/_shared =>
providers/databricks/tests/system/databricks/resources/agent_server}/__init__.py
(100%)
copy dev/mypy/docs/spelling_wordlist.txt =>
providers/databricks/tests/system/databricks/resources/agent_server/app.yaml
(94%)
copy airflow-core/tests/unit/dags/test_dag_xcom_openlineage.py =>
providers/databricks/tests/system/databricks/resources/agent_server/hello_agent.py
(56%)
copy dev/mypy/docs/spelling_wordlist.txt =>
providers/databricks/tests/system/databricks/resources/agent_server/requirements.txt
(94%)
create mode 100644
providers/databricks/tests/unit/databricks/hooks/test_agent.py
create mode 100644
providers/databricks/tests/unit/databricks/operators/test_agent.py
create mode 100644
providers/databricks/tests/unit/databricks/triggers/test_agent.py
copy airflow-core/tests/unit/dags/test_multiple_dags.py =>
providers/standard/src/airflow/providers/standard/example_dags/example_hitl_request_input.py
(50%)
create mode 100644
providers/standard/src/airflow/providers/standard/utils/hitl.py
create mode 100644
providers/standard/tests/unit/standard/operators/test_hitl_request_input.py
create mode 100644 providers/standard/tests/unit/standard/utils/test_hitl.py
create mode 100755 scripts/ci/prek/sync_java_sdk_docs.py
create mode 100644 scripts/tests/ci/prek/test_sync_java_sdk_docs.py
