This is an automated email from the ASF dual-hosted git repository.
ephraimbuddy pushed a change to branch dag-processor-deployment
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 0eaf120f46c Fix the isolated Dag processor deployment examples and
chart checks
discard b37c671959c Handle invalid input in the Dag processor token provisioner
discard 8fd8975681b Use the trusted bundle catalog command in deployment
guidance
discard d33f382c7c2 Clarify the Dag processor rollout status check
discard 67121da05ce Clarify isolated Dag processor credential and migration
guidance
discard 28903c65675 Rotate Dag processor session tokens in Kubernetes Secrets
discard 8f263bf9aba Add API-mode Dag processor deployment support
discard 7b3a9c92950 Keep Dag processor callbacks and reparses through
transient failures
discard 34b9decff69 Correct the Dag processor API-mode docs and Compose example
discard 2a189ebaa3a Fix Dag processor API-mode shutdown, completion and claim
capacity
discard 86e1896d065 Release Dag processor claims for unpublishable sources
discard af6cfccf3c4 Separate trusted Dag bundle catalog provisioning from
processing
discard c0742d72e4b Restore isolated Dag processor regression coverage
discard c8bdf490b06 Keep API Dag processors recoverable through delivery
failures
discard 9c0edc87552 Move API work delivery state out of the Dag processor
manager
discard 5ee4f283049 Allow independent priority parses to finish in either
order in recovery tests
discard d598cf6d30b Preserve requested Dag processing work during bundle
failures
discard 88957cd19ed Wait for fresh inventories before running priority parses
discard a4ff7d8cdbd Preserve Dag processor work across retries and reparses
discard 6c52e0fa424 Run Dag processors without metadata database access
discard 561040614dd Show a failed parse's import error at once without
deactivating Dags
discard e86a04f21ce Keep publishing a Dag archive that has an unrepresentable
member
discard 62766357936 Close gaps in the Dag processor's API-mode tests
discard da0eb756c93 Handle Execution API failures in the API-mode Dag processor
discard 9c18a39c932 Filter invalid Dag discovery paths and deduplicate parse
sources
discard 0da7cb13920 Recover failed Dag parses and bound inventory publication
discard 359ac1175ae Introduce a source interface for requested Dag processing
work
discard 7bde027c6c2 Avoid database sessions when collecting API-mode Dag
results
discard 4e43816e83e Back off inventory retries and report results the API
would refuse
discard 1519b9cc762 Publish Dag processor results through the Execution API
discard 6eb02ca6c29 Clarify which Dag processors parsing cleanup expires
discard 935537f5f9c Tighten test coverage of Dag processor requested-work
delivery
discard f1a594ecb44 Fix Dag processor maintenance logs and stale Dag warning
cleanup
discard b53799ee804 Carry publication receipts through Dag processor
maintenance
discard 2a53dda7ab3 Preserve Dag cleanup and exact priority reparse targets
discard a297b84e8cf Fix Dag processor requested-work cleanup and delivery
discard 6f7a187c2ee Adapt legacy Dag upgrade regression to bundle discovery
results
discard 9d6fd27aa37 Align Dag processor claim migrations with main
discard 03705af75ab Test legacy Dag deactivation after bundle-name backfill
discard 63ca662ed58 Fix Dag processor work retries and metadata cleanup
discard 46c0eae8075 Retain Dag processor work until delivery is acknowledged
discard ca49ca30589 Fix which Dags a Dag processor publication marks stale
discard 4122f70b737 Reject only the invalid Dags in a Dag processor publication
discard 5ed8d1e034d Cover publication API rejections and clarify its docs and
names
discard 31fce26fd3c Reject Dags without a timetable or with incomplete
deadline alerts
discard 7515c4bdba5 Preserve compact source publications and actionable Dag
diagnostics
discard a12c698ab74 Clean asset references for stale Dags removed from bundles
discard 5f9490eb8f8 Align API-mode Dag publication with DB-mode cleanup
discard 165076ccd43 Allow Dag processors to publish metadata through the
Execution API
discard 1eccbc65c8d Describe the git provider's refresh-error probe accurately
discard 48fb3362e3e Stop repeated bundle drains and keep cleanup overrides
working
discard 2206f896bd7 Keep Dag parsing and metrics healthy during bundle outages
discard 55ed93f368d Revise Dag bundle refresh compatibility guidance
discard 956fffc57ca Clarify bundle refresh compatibility and simplify its
regression test
discard 2e38c400318 Keep Git bundles importable on Airflow 3.4
discard 0507b82aab1 Keep Dag parsing responsive during bundle refresh failures
discard 205af8d60e4 Make the bundle refresh regression test independent of
uptime
discard c68d64cfdcd Avoid parsing stalls while a bundle refresh waits for
imports
discard 1ddb3f937ae Keep Dag parsing results aligned with refreshed bundle
sources
add 776e9d2e78a Cache the dialect on PostgresHook and MsSqlHook (#75656)
add cc786845ad3 Build the ADBC database URI from the connection fields
(#74454)
add 31beac5deea disable external entities in SAML idp response parsing
(#71423)
add 7b60a8b86a5 Keep Dag parsing results aligned with refreshed bundle
sources
add 3a6f378d2cf Avoid parsing stalls while a bundle refresh waits for
imports
add 4f98ef56570 Make the bundle refresh regression test independent of
uptime
add bda0ed0fdba Keep Dag parsing responsive during bundle refresh failures
add 97c1d4bed76 Keep Git bundles importable on Airflow 3.4
add d299f5ee105 Clarify bundle refresh compatibility and simplify its
regression test
add 1f2332be9ac Revise Dag bundle refresh compatibility guidance
add 0bd18f5b169 Keep Dag parsing and metrics healthy during bundle outages
add 9170d76ac5d Stop repeated bundle drains and keep cleanup overrides
working
add 9bc77b1f662 Describe the git provider's refresh-error probe accurately
add 884ad9b4774 Allow Dag processors to publish metadata through the
Execution API
add 5cfccd0731f Align API-mode Dag publication with DB-mode cleanup
add 39e93220fbd Clean asset references for stale Dags removed from bundles
add 25d3fff0abd Preserve compact source publications and actionable Dag
diagnostics
add b127580ce34 Reject Dags without a timetable or with incomplete
deadline alerts
add 76b7e8ae7f8 Cover publication API rejections and clarify its docs and
names
add 2f5a9a94ea2 Reject only the invalid Dags in a Dag processor publication
add bc088968b9d Fix which Dags a Dag processor publication marks stale
add e153354ded8 Keep a file's Dags schedulable after a failed parse is
reported
add d57ee098679 Retain Dag processor work until delivery is acknowledged
add 269c6e5d2bb Fix Dag processor work retries and metadata cleanup
add 1015ea22005 Test legacy Dag deactivation after bundle-name backfill
add c3793a1468e Align Dag processor claim migrations with main
add e0d8f2ee77e Adapt legacy Dag upgrade regression to bundle discovery
results
add 6efd4f05f1d Fix Dag processor requested-work cleanup and delivery
add d0e1b327917 Preserve Dag cleanup and exact priority reparse targets
add 13d95093d10 Carry publication receipts through Dag processor
maintenance
add 5cfa450324d Fix Dag processor maintenance logs and stale Dag warning
cleanup
add 6612de200f2 Tighten test coverage of Dag processor requested-work
delivery
add 2d075fed107 Clarify which Dag processors parsing cleanup expires
add 33e3225adb9 Publish Dag processor results through the Execution API
add be5625fda9e Back off inventory retries and report results the API
would refuse
add 464cac8bc7c Avoid database sessions when collecting API-mode Dag
results
add c498960f987 Introduce a source interface for requested Dag processing
work
add f2ebf323258 Recover failed Dag parses and bound inventory publication
add e03b2930c97 Filter invalid Dag discovery paths and deduplicate parse
sources
add 67485f6eaca Handle Execution API failures in the API-mode Dag processor
add d2ed6dd08fb Close gaps in the Dag processor's API-mode tests
add cf8fbe2ecd9 Keep publishing a Dag archive that has an unrepresentable
member
add d31d0514fd4 Show a failed parse's import error at once without
deactivating Dags
add ad4414438cc Resume Dag parsing soon after a transient bundle listing
failure
add 01b3a1c4eeb Run Dag processors without metadata database access
add 0e4887a4513 Preserve Dag processor work across retries and reparses
add c4229fde8f2 Wait for fresh inventories before running priority parses
add 466ad79d60e Preserve requested Dag processing work during bundle
failures
add 47d634b8a2d Allow independent priority parses to finish in either
order in recovery tests
add f82ba4a70e7 Move API work delivery state out of the Dag processor
manager
add 952c049c87f Keep API Dag processors recoverable through delivery
failures
add 3e4d3d977f1 Restore isolated Dag processor regression coverage
add 49ff7222679 Separate trusted Dag bundle catalog provisioning from
processing
add 769055ac2d8 Release Dag processor claims for unpublishable sources
add 13a55ee735c Fix Dag processor API-mode shutdown, completion and claim
capacity
add 14733808ce7 Correct the Dag processor API-mode docs and Compose example
add cc007a040af Keep Dag processor callbacks and reparses through
transient failures
add d072d7711a2 Stop repeating Dag callbacks after an Execution API
request fails
add bb1197afb40 Add API-mode Dag processor deployment support
add 48c34c0d606 Rotate Dag processor session tokens in Kubernetes Secrets
add ea067620219 Clarify isolated Dag processor credential and migration
guidance
add f2799837c51 Clarify the Dag processor rollout status check
add b4764609c85 Use the trusted bundle catalog command in deployment
guidance
add 4e32540f0fe Handle invalid input in the Dag processor token provisioner
add a48a46567f8 Fix the isolated Dag processor deployment examples and
chart checks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0eaf120f46c)
\
N -- N -- N refs/heads/dag-processor-deployment (a48a46567f8)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../administration-and-deployment/dag-bundles.rst | 4 +
.../execution_api/routes/dag_parsing.py | 6 +-
.../src/airflow/config_templates/config.yml | 5 +-
.../src/airflow/dag_processing/collection.py | 27 +++---
airflow-core/src/airflow/dag_processing/manager.py | 36 +++++---
.../src/airflow/dag_processing/processor.py | 23 +++++
.../versions/head/test_dag_parsing.py | 54 +++++++++++-
.../tests/unit/dag_processing/test_manager.py | 9 +-
.../dag_processing/test_manager_api_inventory.py | 53 ++++++++++--
.../tests/unit/dag_processing/test_processor.py | 90 ++++++++++++++++++++
.../dag_processing/test_requested_work_recovery.py | 99 ++++++++++++++++++++--
docs/spelling_wordlist.txt | 1 +
.../airflow/providers/amazon/aws/hooks/base_aws.py | 6 +-
.../tests/unit/amazon/aws/hooks/test_base_aws.py | 43 ++++++++++
providers/apache/arrow/docs/connections/adbc.rst | 22 +++--
providers/apache/arrow/provider.yaml | 12 ++-
.../providers/apache/arrow/get_provider_info.py | 10 ++-
.../airflow/providers/apache/arrow/hooks/adbc.py | 43 +++++++---
.../tests/unit/apache/arrow/hooks/test_adbc.py | 64 ++++++++++++++
.../providers/microsoft/mssql/hooks/mssql.py | 3 +-
.../tests/unit/microsoft/mssql/hooks/test_mssql.py | 22 +++++
.../airflow/providers/postgres/hooks/postgres.py | 3 +-
.../tests/unit/postgres/hooks/test_postgres.py | 16 ++++
23 files changed, 575 insertions(+), 76 deletions(-)
