This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch changes-3.1.3rc1 in repository https://gitbox.apache.org/repos/asf/airflow.git
commit f3fadb30c1f72f6b6e4d81d1b40985832d4873b6 Author: Ephraim Anierobi <[email protected]> AuthorDate: Mon Nov 10 15:17:47 2025 +0100 Update RELEASE_NOTES.rst --- RELEASE_NOTES.rst | 89 ++++++++++++++++++++++++ airflow-core/newsfragments/56583.significant.rst | 49 ------------- airflow-core/newsfragments/56609.significant.rst | 14 ---- reproducible_build.yaml | 4 +- 4 files changed, 91 insertions(+), 65 deletions(-) diff --git a/RELEASE_NOTES.rst b/RELEASE_NOTES.rst index e5fa249aa01..ae665058d8b 100644 --- a/RELEASE_NOTES.rst +++ b/RELEASE_NOTES.rst @@ -24,6 +24,95 @@ .. towncrier release notes start +Airflow 3.1.3 (2025-11-13) +-------------------------- + +Significant Changes +^^^^^^^^^^^^^^^^^^^ + +Fix Connection & Variable access in API server contexts (plugins, log handlers)(#56583) +""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +Previously, hooks used in API server contexts (plugins, middlewares, log handlers) would fail with an ``ImportError`` +for ``SUPERVISOR_COMMS``, because ``SUPERVISOR_COMMS`` only exists in task runner child processes. + +This has been fixed by implementing automatic context detection with three separate secrets backend chains: + +**Context Detection:** + +1. **Client contexts** (task runner in worker): Detected via ``SUPERVISOR_COMMS`` presence +2. **Server contexts** (API server, scheduler): Explicitly marked with ``_AIRFLOW_PROCESS_CONTEXT=server`` environment variable +3. **Fallback contexts** (supervisor, unknown contexts): Neither marker present, uses minimal safe chain + +**Backend Chains:** + +- **Client**: ``EnvironmentVariablesBackend`` → ``ExecutionAPISecretsBackend`` (routes to Execution API via SUPERVISOR_COMMS) +- **Server**: ``EnvironmentVariablesBackend`` → ``MetastoreBackend`` (direct database access) +- **Fallback**: ``EnvironmentVariablesBackend`` only (+ external backends from config like AWS Secrets Manager, Vault) + +The fallback chain is crucial for supervisor processes (worker-side, before task runner starts) which need to access +external secrets for remote logging setup but should not use ``MetastoreBackend`` (to maintain worker isolation). + +**Architecture Benefits:** + +- Workers (supervisor + task runner) never use ``MetastoreBackend``, maintaining strict isolation +- External secrets backends (AWS Secrets Manager, Vault, etc.) work in all three contexts +- Supervisor falls back to Execution API client for connections not found in external backends +- API server and scheduler have direct database access for optimal performance + +**Impact:** + +- Hooks like ``GCSHook``, ``S3Hook`` now work correctly in log handlers and plugins +- No code changes required for existing plugins or hooks +- Workers remain isolated from direct database access (network-level DB blocking fully supported) +- External secrets work everywhere (workers, supervisor, API server) +- Robust handling of unknown contexts with safe minimal chain + +See: `#56120 <https://github.com/apache/airflow/issues/56120>`__, `#56583 <https://github.com/apache/airflow/issues/56583>`__, `#51816 <https://github.com/apache/airflow/issues/51816>`__ + +Remove insecure dagReports API endpoint that executed user code in API server (#56609) +"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + + The ``/api/v2/dagReports`` endpoint has been removed because it loaded user DAG files directly in the API server process, + violating Airflow's security architecture. This endpoint was not used in the UI and had no known consumers. + Use the ``airflow dags report`` CLI command instead for DAG loading reports. + +Bug Fixes +^^^^^^^^^ + +- Fix secrets being exposed in Jinja template rendering error messages (#57467) (#57962) +- UI: Fix slow loading on next run assets page (#58052) (#58064) +- Fix logout not working in airflow-core (#57990) (#58043) +- Fix slow loading on UI [(#57820) (#57856), (#57956) (#57973), (#57957) (#57972),(#57869) (#57882), (#57868) (#57918),(#57624) (#57757)] +- UI: Fix log download to include .txt file extension (#57991) (#58040) +- Fix scheduler using incorrect max_active_runs value from cached DAG (#57619) (#57959) +- Fix database migration failures when XCom contains NaN values (#57866) (#57893) +- Fix incorrect task context in trigger rule scenarios (#57884) (#57892) +- UI: Fix test connection not working (#57811) (#57852) +- Fix worker healthcheck timeout not respecting worker-timeout CLI option (#57731) (#57854) +- Fix provider hooks not loading when FAB provider is not installed (#57717) (#57830) +- Fix slow API responses for task instances list [(#57645) (#57794), (#57646) (#57664),(#57500) (#57735), (#57549) (#57738), (#57450) (#57736),(#57647) (#57732)] +- Fix task instance errors when tasks are triggered by trigger rules (#57474) (#57786) +- Fix type consistency for extra field in Asset, AssetAlias, and AssetEvent (#57352) (#57728) +- Fix upgrade failures when XCom contains NaN in string values (#57614) + +Miscellaneous +^^^^^^^^^^^^^ + +- UI: Add resize functionality to DAG run and task instance notes (#57897) (#58068) +- Add Taiwan translation for UI (#58121) +- UI: Shorten German translation of Asset in navigation (#57671) (#57690) +- Fix code formatting via ruff preview (#57641) (#57670) +- Remove remnants from unlimited parallelism in local executor (#57579) (#57644) + +Doc Only Changes +^^^^^^^^^^^^^^^^ + +- Add learnings from Airflow 3 migration guide (#57989) (#58083) +- Fix duplicate mention of 'DAGs' and 'tasks' in overview documentation (#57524) (#57793) +- Document asset event extra storage behavior (#57727) (#57734) + + Airflow 3.1.2 (2025-11-05) -------------------------- diff --git a/airflow-core/newsfragments/56583.significant.rst b/airflow-core/newsfragments/56583.significant.rst deleted file mode 100644 index 3e80bfe0cbb..00000000000 --- a/airflow-core/newsfragments/56583.significant.rst +++ /dev/null @@ -1,49 +0,0 @@ -Fix Connection & Variable access in API server contexts (plugins, log handlers) - -Previously, hooks used in API server contexts (plugins, middlewares, log handlers) would fail with an ``ImportError`` -for ``SUPERVISOR_COMMS``, because ``SUPERVISOR_COMMS`` only exists in task runner child processes. - -This has been fixed by implementing automatic context detection with three separate secrets backend chains: - -**Context Detection:** - -1. **Client contexts** (task runner in worker): Detected via ``SUPERVISOR_COMMS`` presence -2. **Server contexts** (API server, scheduler): Explicitly marked with ``_AIRFLOW_PROCESS_CONTEXT=server`` environment variable -3. **Fallback contexts** (supervisor, unknown contexts): Neither marker present, uses minimal safe chain - -**Backend Chains:** - -- **Client**: ``EnvironmentVariablesBackend`` → ``ExecutionAPISecretsBackend`` (routes to Execution API via SUPERVISOR_COMMS) -- **Server**: ``EnvironmentVariablesBackend`` → ``MetastoreBackend`` (direct database access) -- **Fallback**: ``EnvironmentVariablesBackend`` only (+ external backends from config like AWS Secrets Manager, Vault) - -The fallback chain is crucial for supervisor processes (worker-side, before task runner starts) which need to access -external secrets for remote logging setup but should not use ``MetastoreBackend`` (to maintain worker isolation). - -**Architecture Benefits:** - -- Workers (supervisor + task runner) never use ``MetastoreBackend``, maintaining strict isolation -- External secrets backends (AWS Secrets Manager, Vault, etc.) work in all three contexts -- Supervisor falls back to Execution API client for connections not found in external backends -- API server and scheduler have direct database access for optimal performance - -**Impact:** - -- Hooks like ``GCSHook``, ``S3Hook`` now work correctly in log handlers and plugins -- No code changes required for existing plugins or hooks -- Workers remain isolated from direct database access (network-level DB blocking fully supported) -- External secrets work everywhere (workers, supervisor, API server) -- Robust handling of unknown contexts with safe minimal chain - -See: `#56120 <https://github.com/apache/airflow/issues/56120>`__, `#56583 <https://github.com/apache/airflow/issues/56583>`__, `#51816 <https://github.com/apache/airflow/issues/51816>`__ - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [ ] API changes - * [ ] CLI changes - * [ ] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes diff --git a/airflow-core/newsfragments/56609.significant.rst b/airflow-core/newsfragments/56609.significant.rst deleted file mode 100644 index b32e4c96116..00000000000 --- a/airflow-core/newsfragments/56609.significant.rst +++ /dev/null @@ -1,14 +0,0 @@ -Remove insecure dagReports API endpoint that executed user code in API server - -The ``/api/v2/dagReports`` endpoint has been removed because it loaded user DAG files directly in the API server process, violating Airflow's security architecture. This endpoint was not used in the UI and had no known consumers. Use the ``airflow dags report`` CLI command instead for DAG loading reports. - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [x] API changes - * [ ] CLI changes - * [ ] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes diff --git a/reproducible_build.yaml b/reproducible_build.yaml index 9b5b657885a..16bc89643a7 100644 --- a/reproducible_build.yaml +++ b/reproducible_build.yaml @@ -1,2 +1,2 @@ -release-notes-hash: 9026e301c66a2c3051e54370be3eae72 -source-date-epoch: 1761930733 +release-notes-hash: 2c1be66f4854e515dcba2446ff800fa4 +source-date-epoch: 1762784248
