This is an automated email from the ASF dual-hosted git repository. vatsrahul1001 pushed a commit to branch changes-3.4.0rc1 in repository https://gitbox.apache.org/repos/asf/airflow.git
commit a841128523c811b1b5b3a311bcddc3984fd3fe61 Author: Rahul Vats <[email protected]> AuthorDate: Mon Oct 26 18:38:54 2026 +0530 Add release notes for 3.4.0rc1 --- RELEASE_NOTES.rst | 284 +++++++++++++++-------- airflow-core/newsfragments/74896.bugfix.rst | 1 - airflow-core/newsfragments/75414.significant.rst | 20 -- docs/spelling_wordlist.txt | 2 + reproducible_build.yaml | 4 +- 5 files changed, 194 insertions(+), 117 deletions(-) diff --git a/RELEASE_NOTES.rst b/RELEASE_NOTES.rst index fe8d90f0f25..d9196b0b891 100644 --- a/RELEASE_NOTES.rst +++ b/RELEASE_NOTES.rst @@ -30,7 +30,7 @@ Airflow 3.4.0 (2026-11-02) Significant Changes ^^^^^^^^^^^^^^^^^^^ -Task Loops (#73005, #74222, #74274, #74275) +Task Loops (#73005, #74222, #74275, #74345) """"""""""""""""""""""""""""""""""""""""""" Airflow 3.4.0 introduces Task Loops (AIP-111): a decorated task group can be repeated a number of @@ -486,20 +486,45 @@ Custom Task State Store backends will likewise need to include ``scope.region_id`` in their storage key to support mapped tasks inside a loop and loop iterations regenerated by clearing. +``airflow.utils.email`` is deprecated in favor of the SMTP provider (#75414) +"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +The functions in ``airflow.utils.email`` (``send_email``, ``send_email_smtp``, ``send_mime_email``, +``build_mime_message`` and ``get_email_address_list``) now live in +``airflow.providers.smtp.utils.emailer``. The old names still work and raise a ``DeprecatedImportWarning``. +Airflow now requires ``apache-airflow-providers-smtp>=3.2.0``. + +Keep ``[email] email_backend`` set to ``airflow.utils.email.send_email_smtp``. Airflow treats that value as +"send task emails with ``SmtpNotifier``", which reads all SMTP settings from the Connection. Changing it to +the provider path sends through the ``[smtp]`` settings instead. + +If you call these functions directly: + +* The SMTP Connection is looked up with ``BaseHook.get_connection``. A script that runs outside Airflow's + processes needs the Connection in an environment variable or a secrets backend, not only in the metadata + database. +* Settings are read through the Task SDK configuration, so values changed at runtime with + ``airflow.configuration.conf.set()`` are not seen. ``airflow.cfg`` and environment variables work as before. +* Tests that patch ``airflow.utils.email.send_mime_email`` no longer intercept the call made by + ``send_email_smtp``. Patch ``airflow.providers.smtp.utils.emailer.send_mime_email`` instead. + New Features ^^^^^^^^^^^^ -- Java SDK: Add edge labels to native Dags (#75173) -- Add a Duplicate action to the Connections list (#75143) -- Make the scheduler Dag cache size and TTL configurable (#75087) -- TS SDK: Add edge labels with label() (#75031) +- UI: Show task group states as stacked cards in the Grid view (#75442) +- Let plugins add their own actions to Dag, run and task instance rows (#75427) +- TS SDK: Reject blank edge labels and label task edges into groups (#75190) +- Expose task instance loop pass and expansion size to listeners (#75132) +- Add a Duplicate action to the Connections list (#75110) - Let retry policies reschedule a task without using a retry (#74993) - Let plugin views be scoped to specific Assets (#74992) -- UI: Add Select Recent Configurations dropdown to Trigger Dag form (#74977) -- Add support for forwarding CeleryExecutor task logs to stdout (#74976) -- Add deadline callback log visibility to the Dag Run view in UI (#74955) -- Allow loop gates to add human-readable decisions (#74932) - Add Map Index search to dag breadcrumb (#74893) +- UI: Show secondary states of mapped tasks in the Grid view (#74883) +- Allow loop gates to add human-readable decisions (#74806) +- Java SDK: Let a native Java Dag set one queue for all its tasks (#74538) +- Java SDK: Add edge labels to native Dags (#74523) +- Add asset ID filtering to ``airflow db clean`` (#74498) +- TS SDK: Add edge labels with label() (#74494) - Add dag version lookup on get task endpoint (#74365) - Add Logical Date column to the Runs table (#74340) - UI: Add a Dag Run ID column to the Task Instances list (#74316) @@ -508,8 +533,10 @@ New Features - Add URL param support to calendar view (#73237) (#73762) - UI: Add team filter to Human-in-the-loop task instances listing (#73726) - add bundle_name tags to missing dag processor metrics (#73717) +- Add deadline callback log visibility to the Dag Run view in UI (#73580) - Add fab-oauth meta-package extra (#73565) - Allow a Job to be associated with several teams (#73561) +- UI: Add time range filtering to the Gantt view (#73555) - Add ``ChainRetryPolicy`` to compose retry policies in order (#73508) - Add task state store API to the Java SDK (#73464) - UI: Add a default preference for expanded task groups (#73429) @@ -531,6 +558,7 @@ New Features - ``is_scheduled`` REST API parameter (#72029) - Allow Variable.get to reuse the caller's database session (#71968) - UI: Add team column and filter to the audit log (#71916) +- Make the scheduler Dag cache size and TTL configurable (#71816) - Support a regex for the API's CORS allowed origins (#71778) - CLI: Add --limit flag to airflow dags list-runs (#71729) - Add SSL cipher list option to the api server (#71645) @@ -542,6 +570,7 @@ New Features - Add a user settings page for cross-page UI defaults (#70687) - Support a Unix Domain Socket for the metrics clients (#70517) - UI: Add a time-window scope to the Dags list run state filter (#70470) +- UI: Add Select Recent Configurations dropdown to Trigger Dag form (#70413) - UI: Add a slowest tasks chart to the Dag Overview (#70305) - Add task log message when the Execution API is unreachable (#70290) - UI: Add team column and filter to Dag Run and Task Instance lists (#70241) @@ -552,6 +581,7 @@ New Features - Add serialized Dag version diff engine (#69864) - UI: Add timetable type filter to Dag list (#69728) - Allow embedding a Dag run in an external trace via run conf (#69633) +- Add support for forwarding CeleryExecutor task logs to stdout (#69597) - Add deadlines to Calendar Tab (#69559) - Add team_name attribute to plugins for multi-team (#69502) - Add exact-match uri filter to GET /assets endpoint (#69489) @@ -563,7 +593,7 @@ New Features - Add metrics for how often a dag gets serialised. (#68906) - Allow forcing trace head-sampling per DAG run via conf (#68860) - Add Calendar Deadlines API and Data Models (#68604) -- Add NodeCoordinator (#68548) +- Add a Node.js coordinator for running TypeScript Dags (#68548) - Add async task store accessors for async tasks (#68232) - Add scheduler observability metrics (#68068) - Add extra field filtering for asset events (#64611) @@ -573,60 +603,88 @@ New Features Bug Fixes ^^^^^^^^^ -- Fix log template sync failing when a custom SQL Alchemy schema is configured (#75194) -- Java SDK: Send log timestamps in UTC (#75186) -- Fix 500 when ordering the Dag list by state or start_date (#75148) -- Keep Dag folder selection distinct across bundle names (#75146) -- Keep parent Dag folders collapsed after one click (#75145) -- Fix ``Trigger.clean_unused`` rejecting a caller-provided session (#75122) -- Scope asset Dag and task references to readable Dags (#75107) -- Reject reserved serialization keys in Dag run conf (#75105) +- Let listeners see the end date of a skipped task instance (#75685) +- Refuse Human-in-the-Loop responses once the task stopped waiting (#75645) +- TS SDK: Reject a blank queue in Dag and task specs (#75573) +- Fix scheduler crash when retrying a task that timed out or was killed externally (#75472) +- Fix task loop iteration log links pointing to a nonexistent page (#75458) +- Go SDK: Fail a task whose return-value XCom push fails (#75443) +- UI: Fix visual and console-error bugs in the Dag list and details pages (#75418) +- Decide the ``min_one_success`` trigger rule per index in mapped task groups (#75417) +- Return a 5xx response on a database deadlock when updating task state (#75368) +- Run team tasks on the team executor when named by class or path (#75362) +- Fix task instance filtering for null start dates (#75321) +- UI: Keep the multiselect filter pill's focus ring visible (#75307) +- UI: Show errors when the task clear confirmation check fails (#75269) +- Skip scheduling tasks when their Dag cannot be deserialized (#75263) +- Read a team's running task logs through the team's executor (#75255) +- Keep the token cookie set by a route in ``JWTRefreshMiddleware`` (#75217) +- UI: Reveal selected Dag folders when names contain pipes (#75196) +- UI: Show a label instead of a broken link for hidden assets (#75178) +- Show source code lines in task log tracebacks (#75100) +- Fix ``Trigger.clean_unused`` rejecting a caller-provided session (#75096) - Reserve graph header width for loop-group iteration badge (#75095) -- Scope asset-wide queued event deletion to editable Dags (#75089) -- Refuse a HITL response for a request the task has since replaced (#74982) -- Stop the secrets backend lookup from importing the task runner outside tasks (#74980) -- Fix ``map_index_template`` failing for numeric-looking labels with ``render_template_as_native_obj`` (#74978) -- Do not abort Dag run creation when a deadline alert fails (#74975) -- Fix ORM models failing to configure in processes that skip database setup (#74971) -- Fix Lang-SDK decoder typing for pydantic 2.14 ``TypeForm`` signature (#74970) -- Fix CLI --bundle-name finding no Dags in Git bundles (#74969) -- Fix clearing with upstream and downstream selecting unrelated tasks (#74968) -- Make ``ZIP_REGEX`` case-insensitive to match .ZIP archives (#74967) -- Fix /ui/dags recent-runs query degradation on large, skewed dag_run tables (#74966) -- Fix missing ``task.queued_duration`` metric in Airflow 3 (#74965) -- UI: Keep JSON text in string HITL params instead of showing [object Object] (#74961) -- UI: Fix the Dags list showing an error page and reduce unnecessary re-renders (#74956) -- Fix Dag list run badges and counters not refreshing after completion (#74954) -- Drain result_queue before dispatch in LocalExecutor to prevent runtime dispatch deadlock (#74949) -- Clear language SDK task XComs in the coordinator when a try starts (#74936) -- Fix dynamically mapped tasks never running on scheduled Dag runs (#74918) -- Match camelCase keys against sensitive field names in masker (#74897) -- Keep queued connection tests from overwriting newer connections (#74895) -- Fix Dag version diff max-changes border, limit wording and duplicate count (#74886) -- Do not warn about bare postgresql:// metadata DB URLs (#74884) +- Keep Dag folder selection distinct across bundle names (#75080) +- Keep parent Dag folders collapsed after one click (#75079) +- Reject non-positive ``max_active_runs`` when creating a backfill (#75071) +- TS SDK: Fix "Dynamic require" error for CommonJS packages in ``airflow-ts-pack`` (#75067) +- Fix 500 when ordering the Dag list by state or start_date (#75025) +- Run the Kerberos ticket renewer without metadata database checks (#75000) +- Fix serialization of datetimes with a fixed UTC offset (#74922) +- Reset Dag autoregistration state when a Python Dag file fails to load (#74911) +- Reject blank pool names and Variable keys in the REST API (#74896) - Resolve a trigger's XCom requests as its deferred task instance (#74868) - Fix JWT token not refreshed when token expires mid-request (#74865) -- Java SDK: Reject byte arrays instead of letting the task hang (#74831) -- Go SDK: pack a bundle that registers no Dags at pack time (#74821) -- Report a Dag whose timetable fails to load as an import error (#74807) -- UI: Fix clearing a task instance doing nothing after Confirm (#74738) +- Do not warn about bare postgresql:// metadata DB URLs (#74863) +- Fix dynamically mapped tasks never running on scheduled Dag runs (#74861) +- Speed up API server requests by keeping JWT checks off the event loop (#74856) +- Keep ``.airflowignore`` rules when the Dags folder symlink is switched (#74853) +- Match camelCase keys against sensitive field names in masker (#74840) +- Scope asset-wide queued event deletion to editable Dags (#74839) +- Keep queued connection tests from overwriting newer connections (#74837) +- Scope asset Dag and task references to readable Dags (#74836) +- Reject reserved serialization keys in Dag run conf (#74834) +- Fix Dag version diff max-changes border, limit wording and duplicate count (#74818) +- Java SDK: Send log timestamps in UTC (#74783) +- Fix Dag list run badges and counters not refreshing after completion (#74734) +- UI: Fix clearing a task instance doing nothing after Confirm (#74725) +- Report a Dag whose timetable fails to load as an import error (#74709) +- UI: Keep JSON text in string HITL params instead of showing [object Object] (#74702) +- Refuse a HITL response for a request the task has since replaced (#74688) +- Fix ORM models failing to configure in processes that skip database setup (#74642) +- Stop the secrets backend lookup from importing the task runner outside tasks (#74641) +- Go SDK: pack a bundle that registers no Dags at pack time (#74637) +- Fix language SDK Dag parsing failing under pydantic 2.14 (#74628) - Fix 400 and 500s on POST /connections/defaults in multi-team mode (#74615) +- Fix clearing with upstream and downstream selecting unrelated tasks (#74548) - Keep the scheduler running when a queued run's Dag fails to deserialize (#74499) +- UI: Fix the Dags list showing an error page and reduce unnecessary re-renders (#74426) - Fix plugin applies_to scoping (#74425) +- Fix /ui/dags recent-runs query degradation on large, skewed dag_run tables (#74401) - Fix XCom query in not_previously_skipped_dep to use xcom_entity (#74377) - Fix short-circuit in a mapped task group not skipping later tasks (#74283) +- Make ``ZIP_REGEX`` case-insensitive to match .ZIP archives (#74269) +- Clear language SDK task XComs in the coordinator when a try starts (#74221) +- Java SDK: Reject byte arrays instead of letting the task hang (#74218) - UI : Fix duplicate "From" label on DAG backfill filters (#74189) - Improve LocalExecutor bookkeeping: correctly add tasks to the ``running`` list (#74142) - UI: Fix XCom list crash when a value request fails (#74131) - Fix date range filter inputs committing on every keystroke (#74011) - Fix short-circuit not skipping already expanded mapped tasks (#73960) - Fix short-circuit inside a mapped task group not skipping downstream (#73959) +- Fix task clear and callback bundles after code-only Dag changes (#73915) - UI: Fix external plugin nav links not navigating on click (#73913) +- Clear retry policy state when a task instance is cleared (#73889) - Correctly shutdown async sessions on exit. (#73838) +- Go SDK: Stop log attributes from displacing standard log fields (#73820) - API: Translate domain ValueError in Task State Store route (#73753) - Filter jobs API ``is_alive`` in SQL so pagination stays correct (#73684) +- Guard ``register_at_fork`` in shared stats for Windows compatibility (#73680) - Fix boolean value of start date used in url start_date_gte filter. (#73676) +- Stop a commented-out ``executable_resumable`` from marking a module durable (#73607) - Fix square corner on fully filled dashboard History bars (#73543) +- Fix empty audit log ``dag_id`` for ``DAG.cli()`` subcommands (#73379) +- Fix CLI --bundle-name finding no Dags in Git bundles (#73377) - Fix airflow plugins and dags pause printing prose with --output json (#73270) - Fix 500 from POST ``/auth/token`` on a request body it cannot parse (#73258) - Fix slow task-log scrolling by memoizing useLogGroups' visible-items index (#73252) @@ -637,26 +695,36 @@ Bug Fixes - Stop the scheduler creating Dag runs for paused partitioned-asset Dags (#73151) - Fix Dags staying blocked by max_active_runs until the next parse (#73032) - Fix airflow standalone announcing readiness before the API server is up (#73025) +- Hide assets the caller may not read from the UI dependencies graph (#73008) - Fix airflow teams list --output json printing prose when no teams exist (#72945) +- Fix ``tasks clear --task-regex`` matching as a plain substring (#72941) - Avoid a full asset table scan per request in simple auth manager (#72934) - Fix Dag callbacks dropped when bundle versioning is disabled (#72930) - Stop one malformed provider from breaking discovery for all providers (#72854) - Fix backfill DagRuns being permanently starved from scheduling (#72730) - Fix airflow db check reporting one fewer retry than it will make (#72708) - Fix airflow dags list --columns help showing wrong default columns (#72607) +- UI: Render Arabic durations with the correct plural form (#72604) - Fix XCom slice endpoint for crossed bounds (#72584) +- Fix the ``dag_processing.last_duration`` timing metric reporting the wrong value (#72553) - Stop emitting the legacy first_task_scheduling_delay metric twice (#72512) - Fix Dag metrics dropped when a tag contains invalid characters (#72364) - Fix misleading error for malformed version in airflow db downgrade (#72331) - Fix timedelta serde to accept int payloads (#72326) - Fix plugin registration returning early on duplicate names in Task SDK (#72253) +- Redact secrets written through ``RedactedIO.writelines()`` (#72063) +- Fix Dag run filtering for null start dates (#71654) +- Fix ``map_index_template`` failing for numeric-looking labels with ``render_template_as_native_obj`` (#71592) - Keep the MeterProvider configured via OTEL_CONFIG_FILE (#71529) - Fix Asset Event table titles showing a raw translation key (#71443) - Fix executor validation error referencing non-existent [core] executors (#71336) - Fix MySQL metadata database failing when only PyMySQL is installed (#71251) - Fix OpenSearch remote logging crash if port is empty (#71170) +- Fix duplicate pending Dag runs for the same asset partition key (#71074) - Fix /ui/assets picking a stale "last event" under concurrent asset writes (#71047) - Stop airflow config update from writing a backup during dry-run (#70979) +- Fix log template sync failing when a custom SQL Alchemy schema is configured (#70793) +- Drain result_queue before dispatch in LocalExecutor to prevent runtime dispatch deadlock (#70545) - Fix TypeError clearing a DAG run when a queued-at deadline interval is stored as JSON (#70370) - Fix manual runs getting a zero-length data interval across DST (#70089) - Use atomic upsert for asset run queue on MySQL to avoid deadlocks (#69977) @@ -671,8 +739,10 @@ Bug Fixes - Fix mapped task group XCom collapsing to a bare value for a single expansion (#69075) - Fix macOS SIGSEGV in triggerer and Dag processor via fork+exec (#69008) - Fix branch not skipping mapped tasks with custom XCom backend (#68954) +- Do not abort Dag run creation when a deadline alert fails (#68917) - Fix duplicate heartbeat-timeout task callbacks (#68008) - Drain result_queue in LocalExecutor to prevent process join deadlock (#67881) +- Fix missing ``task.queued_duration`` metric in Airflow 3 (#67592) - Fix CLI connection display reporting wrong is_extra_encrypted flag (#67561) - Fix remote processor injection happening before dictConfig runs in configure_logging (#66633) - Fix airflow db clean IntegrityError when a dag_version is referenced by a task_instance (#66350) @@ -685,47 +755,51 @@ Bug Fixes Miscellaneous ^^^^^^^^^^^^^ -- TS SDK: Collect ``dag_dependencies`` in the tasks pass (#75191) -- Mask CLI secrets in the audit log by parsed argument, not argv position (#75144) -- UI: Replace stacked Dag header banners with inline status (#75125) -- Pass runtime values to example Dag shell commands through env (#75121) -- Read XCom without object reconstruction during dependency evaluation (#75117) -- Add the resume event to stored next_kwargs without decoding them (#75116) -- Mask XCom values in the audit log (#75113) -- Mask registered secrets at any nesting depth (#75112) -- Open the Dags menu from the Dags nav button instead of navigating (#75109) -- Scope upstream asset events of a Dag run to assets and Dags the caller may read (#75108) -- Mask provider-declared sensitive connection extras on every connection type (#75106) -- Validate ``CallbackTrigger`` callback path before running the callback (#75102) -- Render XCom values on the API server without rebuilding objects (#75088) -- UI: Guard the remaining ``RadioCardRoot`` values instead of casting (#74989) -- UI: Keep unnamed columns out of the table columns menu (#74988) -- Lazy-load the Code tab grammar from the Dag's stored language (#74987) -- UI: Use ``RadioCardRoot``'s typed ``onValueChange`` instead of DOM casts (#74979) -- Highlight YAML Dag source in the Code tab (#74972) -- UI: Drop the loading spinner for base React plugins (#74964) -- Resolve loop iterations through family and path columns on dynamic regions (#74963) -- Show Dag version diff values-hidden pill only when values are hidden (#74960) -- Coerce delta and other typed fields when building operators from YAML Dags (#74959) -- UI: Fold draining into the Dags Paused state filter (#74958) -- UI: Add clicked tags to the Dags list filters instead of replacing them (#74957) -- UI: Open nav sidebar menus on hover and add a Dags menu (#74953) -- Reduce db clean batch probe overhead (#74952) -- Batch triggerer unused trigger cleanup deletes (#74950) -- Upgrade Task Loop UX (#74931) -- Skip the scheduler's Dag lookup for Dag runs with no callback to send (#74857) -- TS SDK: Reject ``triggerDagRun`` tasks in ``dag.if`` and ``dag.switch`` (#74828) -- Java SDK: Rename annotation namespace from Builder to Airflow (#74826) -- Show serializer schema in Dag version diff only when it differs (#74739) +- Add metrics for task and asset state store access (#75729) +- Add ``dag_id`` and ``task_id`` tags to the resumable_job metrics (#75721) +- Run a language SDK runtime in its own process group (#75707) +- Require cadwyn 8.1 to allow FastAPI 0.137+ (#75689) +- Report language SDK runtime launch failures in the task log (#75561) +- UI: Remove the info tooltip from the Dag version comparison (#75526) +- UI: Show the filtered task count in the Dag Tasks tab heading (#75448) +- TS SDK: Take the task state store retention in seconds (#75445) +- UI: Show the Dag version diff occurrence count on the path (#75433) +- UI: Make the Dags sidebar button a link and move Dag Runs and Task Instances under Browse (#75422) +- Optimize the scheduler's asset orphanage check with an ``EXISTS`` subquery instead of a fan-out join (#75404) +- UI: Clean up the Dag Version tab (#75244) +- Go SDK: Write the edge label text before the task (#75185) +- Open the Dags menu from the Dags nav button instead of navigating (#75092) +- TS SDK: Populate Dag dependencies for TypeScript Dags (#75032) +- Warn about overly large task state values from language SDK tasks (#74945) +- Resolve loop iterations through family and path columns on dynamic regions (#74892) +- Validate ``CallbackTrigger`` callback path before running the callback (#74842) +- Skip the scheduler's Dag lookup for Dag runs with no callback to send (#74835) +- Render XCom values on the API server without rebuilding objects (#74829) +- UI: Preload base React plugin bundles with the page (#74777) +- UI: Keep unnamed columns out of the table columns menu (#74774) +- Number task loop iterations from one instead of zero (#74773) +- UI: Drop the loading spinner for base React plugins (#74762) +- Coerce delta and other typed fields when building operators from YAML Dags (#74749) +- Show Dag version diff values-hidden pill only when values are hidden (#74746) +- UI: Open nav sidebar menus on hover and add a Dags menu (#74745) +- Show serializer schema in Dag version diff only when it differs (#74683) +- Upgrade Task Loop UX (#74671) +- UI: Fold draining into the Dags Paused state filter (#74668) +- Lazy-load the Code tab grammar from the Dag's stored language (#74643) +- Highlight YAML Dag source in the Code tab (#74633) +- UI: Add clicked tags to the Dags list filters instead of replacing them (#74622) +- Java SDK: Rename annotation namespace from Builder to Airflow (#74604) - Update waiting tasks to the latest Dag version (#74592) - Disable psycopg3 auto-prepare by default behind connection poolers (#74581) +- UI: Replace stacked Dag header banners with inline status (#74516) +- TS SDK: Reject ``triggerDagRun`` tasks in ``dag.if`` and ``dag.switch`` (#74492) - Make airflow standalone discover new Dag files within seconds (#74479) - Use the configured API port in the default execution API URL (#74478) - Dag run breadcrumb switcher (#74471) +- Reduce db clean batch probe overhead (#74455) - UI: Show important form inputs instead of hiding them in accordions (#74422) - Speed up Migration 0136 on MySQL (#74409) - Scope the mapped task group skip-decision query to its own Dag run (#74390) -- Chore UI deps major upgrades (#74369) - Show loop iterations and outcomes throughout the UI (#74354) - Show current task executions in the UI (#74352) - Read current task executions by exact region and try (#74350) @@ -738,7 +812,7 @@ Miscellaneous - Scope plugin views on any entity field, not a fixed criteria set (#74278) - UI: Open provider connection fields by default and title them by connection type (#74273) - Keep get_dag_details and get_pools database queries off the API server event loop (#74247) -- Guard the Dag Code tab's Monaco grammar exports against drift (#74246) +- Require ``httpx2`` instead of ``httpx`` in airflow-core (#74244) - Scope audit log rows to a Dag only from the request path or body (#74232) - Register TypeScript, Java and Go grammars for the Dag Code tab (#74231) - Discover executable bundles by footer, not by execute bit (#74227) @@ -752,12 +826,15 @@ Miscellaneous - Parse native Lang-SDK Dags in the Dag processor with their runtime (#74035) - Persist the match-anywhere search toggle in the URL (#74019) - UI: Keep Dags list table rows on a single line (#74002) -- Pr/split storage tab (#73989) +- UI: Split task storage into separate XCom and task state store tabs (#73989) - Surface Dag run conf in the header (#73987) - Keep agent framework spans when worker tracing is off (#73903) - Enforce execution_timeout from the task supervisor side (#73806) - Keep the Dag asset schedule the same shape for every caller (#73805) - Update supported Kubernetes versions (#73798) +- Mask provider-declared sensitive connection extras on every connection type (#73794) +- Mask registered secrets at any nesting depth (#73793) +- Mask XCom values in the audit log (#73791) - Always clear the session cookie on logout (#73788) - Eager-load DagCode in the Dag source API (#73752) - Speed up zip Dag discovery and keep member file names (#73727) @@ -765,6 +842,7 @@ Miscellaneous - Correct the meaning of _arg_bindings in the serialization schema (#73644) - Sync the UI Dags list applied sort rule with URL query param (#73622) - Link calendar cell to filter dagruns by the respective start_date and state. (#73611) +- Add the resume event to stored next_kwargs without decoding them (#73576) - Scope plugin scheduling classes to their team in multi-team mode (#73569) - Warn when db clean skips only some rows of a batch (#73568) - Widen revoked_token.jti to store external-issuer token identifiers (#73562) @@ -784,21 +862,22 @@ Miscellaneous - Identity-only DAG discovery and byte-importing (#73118) - Move retry policy decision outside post execute log group block (#73023) - Speed up the event logs API by loading only display names (#73021) +- Scope upstream asset events of a Dag run to assets and Dags the caller may read (#72924) - Warn at provider discovery about an unresolvable connection type (#72904) - Mark task-group aggregate bars on Gantt with a diagonal stripe (#72894) - Hide unreadable assets in the asset expression served by next_run_assets (#72864) - Scope UI next_run_assets endpoint to assets the caller may read (#72862) - Keep date-only range ends inclusive (#72860) -- Require explicit limits for single-row lookups in core (#72856) - Remove a misleading connection type warning and explain lookup failures (#72855) - Keep callback triggers assigned to healthy triggerers (#72848) - Remove ignored --output flag from airflow pools set and delete (#72833) - Show whether a calendar day's run was a backfill (#72782) -- Replace AssertionError as ValueError in production code (#72763) - Keep the task instance in dag callbacks when its Dag version is missing (#72721) -- Keep provider_config_fallback_defaults.cfg in sync with provider.yaml (#72549) +- Read XCom without object reconstruction during dependency evaluation (#72718) +- Mask CLI secrets in the audit log by parsed argument, not argv position (#72650) +- Clarify the backfill reprocessing option labels in the CLI and REST API (#72590) - Warn when connections/variables list output may be incomplete (#72484) -- Adjust warnings, import errors, and Dag source models for sdk/importers (#72474) +- Add a source reference to import errors and a language field to Dag code (REST API + schema) (#72474) - Distinguish empty plugin folder from plugin load failures in logs (#72432) - Preview JSON fields as inline badges in tables (#72353) - More human readable duration strings (#72350) @@ -827,6 +906,7 @@ Miscellaneous - Reject serde-reserved keys in HITL params_input (#71054) - Update grid view refresh (#71004) - Improve assets list performance (#70997) +- Batch triggerer unused trigger cleanup deletes (#70961) - Multi-Team: Add teams verify command to validate multi-team configuration (#70947) - Describe default Airflow loggers (#70842) - Rename dag serialization metrics. (#70838) @@ -862,7 +942,7 @@ Miscellaneous - Display owner_display_name in Audit Log (#68833) - Warn in UI when the same dag_id is registered from multiple files (#68791) - Ignore stale executor success when TI is queued after defer (#68741) -- ``feature/issue-68575``: Add ``bundle_name`` to ``Dag`` (#68583) +- Add ``bundle_name`` to ``Dag`` (#68583) - Optionally emit Dag tags as metric tags (#68568) - Group and filter Dags by folder on the Dags page (#68544) - TriggerRunner: Add batch trigger creation duration metric (#68521) @@ -881,20 +961,33 @@ Miscellaneous Doc Only Changes ^^^^^^^^^^^^^^^^ -- Complete Hindi (hi) UI translations (#75183) -- Document that the HITL operator body is rendered as Markdown (#75136) -- Fix missing and incorrect endpoints in the API permission reference (#75099) -- Add documentation for the new YAML Dag format (#75053) -- Add missing zh-CN translations for new UI strings (#75023) -- Close German Translation Gaps 2026-10-15 (#74990) -- Point the TaskFlow tutorial's retries section to resumable tasks (#74984) -- Close German Translation Gaps 2026-10-14 (#74962) -- Document that ``use_job_schedule`` also stops asset-triggered Dag runs (#74951) -- Complete Hebrew UI translations 2026-10-16 (#74855) +- Go SDK: Correct the documented install command that broke the first bundle build (#75574) +- Add missing German Dag trigger translations (#75519) +- Normalize zh-TW plural forms and wording (#75504) +- Limit the CLI reference's page contents to top-level commands (#75483) +- Fix docs pages listing their sections as separate pages in the sidebar (#75478) +- Fix upgrade guide sections showing as separate pages in the docs sidebar (#75468) +- Add missing zh-TW translations for Dag version path tooltips (#75450) +- Java SDK: Rewrite the user docs around Dag definition (#75439) +- Fix deferral paths in the task lifecycle diagram (#75428) +- Add missing Taiwanese Mandarin translations for Dag start and end date trigger messages (#75371) +- Complete German UI translations (#75301) +- Add missing Taiwanese Mandarin UI translations (#75278) +- Close German translation gaps (#75137) +- Document that the HITL operator body is rendered as Markdown (#75130) +- Complete Hindi (hi) UI translations (#75057) +- Fix missing and incorrect endpoints in the API permission reference (#74871) +- Close German Translation Gaps 2026-10-15 (#74802) +- Close German Translation Gaps 2026-10-14 (#74760) +- Add documentation for the new YAML Dag format (#74748) +- Go SDK: Rewrite the user docs around Dag definition (#74698) +- TS SDK: Align the README with the authoring guide (#74693) +- Point the TaskFlow tutorial's retries section to resumable tasks (#74608) - Add missing Taiwanese Mandarin translations (#74545) - Add missing Polish UI translations (#74533) - Add missing Taiwanese Mandarin translations for task states and Dag list settings (#74486) - Fix Quick Start installing Airflow 3.1.1 instead of the documented version (#74474) +- Complete Hebrew UI translations 2026-10-16 (#74467) - Add missing UI translations for Greek(el) (#74460) - Dedent task docstrings used as doc_md (#74436) - Close Catalan gap (#74361) @@ -912,7 +1005,9 @@ Doc Only Changes - Close German Translation Gaps 2026-09-30 (#73980) - Clarify Dag processor scoping and team config in Multi-Team docs (#73958) - Fix typo in apache-airflow-providers-fab error message (#73956) +- Document that ``use_job_schedule`` also stops asset-triggered Dag runs (#73952) - Fix misleading timetable Variable example in best practices (#73945) +- Add missing zh-CN translations for new UI strings (#73943) - Update the TypeScript SDK docs for Dags declared in TypeScript (#73875) - i18n(ko): Align particles after Dag with the agreed pronunciation (#73799) - i18n(ko): Add missing translations (Sep25) (#73757) @@ -952,6 +1047,7 @@ Doc Only Changes - Improve TypeScript SDK API reference navigation (#71833) - Close spanish translation gap (#71796) - i18n(Ko): add missing translations in assets.json (Aug 18) (#71760) +- Add a consolidated language SDK compatibility matrix (#71757) - Fix inverted docstring for config write hide_sensitive option (#71623) - Fix the translation Gap for Taiwanese Mandarin (#71482) - Update Deadline Reference docs to match the code (#71469) diff --git a/airflow-core/newsfragments/74896.bugfix.rst b/airflow-core/newsfragments/74896.bugfix.rst deleted file mode 100644 index 0ea1420f44e..00000000000 --- a/airflow-core/newsfragments/74896.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -The REST API now rejects a pool name or variable key that is empty or only whitespace. ``POST /api/v2/pools`` and ``POST /api/v2/variables`` return 422 for such a value, and ``PATCH /api/v2/pools/{pool_name}`` returns 400 instead of renaming the pool to the empty string. Pools and variables that already have a blank name can still be read and deleted. diff --git a/airflow-core/newsfragments/75414.significant.rst b/airflow-core/newsfragments/75414.significant.rst deleted file mode 100644 index 8feb51b2682..00000000000 --- a/airflow-core/newsfragments/75414.significant.rst +++ /dev/null @@ -1,20 +0,0 @@ -``airflow.utils.email`` is deprecated in favor of the SMTP provider - -The functions in ``airflow.utils.email`` (``send_email``, ``send_email_smtp``, ``send_mime_email``, -``build_mime_message`` and ``get_email_address_list``) now live in -``airflow.providers.smtp.utils.emailer``. The old names still work and raise a ``DeprecatedImportWarning``. -Airflow now requires ``apache-airflow-providers-smtp>=3.2.0``. - -Keep ``[email] email_backend`` set to ``airflow.utils.email.send_email_smtp``. Airflow treats that value as -"send task emails with ``SmtpNotifier``", which reads all SMTP settings from the Connection. Changing it to -the provider path sends through the ``[smtp]`` settings instead. - -If you call these functions directly: - -* The SMTP Connection is looked up with ``BaseHook.get_connection``. A script that runs outside Airflow's - processes needs the Connection in an environment variable or a secrets backend, not only in the metadata - database. -* Settings are read through the Task SDK configuration, so values changed at runtime with - ``airflow.configuration.conf.set()`` are not seen. ``airflow.cfg`` and environment variables work as before. -* Tests that patch ``airflow.utils.email.send_mime_email`` no longer intercept the call made by - ``send_email_smtp``. Patch ``airflow.providers.smtp.utils.emailer.send_mime_email`` instead. diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt index 8df92f5f9ed..e66f9466c55 100644 --- a/docs/spelling_wordlist.txt +++ b/docs/spelling_wordlist.txt @@ -138,6 +138,7 @@ autocomplete autodetect autogenerated automl +autoregistration Autoscale autoscale autoscaled @@ -1141,6 +1142,7 @@ mTLS multi-cloud multimodal multiprocess +multiselect mutex mv mwaa diff --git a/reproducible_build.yaml b/reproducible_build.yaml index 9aaa1727bab..3889c951f39 100644 --- a/reproducible_build.yaml +++ b/reproducible_build.yaml @@ -1,2 +1,2 @@ -release-notes-hash: 20fa29512a9fcb66caeeb13a65a38fe2 -source-date-epoch: 1792517881 +release-notes-hash: 581265b7dbb2f8ffca89b968b5f1f1f6 +source-date-epoch: 1793020461
