This is an automated email from the ASF dual-hosted git repository. vatsrahul1001 pushed a commit to branch v3-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 7318cd901cbaa1345e29e1ec3209a860ad78ca06 Author: Rahul Vats <[email protected]> AuthorDate: Wed Jun 17 01:08:37 2026 +0530 Add 3.3.0 release notes --- RELEASE_NOTES.rst | 475 +++++++++++++++++++++ airflow-core/newsfragments/54505.significant.rst | 61 --- airflow-core/newsfragments/55850.significant.rst | 13 - airflow-core/newsfragments/56866.significant.rst | 38 -- airflow-core/newsfragments/57069.significant.rst | 16 - airflow-core/newsfragments/58337.feature.rst | 1 - airflow-core/newsfragments/58524.significant.rst | 7 - airflow-core/newsfragments/58992.significant.rst | 43 -- airflow-core/newsfragments/59239.feature.rst | 1 - airflow-core/newsfragments/59780.significant.rst | 9 - airflow-core/newsfragments/59785.significant.rst | 10 - airflow-core/newsfragments/59835.significant.rst | 6 - airflow-core/newsfragments/59855.significant.rst | 1 - airflow-core/newsfragments/59880.bugfix.rst | 1 - airflow-core/newsfragments/59938.bugfix.rst | 1 - airflow-core/newsfragments/60268.improvement.rst | 1 - airflow-core/newsfragments/60619.significant.rst | 22 - airflow-core/newsfragments/60803.significant.rst | 1 - airflow-core/newsfragments/60804.feature.rst | 1 - airflow-core/newsfragments/60921.significant.rst | 51 --- airflow-core/newsfragments/60951.significant.rst | 7 - airflow-core/newsfragments/61153.significant.rst | 19 - airflow-core/newsfragments/61274.improvement.rst | 1 - airflow-core/newsfragments/62055.significant.rst | 16 - airflow-core/newsfragments/62343.feature.rst | 1 - airflow-core/newsfragments/62344.feature.rst | 1 - airflow-core/newsfragments/62369.significant.rst | 30 -- airflow-core/newsfragments/62487.significant.rst | 1 - airflow-core/newsfragments/62561.bugfix.rst | 1 - airflow-core/newsfragments/62964.bugfix.rst | 1 - airflow-core/newsfragments/63141.bugfix.rst | 1 - airflow-core/newsfragments/63205.bugfix.rst | 1 - airflow-core/newsfragments/63266.bugfix.rst | 1 - airflow-core/newsfragments/63296.significant.rst | 16 - airflow-core/newsfragments/63312.feature.rst | 1 - airflow-core/newsfragments/63365.significant.rst | 9 - airflow-core/newsfragments/63452.significant.rst | 7 - airflow-core/newsfragments/63664.bugfix.rst | 1 - airflow-core/newsfragments/63878.bugfix.rst | 1 - airflow-core/newsfragments/63884.significant.rst | 34 -- airflow-core/newsfragments/64067.bugfix.rst | 1 - airflow-core/newsfragments/64207.significant.rst | 1 - airflow-core/newsfragments/64232.feature.rst | 1 - airflow-core/newsfragments/64503.bugfix.rst | 1 - airflow-core/newsfragments/64552.improvement.rst | 1 - airflow-core/newsfragments/64571.significant.rst | 61 --- airflow-core/newsfragments/64751.feature.rst | 1 - airflow-core/newsfragments/64822.significant.rst | 1 - airflow-core/newsfragments/65269.significant.rst | 1 - airflow-core/newsfragments/65346.significant.rst | 3 - airflow-core/newsfragments/65422.bugfix.rst | 1 - airflow-core/newsfragments/65590.feature.rst | 1 - airflow-core/newsfragments/66015.significant.rst | 3 - airflow-core/newsfragments/66030.feature.rst | 1 - airflow-core/newsfragments/66161.significant.rst | 12 - airflow-core/newsfragments/66210.bugfix.rst | 1 - airflow-core/newsfragments/66412.significant.rst | 26 -- airflow-core/newsfragments/66499.significant.rst | 14 - airflow-core/newsfragments/66584.feature.rst | 1 - airflow-core/newsfragments/66737.significant.rst | 1 - airflow-core/newsfragments/66807.feature.rst | 1 - airflow-core/newsfragments/66848.feature.rst | 1 - airflow-core/newsfragments/67056.significant.rst | 101 ----- airflow-core/newsfragments/67164.significant.rst | 34 -- airflow-core/newsfragments/67184.feature.rst | 1 - airflow-core/newsfragments/67288.improvement.rst | 1 - airflow-core/newsfragments/67475.feature.rst | 1 - airflow-core/newsfragments/67494.significant.rst | 43 -- airflow-core/newsfragments/67523.feature.rst | 1 - airflow-core/newsfragments/67586.significant.rst | 4 - airflow-core/newsfragments/67684.bugfix.rst | 1 - airflow-core/newsfragments/67688.improvement.rst | 1 - airflow-core/newsfragments/67716.feature.rst | 1 - airflow-core/newsfragments/67868.bugfix.rst | 1 - airflow-core/newsfragments/67873.bugfix.rst | 1 - airflow-core/newsfragments/68028.feature.rst | 1 - airflow-core/newsfragments/68175.significant.rst | 24 -- airflow-core/newsfragments/68266.bugfix.rst | 1 - .../continuous-optional-start-date.improvement.rst | 1 - reproducible_build.yaml | 4 +- 80 files changed, 477 insertions(+), 789 deletions(-) diff --git a/RELEASE_NOTES.rst b/RELEASE_NOTES.rst index 0a6966cf598..6f218013b26 100644 --- a/RELEASE_NOTES.rst +++ b/RELEASE_NOTES.rst @@ -24,6 +24,481 @@ .. towncrier release notes start +Airflow 3.3.0b1 (2026-06-29) +---------------------------- + +Significant Changes +^^^^^^^^^^^^^^^^^^^ + +Asset Partitioning +"""""""""""""""""" + +Building on the asset partitioning introduced in 3.2.0, Airflow 3.3.0 substantially expands how a +single upstream asset event fans out to partitioned downstream Dag runs. New partition mappers — +``FanOutMapper`` (one-to-many), and ``FixedKeyMapper`` + ``SegmentWindow`` (categorical rollup) — join +the existing ``RollupMapper``, and compose with time windows (day/week/month/quarter/year) and a +``wait_policy`` (``WaitForAll`` or ``MinimumCount(n)``) to control when partitioned runs fire. Windows +can fan out forward or backward in time, and total fan-out per upstream event is bounded by the new +``[scheduler] partition_mapper_max_downstream_keys`` config (configurable per mapper). +(#66030, #66848, #67184, #67475, #67716) + +For detailed usage instructions, see :doc:`/authoring-and-scheduling/assets`. + +Task and Asset State Store +"""""""""""""""""""""""""" + +Airflow 3.3.0 introduces a first-class state store for tasks and assets (AIP-103). Tasks can persist +arbitrary key-value state that survives across retries and runs via a new ``task_state_store`` accessor, and +assets can carry their own state via ``asset_state_store`` — both available from the Task SDK. State is kept +in the metadata database by default, or in a custom worker-side backend (``[workers] +state_store_backend``), supports per-key retention with periodic garbage collection and an optional +``clear_on_success``, and is fully manageable through the Core API and Execution API. +(#65759, #66073, #66160, #66463, #66586, #66859, #67041, #67319) + +.. warning:: + + Task and asset state storage is experimental in 3.3.0 and may change in future versions based on + user feedback. + +Pluggable Retry Policies +"""""""""""""""""""""""" + +Task retry behaviour is now pluggable (AIP-105). In addition to a fixed ``retries`` count, you can +attach a custom retry policy that decides whether and when a task is retried, enabling strategies such +as retrying only on specific exceptions or backing off based on custom logic. (#65474) + +Language Task SDK (Java and Go) +""""""""""""""""""""""""""""""" + +Airflow 3.3.0 adds a Coordinator layer (AIP-108) that lets individual task implementations be written +in non-Python languages while the Dag and its scheduling stay in Python. A task is declared in the Dag +with ``@task.stub(queue=...)``; the worker routes it to a configured coordinator +(``JavaCoordinator`` for JVM languages, ``ExecutableCoordinator`` for self-contained native binaries +such as Go) that runs the task in a language runtime and proxies Variables, Connections, and XComs back +through the Execution API. (#65958, #67118, #67161, #67635, #67699) + +.. warning:: + + The Coordinator layer and the Java/Go SDKs are experimental in 3.3.0 and may change in future + versions based on user feedback. + +DAG bundle version on clear, rerun, and backfill +"""""""""""""""""""""""""""""""""""""""""""""""" + +The new ``rerun_with_latest_version`` setting controls whether a cleared, rerun, or backfilled Dag run +uses the latest bundle version or the original version from the initial run. The default is resolved +by precedence: an explicit request parameter/CLI flag, then the DAG-level ``rerun_with_latest_version``, +then ``[core] rerun_with_latest_version``, and finally ``False`` for clear/rerun and ``True`` for +backfills (preserving historical behaviour). Airflow 2.x always reran with the latest code; 3.x +introduced bundle versioning defaulting to the original version, and this setting gives users control. +(#63884) + +See :doc:`/administration-and-deployment/dag-bundles` for full details. + +Provider example DAGs as dedicated bundles +"""""""""""""""""""""""""""""""""""""""""" + +Example DAGs shipped by provider distributions are now discovered via ``ProvidersManager`` and +registered as their own Dag bundles — one per provider, named +``apache-airflow-providers-<distribution>-example-dags`` (or ``<distribution>-example-dags`` for +third-party providers). The ``[core] load_examples`` option still gates whether they are registered. +REST API clients that filtered ``bundle_name`` by ``"dags-folder"`` for provider-shipped example DAGs +must update to the new per-provider bundle names; DAG identifiers are unchanged. (#66161) + +Remote logging resolution decoupled from ``airflow.logging_config`` +""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +Remote task log handler resolution is now owned by the shared ``airflow_shared.logging.factory`` +module and applies a single, well-defined precedence: + +1. a user-defined ``[logging] logging_config_class`` exporting ``REMOTE_TASK_LOG`` / + ``DEFAULT_REMOTE_CONN_ID`` (existing custom configs keep working); +2. ``ProvidersManager`` scheme dispatch — the scheme of ``[logging] remote_base_log_folder`` selects a + provider ``RemoteLogIO`` class, instantiated via a no-argument ``from_config()`` classmethod; +3. a transitional legacy fallback reading ``airflow_local_settings.py`` (to be removed in Airflow 4.0). + +``airflow.logging_config.load_logging_config`` is deprecated (it now emits ``DeprecationWarning`` and +delegates to new private helpers), and ``configure_logging`` no longer eagerly resolves the remote +handler — resolution is lazy on first use. Providers that registered a ``remote-logging:`` block but do +not implement ``from_config`` are skipped with a warning and fall through to the legacy path. + +Migration: replace direct calls to ``airflow.logging_config.load_logging_config()`` with the new +helpers, and have provider remote-log handler classes implement a no-argument ``from_config`` +classmethod that reads ``airflow.providers.common.compat.sdk.conf``. (#67056) + +OpenTelemetry timer metrics now use Histogram +""""""""""""""""""""""""""""""""""""""""""""" + +OpenTelemetry timer and timing metrics are now recorded as Histograms instead of Gauges, preserving +count, sum, and bucket distribution across recordings. (#64207) + +DAG-processing "seconds ago" metric is now tagged +""""""""""""""""""""""""""""""""""""""""""""""""" + +``dag_processing.last_run.seconds_ago.{dag_file}`` is now a legacy metric. The new +``dag_processing.last_run.seconds_ago`` is emitted with ``file_path``, ``bundle_name`` and +``file_name`` tags (``file_path`` + ``bundle_name`` uniquely identify the Dag file). The legacy metric +is still emitted by default and can be disabled via ``[metrics] legacy_names_on``. (#62487) + +New Deadlines page under Browse +""""""""""""""""""""""""""""""" + +A new **Deadlines** page is available under the Browse menu, accessible to any role that already has +``can_read`` and ``menu_access`` on **DAG Runs**. (#67586) + +Airflow CLI commands now talk to the API server +""""""""""""""""""""""""""""""""""""""""""""""" + +Airflow CLI commands are migrating to reach Airflow through the API server (via the ``airflowctl`` +client) instead of the metadata database directly. Migrated so far: ``dags trigger``, ``dags delete``, +``pools`` (list/get/set/delete/import/export), and ``assets materialize``. These commands now require a +reachable API server and mint a short-lived token in memory (set ``AIRFLOW_CLI_TOKEN`` where an auth +manager cannot mint locally). ``airflow.api.client`` is removed — use +``airflow.cli.api_client.get_cli_api_client``. Each migrated command emits a +``RemovedInAirflow4Warning``; use the equivalent ``airflowctl`` command instead. (#68175) + + +New Features +^^^^^^^^^^^^ + +- UI: Add team name to the asset graph view (#68457) +- UI: Add a column to the asset store table linking to the task instance that wrote it (#68395) +- UI: Add a custom expiration datetime picker for the task store modal (#68394) +- UI: Add additional task instance attributes to the task instance details section (#68378) +- UI: Add a Details tab to the mapped task instance view (#68340) +- UI: Add bulk marking of Dag runs as success or failed from multi-select (#68278) +- Add ``--team-name`` support to the pool CLI commands (#68110) +- UI: Add a full-screen toggle to the code viewer (#68044) +- Add API endpoint support for consumer team asset filtering (#68034) +- UI: Add bulk clear selection for task instances (#68029) +- Add ``awaiting_input`` task state for Human-in-the-Loop, running off the triggerer (#68028) +- Add bulk API to mark Dag runs as success or failed (#67948) +- Record writer info for every asset store write for better cross-linkage (#67902) +- Register XCom ``output_type`` classes from a worker-side DAG walk (#67875) +- Add ``FixedKeyMapper`` and ``SegmentWindow`` for categorical asset-partition rollup (#67716) +- Add a bulk ``POST /dags/{dag_id}/clearDagRuns`` API endpoint (#67709) +- Return Pydantic model instances through XCom for structured output (#67644) +- Add the ability to apply a note when clearing a Dag run or task instances (#67639) +- Add ``consumer_teams`` to ``AssetAccessControl`` in the Task SDK (#67625) +- Populate trigger ``team_name`` at creation time for multi-team support (#67605) +- UI: Add bulk Clear on the Dag Runs list page (#67564) +- Add multi-team query filtering to triggerer trigger assignment (#67517) +- Add forward fan-out support via the ``forward`` kwarg on ``Window`` (AIP-76) (#67475) +- Add patch task state API and ``expires_at`` support in the set API (AIP-103) (#67319) +- Add a ``team_name`` column to the trigger table for multi-team triggerer support (#67305) +- UI: Add asset and task store views (#67292) +- Add a ``--team-name`` CLI argument to the triggerer for multi-team (#67254) +- Add an ``allow_global`` option to asset access control (#67251) +- Add mTLS and private CA support to the API client and server (#67214) +- Add Markdown documentation support for TaskGroups (#67207) +- Add a per-mapper ``max_fan_out`` override for partition fan-out cap (#67184) +- Add timezone support to the SDK temporal partition mappers (#67164) +- Add ``ResumableJobMixin`` with ``SparkSubmitOperator`` for surviving worker failures (#67118) +- Add bulk delete for Dag runs (#67095) +- Add a ``nav_top_level`` option for plugin nav items (#67084) +- Add Core API endpoints for task state and asset state (AIP-103) (#67041) +- Replace ``allow_producer_teams`` with ``access_control`` on ``Asset`` (#66954) +- Add worker-side custom state backend support (AIP-103) (#66859) +- Let partitioned Dag runs fire on a partial upstream window with ``wait_policy`` (#66848) +- Consume task-emitted partition keys on asset events (AIP-76) (#66782) +- Add per-task state key retention from operators (AIP-103) (#66699) +- Add a ``callback_execution_timeout`` config for deadline callbacks (#66609) +- Add a ``clear_on_success`` config to wipe task state on success (AIP-103) (#66586) +- Add a ``partitions clear`` CLI command to reset DagRun partition fields (#66520) +- Make CORS ``allow_credentials`` configurable (#66503) +- Add periodic task state garbage collection and retention support (AIP-103) (#66463) +- Add URI sanitizers and asset factories for new schemes (#66426) +- Add a ``teams sync`` CLI command (#66418) +- Add remote log upload support for callback subprocesses (#66379) +- Add by-name/by-uri asset state routes and ``AssetUriRef`` support (AIP-103) (#66336) +- UI: Add support for rendering multi-type params (#66278) +- Filter Dags by teams when registering asset changes (#66168) +- Wire up Task SDK communication and context access for task/asset state (AIP-103) (#66160) +- UI: Add marking a task group as success or failed (#66146) +- Add Execution API endpoints for task and asset states (AIP-103) (#66073) +- Add ``FanOutMapper`` for one-to-many partition fan-out (#66030) +- Add ``Variable.keys()`` to list variable keys by prefix in the Task SDK (#66022) +- Add an ``airflow dags clear`` command for partition-range reprocessing (#66004) +- Add a ``memray_detailed_tracing`` option for deeper memory profiling (#65996) +- UI: Add support for different graph directions in the asset graph view (#65948) +- UI: Add a Clear All Mapped Tasks button (#65813) +- Add ``allow_producer_teams`` to the Asset SDK class (#65790) +- UI: Show expected duration based on historical average in Dag Run details (#65722) +- Add team name to the task context (#65617) +- Add an ``on_kill()`` hook to ``BaseTrigger`` to handle user actions on triggers (#65590) +- Allow accessing a DAG's members via ``[]`` (#65586) +- Add pluggable retry policies for Airflow tasks (AIP-105) (#65474) +- Add support for ``format="Duration"`` in params (#65469) +- Add the ``PartitionAtRuntime`` authoring API to the Task SDK (AIP-76) (#65447) +- UI: Add pagination to the grid view (#65388) +- Add ``partition_key`` to the task context (#65359) +- Make the blocked-thread warning threshold configurable (#65009) +- Add name fields to SDK deadline alerts (#64926) +- Add dynamic interval resolution support via Variables for deadline alerts (#64751) +- Add an ``is_backfillable`` property to DAG API responses (#64644) +- Return dag-specified results in the dag run wait API (#64577) +- Hold a Dag run until all upstream partitions arrive (AIP-76) (#64571) +- Add a way to mark a return-value XCom as the dag result (#64522) +- Allow accessing a TaskGroup's members via ``[]`` (#64430) +- UI: Redo the Gantt chart (#64335) +- UI: Add task-level filters to the DAG graph tab (#64271) +- UI: Add bulk Clear, Mark Success/Fail, and delete for multiple task instances (#64141) +- Check that multi-team is enabled when a team name is provided to the API (#63994) +- Add a ``DagRunType`` for operators (#63733) +- UI: Add search functionality to the task log viewer (#63467) +- Add patching of task group instances in the API (#62812) +- Add deadlines API endpoints (#62583) +- Add async connection testing via workers for security isolation (#62343) +- Add ``run_after`` to ``TriggerDagRunOperator`` (#62259) +- UI: Display deadlines on the Dag Run and Overview tabs (#62195) +- Re-enable the ``start_from_trigger`` feature with template-field rendering (#55068) +- Backfill partitioned Dags by partition-date range (#67537) +- Add a producer-side acknowledgement channel to shared-stream triggers (#67523) + +Bug Fixes +^^^^^^^^^ + +- Fix Stats not being initialized in the API server lifespan (#68514) +- Fix ``BackfillDagRun.partition_key`` type annotation (#68432) +- Fix backward compatibility for ``DagRunInfo`` partition fields (#68342) +- Fix ``airflow db clean`` failing on foreign-key-referenced ``dag_version`` rows (#68339) +- Fix 500 error when listing event logs with a NULL timestamp (#68338) +- Fix MySQL downgrade from 3.3.0 for the ``deadline_alert.interval`` JSON conversion (#68337) +- Fix older and custom secrets backends breaking on Airflow 3.2 (#68302) +- Fix secrets backend connection errors being silently swallowed at DEBUG level (#68301) +- UI: Fix the instance name title shown on non-Dag pages (#68288) +- Fix scheduler not populating ``partition_date`` for temporal asset partitions (#68266) +- UI: Fix wrong language being auto-detected from browser preferences (#68258) +- Honor ``retry_policy`` on non-deferrable ``TriggerDagRunOperator`` wait failures (#68254) +- Fix scheduler crash loop when the last task instance predates Dag versioning (#68253) +- Fix team consumer asset filtering (#68242) +- UI: Fix sluggish multi-selection behavior in tables (#68229) +- UI: Fix mapped task instance links for tasks without a start date (#68194) +- Fix setup/teardown auto-inclusion when clearing or marking tasks (#68193) +- UI: Remove redundant columns from the XCom panel on the task instance page (#68188) +- UI: Fix Gantt tooltip showing the wrong start date on queued/scheduled segments (#68176) +- Fix Java SDK coordinator rejecting IPv4-mapped IPv6 connections (#68169) +- Fix Java SDK tasks being rejected by the coordinator connection-ownership check (#68147) +- UI: Fix language key for the Dag bundle filter (#68131) +- Fix ``DagFileProcessorManager`` silent hang on database lock contention (#68118) +- Mask all connection extra and variable values in the API audit log (#68049) +- Fix spurious "Failed to detach context" error on Execution API disconnects (#68039) +- UI: Fix Dag code highlighting for triple-quoted and escaped-brace f-strings (#68026) +- Fix cursor encoding for column-form sort parameters in the REST API (#67973) +- Fix ``SimpleAuthManager`` not preserving the deep-link next URL on first login (#67965) +- Guard the task stats emission to prevent errors (#67955) +- UI: Fix task instance state badge staying stale after a Mark-as action (#67950) +- Register nested Pydantic models for XCom deserialization (#67932) +- Fix ``example_asset_store`` consumer crash (#67922) +- Raise ``InvalidJwtError`` in ``JWTValidator.avalidated_claims()`` when the key ID does not match (#67909) +- Fix Kubernetes executor ``pod_override`` being stringified without the cncf provider (#67895) +- UI: Prevent duplicate task instance summary stream refreshes after mutations (#67892) +- Reject negative ``default_retention_days`` in the Task SDK and core API routes (#67890) +- Fix ``none_failed_min_one_success`` trigger rule checks (#67873) +- Remove trigger kwargs from the REST API response (#67868) +- UI: Fix long parameter names overflowing the Trigger DAG modal (#67859) +- Fix misleading log message in the task runner clear-on-success block (#67836) +- Fix scheduler crash when logging orphaned task resets (#67822) +- UI: Fix dashboard pool summary showing incorrect deferred slot usage (#67818) +- Fix trigger datetime deserialization (#67795) +- UI: Fix Graph layout for TaskGroup tasks wired to external nodes (#67720) +- Fix ``airflow dags clear`` clearing the wrong day for non-UTC partitioned timetables (#67717) +- Fix per-index evaluation of ``ONE_FAILED`` in mapped task groups (#67684) +- UI: Fix dialog dismissal for the Chakra upgrade (#67674) +- UI: Hide dashboard metric percentages when a state count is capped (#67664) +- Apply per-file authorization to the dag-source endpoint (#67662) +- Fix ``airflow dags next-execution --table`` crash when no next run exists (#67642) +- UI: Fix the time picker omitting seconds (#67636) +- Filter scheduling-dependencies graph edges by readable-DAG access (#67627) +- Mask per-key secrets-backend-kwarg overrides on the Config API (#67622) +- Fix ``GET /auth/login`` missing a 400 response in the OpenAPI spec (#67571) +- Fix ``GET /pools`` incorrectly documenting a 404 response in the OpenAPI spec (#67570) +- Add a compatibility layer for import errors caused by ``AirflowSecretsBackendAccessDenied`` (#67560) +- UI: Fix rendering of None child state (#67552) +- Fix sort order for mapped task instances (#67551) +- Fix import errors total-entries count with multiple DAGs per file (#67550) +- UI: Prefer active over queued state for collapsed groups (#67543) +- Fix callback state not updating from executor events due to a UUID type mismatch (#67542) +- Reject wildcard origin in CORS config instead of toggling credentials (#67502) +- Guard the finally-block logger in the HTTP access log middleware (#67501) +- Strip CR/LF from user-supplied logical date before logging (#67500) +- Redact secret-looking query parameters in the HTTP access log (#67498) +- UI: Fix Calendar view to respect the user-selected timezone (#67497) +- Escape LIKE wildcards in non-search filter parameters (#67496) +- Fix missing redaction of secret values in variable JSON (#67495) +- Fix bulk CREATE+OVERWRITE team-context authorization bypass (#67493) +- UI: Return 400 instead of 500 from ``structure_data`` on a malformed asset expression (#67489) +- Fix ``SimpleAuthManager`` redirect to the next URL after login (#67483) +- Return 400 instead of 500 from ``materialize_asset`` on invalid input (#67445) +- UI: Restore the Monaco find widget in the Dag Code view (#67391) +- UI: Fix an HTTPException import that turned a 400 into a 500 in the dags endpoint (#67363) +- Restore ``fail_fast`` handling when reschedule exceeds the MySQL ``TIMESTAMP`` limit (#67353) +- Fix the Triggered DAG button not being visible during queued/running state (#67327) +- Fix variables import with structured falsy values (#67060) +- Avoid logging Execution API bearer credentials (#67059) +- Sanitize Dag processor metric file names (#67029) +- Return a 422 when the database rejects an API payload (#66888) +- Prevent ``AlreadyRunningBackfill`` error caused by an invalid date range request (#66874) +- Restrict owner-link and extra-link ``href`` values to safe schemes (http, https, mailto, relative) (#66741) +- Add a session parameter to the ``BaseStateBackend`` interface to fix custom backends (#66708) +- Allow deadline callbacks within the same DAG module (#66702) +- UI: Fix relative React plugin bundle URLs in dev mode (#66618) +- Validate DAG trigger conf as a JSON object or null (#66617) +- Require a trust sentinel for ``state.user`` injection in ``get_user()`` (#66562) +- Use ``hmac.compare_digest`` for ``SimpleAuthManager`` password comparison (CWE-208) (#66556) +- Set ``SameSite=Lax`` on the ``SimpleAuthManager`` all-admins login cookie (#66502) +- Reserve ``/auth`` and ``/pluginsv2`` from plugin URL prefixes (#66501) +- Use a cryptographically secure RNG for ``SimpleAuthManager`` passwords (#66500) +- Fix Triggerer ``runner_health_check_threshold`` log formatting (#66486) +- Fix Dag processor callback cleanup for versioned bundle files (#66484) +- Default ``AIRFLOW_UID`` to 50000 in the airflow-init chown lines (#66481) +- Strip CR/LF from MySQL URL query values before forwarding to ``my.cnf`` (#66325) +- Fix ``CronMixin`` not resolving cron presets before validation (#66102) +- Fix ``AirflowSDKConfigParser`` missing the ``mask_secrets`` method (#66077) +- Fix ``resolve_xcom_backend`` to rely on the config schema default (#65938) +- Fix a missing import cast error in the dag_run API route (#65748) +- Mask Dag processor connection and variable responses (#65704) +- UI: Show import error for deactivated DAGs (#65687) +- Forward MySQL SSL params from ``sql_alchemy_conn`` to ``airflow db shell`` (#65575) +- Disable SQLite FK checks in the 0111 migration downgrade (#65545) +- Handle Variable values that cannot be decrypted gracefully in the stable REST API (#65452) +- Retry ``TriggerDagRunOperator`` when the triggered DagRun fails (#65390) +- Fix task run exceptions never being caught by Sentry (#65161) +- Fix the bulk task instance authorization error message rendering (#64719) +- Fix trigger template rendering failure when operator ``template_fields`` differ from trigger attributes (#64715) +- Fix ``task_defer`` with non-JSON ``next_kwargs`` in ``TaskInstance`` (#64714) +- Add the error as ``context["exception"]`` in ``InProcessTestSupervisor`` (#64568) +- Fix NPM security alerts in the simple auth manager (#64309) +- Fix DAG run trigger to surface errors instead of swallowing them (#64130) +- Fix Task SDK Connection extras built from a URI constructor (#64120) +- Add insert/update-on-conflict for rendered task instance fields (#63874) +- Fix ``timeout_with_traceback`` crashes on Windows and non-main threads (#63664) +- UI: Wrap long lines in the rendered templates view (#63492) +- Block path traversal via ".." in ``dag_id`` and ``run_id`` (#63296) +- Fix the scheduler health check command in ``docker-compose.yaml`` (#62280) +- Fix unmapped task deadlock when upstream tasks are removed (#62034) +- Forward termination signals from the supervisor to the task subprocess (#61627) +- Check destination team permission when using bulk APIs for connections, variables, and pools (#68573) +- Fix the execution API ``/health`` check failing on the empty-path route (#68578) +- Fix Dag run partition key filter breaking on composite keys containing ``|`` (#68459) +- Fix the partition clear date range for non-UTC partitioned timetables (#68460) +- Validate that partition keys are non-empty and within the column length (#68443) +- Fix the scheduler serving stale Dag code after an in-place serialized Dag version update (#68558) +- Determine the latest Dag version by version number to avoid collisions when timestamps tie (#68389) +- Fix new runs and reruns executing an outdated bundle version when the Dag serialization is unchanged (#68336) +- Fix remote logging from the task supervisor (#68370) +- Upload task logs even when the final state update fails (#67935) +- Escape URLs in the Task SDK client when looking up Dag operations (#68129) +- Fix task scheduling when multi-team is enabled (#68634) +- Fix secret values not being masked in rendered templates when keys use dot or dash separators (#68624) +- Fix ``jwt_audience`` for the public API being read from two different config sections (#67494) +- Fix duplicate deadline-miss callbacks firing from multiple HA scheduler replicas (#64737) +- Fix scheduler crash on non-ASCII Dag names when OpenTelemetry metrics are enabled (#68023) +- Fix Dag processor crash on non-ASCII names in OpenTelemetry gauge and timer metrics (#68284) +- Report duplicate plugin names as import errors instead of silently ignoring them (#66649) +- Require edit permission for async connection tests that update an existing connection (#68127) +- Restore the deprecated ``[core] execution_api_server_url`` mapping to ``[workers] execution_api_server_url`` (#63949) +- Fix ``dag.test()`` not re-syncing sibling Dags across repeated calls (#66205) +- UI: Invalidate per-attempt task instance caches after actions so logs and details are not stale (#67212) + +Miscellaneous +^^^^^^^^^^^^^ + +- Optimize bulk variable deletion to avoid N+1 queries (#68508) +- UI: Unify the Dag Code tab toolbar styling with the Logs toolbar (#68449) +- Optimize bulk Dag run authorization to avoid N+1 team-name queries (#68286) +- Improve ``airflow dags`` command to use bulk clear (#68280) +- Add the ``task_state_store`` table to the ``airflow db clean`` mechanism (#68218) +- Add metrics and traces to ``ResumableJobMixin`` for crash recovery (#68213) +- Improve ``ResumableJobMixin`` crash-recovery observability with better logging (#68206) +- Pass ``DagRun`` to ``task_instance_mutation_hook`` for run-aware task mutation (#68198) +- Add ``team_name`` to multi-team metrics (#68108) +- Reduce redundant Dag team lookups in authorization checks (#68020) +- Enhance ``ResumableJobMixin.get_job_status`` with context for better job status tracking (#68009) +- Propagate OpenTelemetry trace headers from the client to Execution API server-side spans (#67904) +- Widen the type hint for the ``DagRun.get_task_instances`` / ``fetch_task_instances`` state parameter (#67880) +- UI: Use the bulk clear Dag runs endpoint for bulk Dag run clear (#67846) +- Add a ``default`` parameter to the task and asset state ``get()`` method (#67842) +- Make core API routes for task and asset states interact only with the database (#67835) +- Optimize Dag processor file-queue deduplication from O(N^2) to O(N) (#67750) +- Add ``allow_consumer_teams`` and ``allow_global_consumers`` columns to ``TaskOutletAssetReference`` (#67730) +- Speed up the Dags list and dashboard queries on large ``DagRun`` tables (#67721) +- Make ``partition_key`` provenance-only and inherit it onto asset events (#67718) +- Speed up Dag serialization by skipping a redundant asset roundtrip (#67702) +- Cache ``BaseOperator.__init__`` signature in operator serialization (#67701) +- Optimize ``TaskGroup.topological_sort`` for reverse-declared Dags (#67688) +- Update serialization for producer-side asset access control (#67658) +- Allow outlets to be added and accessed in ``AssetStateAccessor`` (#67619) +- Unify task/asset state storage between the Core API and Execution API (#67547) +- Decorate custom state references with an envelope for UI clarity (#67530) +- Simplify authoring of task and asset states by allowing JSON types (#67418) +- Replace Sphinx Redoc with Swagger for the API docs (#67390) +- Emit OpenTelemetry spans around listener hook calls (#67347) +- UI: Update verbiage for lower-priority backfill runs (#67338) +- Fix N+1 query in the bulk task instance delete endpoint (#67304) +- Speed up ``TaskGroup.topological_sort`` with an int-indexed projected sweep (#67288) +- UI: Use react-query native error state for bulk action hooks (#67284) +- Wrap ``executor.heartbeat()`` in a timer to localize scheduler loop slowdowns (#66808) +- Emit ``dagrun.first_task_start_delay`` separately from scheduling delay (#66807) +- Share one poll loop across sibling event triggers (#66584) +- UI: Upgrade icons, spacing, and default component themes (#66569) +- Warn when ``SimpleAuthManager`` runs in a production-shaped deployment (#66563) +- Migrate Stackdriver logging config to the ``RemoteLogIO`` pattern (#66513) +- Add a ``BundleVersion`` dataclass and ``version_data`` persistence to ``DagVersion`` (#66491) +- Avoid lazy-loading timetable fields for latest DagRuns (#66488) +- Move ``allow_producer_teams`` to ``DagScheduleAssetReference`` (#66487) +- Pass user teams to the ``create_asset_event`` endpoint (#66367) +- Load ``USFederalHolidayCalendar`` lazily to reduce memory usage when loading examples (#66303) +- Propagate task OpenTelemetry trace context through IPC into Execution API requests (#66151) +- Surface worker DAG parse duration in the task log (#66138) +- Skip deserializing ``trigger_kwargs`` when loading serialized DAGs (#66002) +- Honor ``AUTH_ROLE_PUBLIC`` in the FastAPI API server (#65685) +- Add extended sysinfo for the Edge worker (#65472) +- Clarify logs when a DAG is being processed in the Dag processor (#65196) +- Add indexes on ``task_instance.dag_version_id`` and ``dag_run.created_dag_version_id`` (#64818) +- Improve creation of ``RuntimeTaskInstance`` in ``TriggerRunner`` for ``start_for_trigger`` functionality (#64298) +- Mark the Triggerer supervisor as a server context so it can read metastore connections (#64022) +- Load hook metadata from YAML without importing the hook class (#63826) +- Add detailed task spans (#63568) +- Downgrade logging on query JSON parsing and add a JSON load condition (#62044) +- UI: Add a Deadlines section with a time-range selector to the Dashboard page (#68038) +- UI: Add a modal for editing notes with Markdown support (#68362) +- UI: Improve the Human-In-The-Loop form UX (#68397) +- Add a ``team_name`` tag to executor metrics for multi-team deployments (#68593) +- Make task and asset state store row size limits configurable (#68133) +- UI: Add notification UX for Human-In-The-Loop actions (#68346) +- Allow synchronous deadline callbacks (``SyncCallback``) to access Connections and Variables (#65269) + +Doc Only Changes +^^^^^^^^^^^^^^^^ + +- Add a contributing guide for language SDKs (#68330) +- Add ``sdk.TIRunContext`` documentation for the Go SDK (#68319) +- Add a Go Task SDK authoring guide to the docs (#68223) +- Update supported-versions doc to mark 2.11.2 as EOL (#68212) +- Add documentation for ``ResumableJobMixin`` and resumable tasks (#68136) +- Add CLI examples for team-scoped pools (#68111) +- Add docs for multi-team triggerer support (#67608) +- Clarify trigger rule behavior for the ``removed`` upstream state (#67452) +- Fix outdated image links in ``dags.rst`` (#67357) +- Add an example and docs for runtime asset partitioning (AIP-76) (#67307) +- Add documentation for the Task and Asset Store (AIP-103) (#67299) +- Add a dynamic task mapping no-op example (#67022) +- Add documentation about adding ``access_control`` to the ``Asset`` object (#66949) +- Add a how-to for Dag-level retry via ``on_failure_callback`` (#66277) +- Fix documentation after PR 62645 (#65843) +- Add documentation for team-based asset event filtering (#65690) +- Document ``on_kill()``/``cleanup()`` for triggers (#65671) +- Explain ``xcom_pull`` behaviour without ``task_ids`` in the docs (#65406) +- Improve standalone authentication documentation for Airflow 3.x (#65330) +- Clarify manual DAG run data interval semantics in Airflow 3 (#64740) +- Document and test ``xcom_pull`` ``run_id`` usage for triggered DAG runs (#63030) +- Update params in the backfill documentation (#61821) +- Document the ``apache-airflow-mypy`` package in the core docs (#68561) +- Fix typos and formatting in the Fundamentals documentation (#68524) +- Complete the Hindi (``hi``) UI translation (#68574) +- Fill the Taiwanese Mandarin (``zh-TW``) UI translation gap (#68563) + Airflow 3.2.2 (2026-05-29) -------------------------- diff --git a/airflow-core/newsfragments/54505.significant.rst b/airflow-core/newsfragments/54505.significant.rst deleted file mode 100644 index 3289308fa0f..00000000000 --- a/airflow-core/newsfragments/54505.significant.rst +++ /dev/null @@ -1,61 +0,0 @@ -Move task-level exception imports into the Task SDK - -Airflow now sources task-facing exceptions (``AirflowSkipException``, ``TaskDeferred``, etc.) from -``airflow.sdk.exceptions``. ``airflow.exceptions`` still exposes the same exceptions, but they are -proxies that emit ``DeprecatedImportWarning`` so Dag authors can migrate before the shim is removed. - -**What changed:** - -- Runtime code now consistently raises the SDK versions of task-level exceptions. -- The Task SDK redefines these classes so workers no longer depend on ``airflow-core`` at runtime. -- ``airflow.providers.common.compat.sdk`` centralizes compatibility imports for providers. - -**Behaviour changes:** - -- Sensors and other helpers that validate user input now raise ``ValueError`` (instead of - ``AirflowException``) when ``poke_interval``/ ``timeout`` arguments are invalid. -- Importing deprecated exception names from ``airflow.exceptions`` logs a warning directing users to - the SDK import path. - -**Exceptions now provided by ``airflow.sdk.exceptions``:** - -- ``AirflowException`` and ``AirflowNotFoundException`` -- ``AirflowRescheduleException`` and ``AirflowSensorTimeout`` -- ``AirflowSkipException``, ``AirflowFailException``, ``AirflowTaskTimeout``, ``AirflowTaskTerminated`` -- ``TaskDeferred``, ``TaskDeferralTimeout``, ``TaskDeferralError`` -- ``DagRunTriggerException`` and ``DownstreamTasksSkipped`` -- ``AirflowDagCycleException`` and ``AirflowInactiveAssetInInletOrOutletException`` -- ``ParamValidationError``, ``DuplicateTaskIdFound``, ``TaskAlreadyInTaskGroup``, ``TaskNotFound``, ``XComNotFound`` -- ``AirflowOptionalProviderFeatureException`` - -**Backward compatibility:** - -- Existing Dags/operators that still import from ``airflow.exceptions`` continue to work, though - they log warnings. -- Providers can rely on ``airflow.providers.common.compat.sdk`` to keep one import path that works - across supported Airflow versions. - -**Migration:** - -- Update custom operators, sensors, and extensions to import exception classes from - ``airflow.sdk.exceptions`` (or from the provider compat shim). -- Adjust custom validation code to expect ``ValueError`` for invalid sensor arguments if it - previously caught ``AirflowException``. - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [ ] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [x] Code interface changes - -* Migration rules needed - - * Import task-level exceptions such as ``AirflowSkipException``, ``TaskDeferred``, - ``AirflowFailException``, etc. from ``airflow.sdk.exceptions`` instead of ``airflow.exceptions``. - * Update custom sensors/operators that validated arguments by catching ``AirflowException`` to - expect ``ValueError`` for invalid ``poke_interval`` / ``timeout`` inputs. diff --git a/airflow-core/newsfragments/55850.significant.rst b/airflow-core/newsfragments/55850.significant.rst deleted file mode 100644 index 3a0407b1168..00000000000 --- a/airflow-core/newsfragments/55850.significant.rst +++ /dev/null @@ -1,13 +0,0 @@ -Easily allow setting level of individual loggers - -Add ``namespace_levels`` config option (``AIRFLOW__LOGGING__NAMESPACE_LEVELS``) to set log -levels for individual named loggers, independent of the global ``logging_level``. - -This is particularly useful for debugging. For example, it allows enabling debug -logging for ``botocore`` while keeping ``sqlalchemy.engine`` quiet:: - - AIRFLOW__LOGGING__NAMESPACE_LEVELS='botocore=debug sqlalchemy.engine=error' - -The format is whitespace- or comma-separated ``<logger>=<level>`` pairs. Logger names are -visible in task logs as the ``source`` attribute, or in the ``[]`` after the message in the -console log appearance. diff --git a/airflow-core/newsfragments/56866.significant.rst b/airflow-core/newsfragments/56866.significant.rst deleted file mode 100644 index ac3806334a5..00000000000 --- a/airflow-core/newsfragments/56866.significant.rst +++ /dev/null @@ -1,38 +0,0 @@ -Support numeric multiplier values for retry_exponential_backoff parameter - -The ``retry_exponential_backoff`` parameter now accepts numeric values to specify custom exponential backoff multipliers for task retries. Previously, this parameter only accepted boolean values (``True`` or ``False``), with ``True`` using a hardcoded multiplier of ``2.0``. - -**New behavior:** - -- Numeric values (e.g., ``2.0``, ``3.5``) directly specify the exponential backoff multiplier -- ``retry_exponential_backoff=2.0`` doubles the delay between each retry attempt -- ``retry_exponential_backoff=0`` or ``False`` disables exponential backoff (uses fixed ``retry_delay``) - -**Backwards compatibility:** - -Existing DAGs using boolean values continue to work: - -- ``retry_exponential_backoff=True`` → converted to ``2.0`` (maintains original behavior) -- ``retry_exponential_backoff=False`` → converted to ``0.0`` (no exponential backoff) - -**API changes:** - -The REST API schema for ``retry_exponential_backoff`` has changed from ``type: boolean`` to ``type: number``. API clients must use numeric values (boolean values will be rejected). - -**Migration:** - -While boolean values in Python DAGs are automatically converted for backwards compatibility, we recommend updating to explicit numeric values for clarity: - -- Change ``retry_exponential_backoff=True`` → ``retry_exponential_backoff=2.0`` -- Change ``retry_exponential_backoff=False`` → ``retry_exponential_backoff=0`` - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [x] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes diff --git a/airflow-core/newsfragments/57069.significant.rst b/airflow-core/newsfragments/57069.significant.rst deleted file mode 100644 index a4d1c22bb7a..00000000000 --- a/airflow-core/newsfragments/57069.significant.rst +++ /dev/null @@ -1,16 +0,0 @@ -Git provider: Remove '.git' folder from versions in GitDagBundle - -A new option(``prune_dotgit_folder``) has been added to the GitDagBundle to remove ``.git`` from -versioned bundles by default to reduce disk usage; set prune_dotgit_folder=False to keep -repo metadata in the dag bundle's versions folders. - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [ ] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes diff --git a/airflow-core/newsfragments/58337.feature.rst b/airflow-core/newsfragments/58337.feature.rst deleted file mode 100644 index e2720735819..00000000000 --- a/airflow-core/newsfragments/58337.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Enable FIPS Support by making Python LTO configurable via ``PYTHON_LTO`` build argument diff --git a/airflow-core/newsfragments/58524.significant.rst b/airflow-core/newsfragments/58524.significant.rst deleted file mode 100644 index ef94f296dcf..00000000000 --- a/airflow-core/newsfragments/58524.significant.rst +++ /dev/null @@ -1,7 +0,0 @@ -FastAPI dependency is updated and lower bound set to ``0.121.0`` - -Airflow now requires FastAPI ``>=0.121.0`` and removes the previous upper -constraint. This aligns with FastAPI's new dependency lifecycle scopes -(e.g. request/function) that change how dependencies with ``yield`` are -exited. -See `FastAPI PR #14262 <https://github.com/fastapi/fastapi/pull/14262>`_. diff --git a/airflow-core/newsfragments/58992.significant.rst b/airflow-core/newsfragments/58992.significant.rst deleted file mode 100644 index 22eb8fe6d8a..00000000000 --- a/airflow-core/newsfragments/58992.significant.rst +++ /dev/null @@ -1,43 +0,0 @@ -Move serialization/deserialization (serde) logic into Task SDK - -Airflow now sources serde logic from ``airflow.sdk.serde`` instead of -``airflow.serialization.serde``. Serializer modules have moved from ``airflow.serialization.serializers.*`` -to ``airflow.sdk.serde.serializers.*``. The old import paths still work but emit ``DeprecatedImportWarning`` -to guide migration. The backward compatibility layer will be removed in Airflow 4. - -**What changed:** - -- Serialization/deserialization code moved from ``airflow-core`` to ``task-sdk`` package -- Serializer modules moved from ``airflow.serialization.serializers.*`` to ``airflow.sdk.serde.serializers.*`` -- New serializers should be added to ``airflow.sdk.serde.serializers.*`` namespace - -**Code interface changes:** - -- Import serializers from ``airflow.sdk.serde.serializers.*`` instead of ``airflow.serialization.serializers.*`` -- Import serialization functions from ``airflow.sdk.serde`` instead of ``airflow.serialization.serde`` - -**Backward compatibility:** - -- Existing serializers importing from ``airflow.serialization.serializers.*`` continue to work with deprecation warnings -- All existing serializers (builtin, datetime, pandas, numpy, etc.) are available at the new location - -**Migration:** - -- **For existing custom serializers**: Update imports to use ``airflow.sdk.serde.serializers.*`` -- **For new serializers**: Add them to ``airflow.sdk.serde.serializers.*`` namespace (e.g., create ``task-sdk/src/airflow/sdk/serde/serializers/your_serializer.py``) - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [ ] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [x] Code interface changes - -* Migration rules needed - - * Import serializers from ``airflow.sdk.serde.serializers.*`` instead of ``airflow.serialization.serializers.*`` - * Add new custom serializers to ``airflow.sdk.serde.serializers.*`` namespace diff --git a/airflow-core/newsfragments/59239.feature.rst b/airflow-core/newsfragments/59239.feature.rst deleted file mode 100644 index e4292a4b906..00000000000 --- a/airflow-core/newsfragments/59239.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Support for task queue-based Trigger assignment to specific Triggerer hosts via the new ``--queues`` CLI option for the ``trigger`` command. diff --git a/airflow-core/newsfragments/59780.significant.rst b/airflow-core/newsfragments/59780.significant.rst deleted file mode 100644 index fb989784a25..00000000000 --- a/airflow-core/newsfragments/59780.significant.rst +++ /dev/null @@ -1,9 +0,0 @@ -Methods removed from PriorityWeightStrategy and TaskInstance - -On (experimental) class ``PriorityWeightStrategy``, functions ``serialize()`` -and ``deserialize()`` were never used anywhere, and have been removed. They -should not be relied on in user code. - -On class ``TaskInstance``, functions ``run()``, ``render_templates()``, and -private members related to them have been removed. The class has been -considered internal since 3.0, and should not be relied on in user code. diff --git a/airflow-core/newsfragments/59785.significant.rst b/airflow-core/newsfragments/59785.significant.rst deleted file mode 100644 index f92ccbc0122..00000000000 --- a/airflow-core/newsfragments/59785.significant.rst +++ /dev/null @@ -1,10 +0,0 @@ -Modify the information returned by ``DagBag`` - -**New behavior:** -- ``DagBag`` now uses ``Path.relative_to`` for consistent cross-platform behavior. -- ``FileLoadStat`` now has two additional nullable fields: ``bundle_path`` and ``bundle_name``. - -**Backward compatibility:** -``FileLoadStat`` will no longer produce paths beginning with ``/`` with the meaning of "relative to the dags folder". -This is a breaking change for any custom code that performs string-based path manipulations relying on this behavior. -Users are advised to update such code to use ``pathlib.Path``. diff --git a/airflow-core/newsfragments/59835.significant.rst b/airflow-core/newsfragments/59835.significant.rst deleted file mode 100644 index 02b9d709a49..00000000000 --- a/airflow-core/newsfragments/59835.significant.rst +++ /dev/null @@ -1,6 +0,0 @@ -Methods removed from TaskInstance - -On class ``TaskInstance``, functions ``run()``, ``render_templates()``, -``get_template_context()``, and private members related to them have been -removed. The class has been considered internal since 3.0, and should not be -relied on in user code. diff --git a/airflow-core/newsfragments/59855.significant.rst b/airflow-core/newsfragments/59855.significant.rst deleted file mode 100644 index 94507ca9e3c..00000000000 --- a/airflow-core/newsfragments/59855.significant.rst +++ /dev/null @@ -1 +0,0 @@ -Removed the redundant ``--conn-id`` option from the ``airflow connections list`` CLI command. Use ``airflow connections get`` instead. diff --git a/airflow-core/newsfragments/59880.bugfix.rst b/airflow-core/newsfragments/59880.bugfix.rst deleted file mode 100644 index e1c35a9d640..00000000000 --- a/airflow-core/newsfragments/59880.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Always mask sensitive configuration values in public config APIs and treat the deprecated ``non-sensitive-only`` value as ``True``. diff --git a/airflow-core/newsfragments/59938.bugfix.rst b/airflow-core/newsfragments/59938.bugfix.rst deleted file mode 100644 index db9ecabfc0c..00000000000 --- a/airflow-core/newsfragments/59938.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Pool names with invalid characters for stats reporting are now automatically normalized (invalid characters replaced with underscores) when emitting metrics, preventing ``InvalidStatsNameException``. A warning is logged when normalization occurs, suggesting the pool be renamed. diff --git a/airflow-core/newsfragments/60268.improvement.rst b/airflow-core/newsfragments/60268.improvement.rst deleted file mode 100644 index 8c7e92b8f0d..00000000000 --- a/airflow-core/newsfragments/60268.improvement.rst +++ /dev/null @@ -1 +0,0 @@ -The ``PythonOperator`` parameter ``python_callable`` now also supports async callables in Airflow 3.2, allowing users to run async def functions without manually managing an event loop. diff --git a/airflow-core/newsfragments/60619.significant.rst b/airflow-core/newsfragments/60619.significant.rst deleted file mode 100644 index 2d975f624bd..00000000000 --- a/airflow-core/newsfragments/60619.significant.rst +++ /dev/null @@ -1,22 +0,0 @@ -Add operator-level ``render_template_as_native_obj`` override - -Operators can now override the DAG-level ``render_template_as_native_obj`` setting, -enabling fine-grained control over whether templates are rendered as native Python -types or strings on a per-task basis. Set ``render_template_as_native_obj=True`` or -``False`` on any operator to override the DAG setting, or leave as ``None`` (default) -to inherit from the DAG. - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [ ] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes - -* Migration rules needed - - * None - this is a new optional feature with backwards-compatible defaults diff --git a/airflow-core/newsfragments/60803.significant.rst b/airflow-core/newsfragments/60803.significant.rst deleted file mode 100644 index 1054258ad38..00000000000 --- a/airflow-core/newsfragments/60803.significant.rst +++ /dev/null @@ -1 +0,0 @@ -Move ``upstream_map_indexes`` computation from API server to Task SDK, reducing memory usage on task start by eliminating ``SerializedDAG`` loads. diff --git a/airflow-core/newsfragments/60804.feature.rst b/airflow-core/newsfragments/60804.feature.rst deleted file mode 100644 index 7b75d02ede6..00000000000 --- a/airflow-core/newsfragments/60804.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Add configurable LRU+TTL caching for API server DAG retrieval via ``dag_cache_size`` and ``dag_cache_ttl`` config options in the ``[api]`` section. This bounds memory growth from accumulated SerializedDAG objects in long-running API server processes. diff --git a/airflow-core/newsfragments/60921.significant.rst b/airflow-core/newsfragments/60921.significant.rst deleted file mode 100644 index c97bbcdf97a..00000000000 --- a/airflow-core/newsfragments/60921.significant.rst +++ /dev/null @@ -1,51 +0,0 @@ -Add gunicorn support for API server with zero-downtime worker recycling - -The API server now supports gunicorn as an alternative server with rolling worker restarts -to prevent memory accumulation in long-running processes. - -**Key Benefits:** - -* **Rolling worker restarts**: New workers spawn and pass health checks before old workers - are killed, ensuring zero downtime during worker recycling. - -* **Memory sharing**: Gunicorn uses preload + fork, so workers share memory via - copy-on-write. This significantly reduces total memory usage compared to uvicorn's - multiprocess mode where each worker loads everything independently. - -* **Correct FIFO signal handling**: Gunicorn's SIGTTOU kills the oldest worker (FIFO), - not the newest (LIFO), which is correct for rolling restarts. - -**Configuration:** - -.. code-block:: ini - - [api] - # Use gunicorn instead of uvicorn - server_type = gunicorn - - # Enable rolling worker restarts every 12 hours - worker_refresh_interval = 43200 - - # Restart workers one at a time - worker_refresh_batch_size = 1 - -Or via environment variables: - -.. code-block:: bash - - export AIRFLOW__API__SERVER_TYPE=gunicorn - export AIRFLOW__API__WORKER_REFRESH_INTERVAL=43200 - -**Requirements:** - -Install the gunicorn extra: ``pip install 'apache-airflow-core[gunicorn]'`` - -**Note on uvicorn (default):** - -The default uvicorn mode does not support rolling worker restarts because: - -1. With workers=1, there is no master process to send signals to -2. uvicorn's SIGTTOU kills the newest worker (LIFO), defeating rolling restart purposes -3. Each uvicorn worker loads everything independently with no memory sharing - -If you need worker recycling or memory-efficient multi-worker deployment, use gunicorn. diff --git a/airflow-core/newsfragments/60951.significant.rst b/airflow-core/newsfragments/60951.significant.rst deleted file mode 100644 index 1f203b34a62..00000000000 --- a/airflow-core/newsfragments/60951.significant.rst +++ /dev/null @@ -1,7 +0,0 @@ -Improved performance of rendered task instance fields cleanup for DAGs with many mapped tasks (~42x faster). - -The config ``max_num_rendered_ti_fields_per_task`` is renamed to ``num_dag_runs_to_retain_rendered_fields`` -(old name still works with deprecation warning). - -Retention is now based on the N most recent dag runs rather than N most recent task executions, -which may result in fewer records retained for conditional/sparse tasks. diff --git a/airflow-core/newsfragments/61153.significant.rst b/airflow-core/newsfragments/61153.significant.rst deleted file mode 100644 index 51f4727c240..00000000000 --- a/airflow-core/newsfragments/61153.significant.rst +++ /dev/null @@ -1,19 +0,0 @@ -Add synchronous callback support (``SyncCallback``) for Deadline Alerts - -Deadline Alerts now support synchronous callbacks via ``SyncCallback`` in addition to the existing -asynchronous ``AsyncCallback``. Synchronous callbacks are executed by the executor (rather than -the triggerer), and can optionally target a specific executor via the ``executor`` parameter. - -A DAG can also define multiple Deadline Alerts by passing a list to the ``deadline`` parameter, -and each alert can use either callback type. - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [ ] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes diff --git a/airflow-core/newsfragments/61274.improvement.rst b/airflow-core/newsfragments/61274.improvement.rst deleted file mode 100644 index ab9f4f4c0f0..00000000000 --- a/airflow-core/newsfragments/61274.improvement.rst +++ /dev/null @@ -1 +0,0 @@ -Improve Dag callback relevancy by passing a context-relevant task instance based on the Dag's final state (e.g., the last failed, timed out, or successful task) instead of an arbitrary lexicographical selection. diff --git a/airflow-core/newsfragments/62055.significant.rst b/airflow-core/newsfragments/62055.significant.rst deleted file mode 100644 index 7e4ddaed981..00000000000 --- a/airflow-core/newsfragments/62055.significant.rst +++ /dev/null @@ -1,16 +0,0 @@ -Airflow scheduler CLI command have a new ``--only-idle`` flag to only count runs when the scheduler is idle. - -It will help users to run the scheduler once and process all the triggered DAGs and all the queued tasks. -It requires and complements the ``--num-runs`` flag so one can set a small value to it instead of gessing how many times the scheduler will run. - - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [ ] API changes - * [X] CLI changes - * [ ] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes diff --git a/airflow-core/newsfragments/62343.feature.rst b/airflow-core/newsfragments/62343.feature.rst deleted file mode 100644 index b69d2868d1f..00000000000 --- a/airflow-core/newsfragments/62343.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Add worker-dispatched (asynchronous) connection testing: connection tests can run on a worker instead of in-process on the API server (so credentials are exercised on a worker), submitted via ``POST /connections/enqueue-test`` (which returns a token) and polled via ``GET /connections/enqueue-test`` with the token in the ``Airflow-Connection-Test-Token`` header, configured under the new ``[connection_test]`` section (``timeout``, ``max_concurrency``, ``reaper_interval``); the synchronous [...] diff --git a/airflow-core/newsfragments/62344.feature.rst b/airflow-core/newsfragments/62344.feature.rst deleted file mode 100644 index 7fd43b0c03f..00000000000 --- a/airflow-core/newsfragments/62344.feature.rst +++ /dev/null @@ -1 +0,0 @@ -CLI ``connections list`` and ``variables list`` now hide sensitive values by default. Use ``--show-values`` to display full details and ``--hide-sensitive`` to mask passwords, URIs, and extras. diff --git a/airflow-core/newsfragments/62369.significant.rst b/airflow-core/newsfragments/62369.significant.rst deleted file mode 100644 index b5d4113ae58..00000000000 --- a/airflow-core/newsfragments/62369.significant.rst +++ /dev/null @@ -1,30 +0,0 @@ -Replace per-run TI summary requests with a single NDJSON stream - -The grid, graph, gantt, and task-detail views now fetch task-instance -summaries through a single streaming HTTP request -(``GET /ui/grid/ti_summaries/{dag_id}?run_ids=...``) instead of one request -per run. The server emits one JSON line per run as soon as that run's task -instances are ready, so columns appear progressively rather than all at once. - -**What changed:** - -- ``GET /ui/grid/ti_summaries/{dag_id}?run_ids=...`` is now the sole endpoint - for TI summaries, returning an ``application/x-ndjson`` stream where each - line is a serialized ``GridTISummaries`` object for one run. -- The old single-run endpoint ``GET /ui/grid/ti_summaries/{dag_id}/{run_id}`` - has been removed. -- The serialized Dag structure is loaded once and shared across all runs that - share the same ``dag_version_id``, avoiding redundant deserialization. -- All UI views (grid, graph, gantt, task instance, mapped task instance, group - task instance) use the stream endpoint, passing one or more ``run_ids``. - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [x] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes diff --git a/airflow-core/newsfragments/62487.significant.rst b/airflow-core/newsfragments/62487.significant.rst deleted file mode 100644 index 5de4f8fe479..00000000000 --- a/airflow-core/newsfragments/62487.significant.rst +++ /dev/null @@ -1 +0,0 @@ -``dag_processing.last_run.seconds_ago.{dag_file}`` is now a legacy metric. The new metric ``dag_processing.last_run.seconds_ago`` is emitted with ``file_path``, ``bundle_name`` and ``file_name`` as tags, where ``file_path`` and ``bundle_name`` uniquely identify the DAG file. The legacy metric is still emitted by default and can be disabled via ``[metrics] legacy_names_on``. diff --git a/airflow-core/newsfragments/62561.bugfix.rst b/airflow-core/newsfragments/62561.bugfix.rst deleted file mode 100644 index 9b40b89e211..00000000000 --- a/airflow-core/newsfragments/62561.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix backfill marked complete before DagRuns are created; add age-based cleanup for orphaned backfills. diff --git a/airflow-core/newsfragments/62964.bugfix.rst b/airflow-core/newsfragments/62964.bugfix.rst deleted file mode 100644 index 048ba18893b..00000000000 --- a/airflow-core/newsfragments/62964.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Prevent JWT tokens from appearing in task logs by excluding the token field from workload object representations. diff --git a/airflow-core/newsfragments/63141.bugfix.rst b/airflow-core/newsfragments/63141.bugfix.rst deleted file mode 100644 index c9855b3f5e4..00000000000 --- a/airflow-core/newsfragments/63141.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix security iframe navigation when AIRFLOW__API__BASE_URL basename is configured diff --git a/airflow-core/newsfragments/63205.bugfix.rst b/airflow-core/newsfragments/63205.bugfix.rst deleted file mode 100644 index 7e1781bc8ed..00000000000 --- a/airflow-core/newsfragments/63205.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix grid view URL for dynamic task groups producing 404 by not appending ``/mapped`` to group URLs. diff --git a/airflow-core/newsfragments/63266.bugfix.rst b/airflow-core/newsfragments/63266.bugfix.rst deleted file mode 100644 index a8e1ff44aec..00000000000 --- a/airflow-core/newsfragments/63266.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix ``ti_skip_downstream`` overwriting RUNNING tasks to SKIPPED in HA deployments. diff --git a/airflow-core/newsfragments/63296.significant.rst b/airflow-core/newsfragments/63296.significant.rst deleted file mode 100644 index 566e21e0de6..00000000000 --- a/airflow-core/newsfragments/63296.significant.rst +++ /dev/null @@ -1,16 +0,0 @@ -Block path traversal via ``..`` in ``dag_id`` and ``run_id`` - -DAG IDs and run IDs containing ``..`` are now rejected by default to prevent path traversal. -A configuration flag ``[core] allow_double_dot_in_ids`` (default: ``False``) is available for -environments that rely on ``..`` in identifiers. - -* Types of change - - * [ ] Dag changes - * [x] Config changes - * [x] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes diff --git a/airflow-core/newsfragments/63312.feature.rst b/airflow-core/newsfragments/63312.feature.rst deleted file mode 100644 index 1af30e04ded..00000000000 --- a/airflow-core/newsfragments/63312.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Allow individual secrets backend kwargs to be set via ``AIRFLOW__SECRETS__BACKEND_KWARG__<KEY>`` environment variables diff --git a/airflow-core/newsfragments/63365.significant.rst b/airflow-core/newsfragments/63365.significant.rst deleted file mode 100644 index 494055ec525..00000000000 --- a/airflow-core/newsfragments/63365.significant.rst +++ /dev/null @@ -1,9 +0,0 @@ -Structured JSON logging for all API server output - -The new ``json_logs`` option under the ``[logging]`` section makes Airflow -produce all its output as newline-delimited JSON (structured logs) instead of -human-readable formatted logs. This covers the API server (gunicorn/uvicorn), -including access logs, warnings, and unhandled exceptions. - -Not all components support this yet — notably ``airflow celery worker`` but -any non-JSON output when ``json_logs`` is enabled will be treated as a bug. diff --git a/airflow-core/newsfragments/63452.significant.rst b/airflow-core/newsfragments/63452.significant.rst deleted file mode 100644 index b0ffc64d4ea..00000000000 --- a/airflow-core/newsfragments/63452.significant.rst +++ /dev/null @@ -1,7 +0,0 @@ -Remove legacy OTel Trace metaclass and shared tracer wrappers - -The interfaces and functions located in ``airflow.traces`` were -internal code that provided a standard way to manage spans in -internal Airflow code. They were not intended as user-facing code -and were never documented. They are no longer needed so we -remove them in 3.2. diff --git a/airflow-core/newsfragments/63664.bugfix.rst b/airflow-core/newsfragments/63664.bugfix.rst deleted file mode 100644 index e759d608418..00000000000 --- a/airflow-core/newsfragments/63664.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix ``timeout_with_traceback`` crashes on Windows and non-main threads diff --git a/airflow-core/newsfragments/63878.bugfix.rst b/airflow-core/newsfragments/63878.bugfix.rst deleted file mode 100644 index f85875f7433..00000000000 --- a/airflow-core/newsfragments/63878.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed ``airflow`` CLI commands (including ``airflow db migrate``) crashing with ``PermissionError`` on startup when ``base_log_folder`` is configured to a path on a mounted filesystem (e.g. NFS) whose parent directories the Airflow user cannot create. diff --git a/airflow-core/newsfragments/63884.significant.rst b/airflow-core/newsfragments/63884.significant.rst deleted file mode 100644 index 2695006b183..00000000000 --- a/airflow-core/newsfragments/63884.significant.rst +++ /dev/null @@ -1,34 +0,0 @@ -Add ``rerun_with_latest_version`` configuration for DAG bundle versioning - -When clearing, rerunning, or backfilling tasks, this setting controls whether the -new DAG run uses the latest bundle version or the original version from the -initial run. It applies to clear/rerun actions (UI and API) and to backfill -creation (API and CLI). The default is resolved using the following precedence: - -1. **Explicit request**: ``run_on_latest_version`` parameter in the API request body - or the ``--run-on-latest-version`` / ``--no-run-on-latest-version`` CLI flag. -2. **DAG-level**: ``rerun_with_latest_version`` parameter on the DAG definition. -3. **Global config**: ``[core] rerun_with_latest_version`` in ``airflow.cfg``. -4. **Default**: ``False`` for clear/rerun, ``True`` for backfills (preserves - historical behavior). - -In Airflow 2.x, reruns always used the latest code. Airflow 3.x introduced bundle -versioning, defaulting to the original version. This setting gives users control -over which behaviour is the default. - -See :doc:`/administration-and-deployment/dag-bundles` for full details. - -* Types of change - - * [ ] Dag changes - * [x] Config changes - * [x] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes - -* Migration rules needed - - * None - this is a new optional feature with backwards-compatible defaults diff --git a/airflow-core/newsfragments/64067.bugfix.rst b/airflow-core/newsfragments/64067.bugfix.rst deleted file mode 100644 index 8ae9a97f544..00000000000 --- a/airflow-core/newsfragments/64067.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Restore live stdout logging for Elasticsearch in Airflow 3 by correctly configuring the handler in ``airflow_local_settings.py`` and forwarding task logs to stdout in ``LocalExecutor``. diff --git a/airflow-core/newsfragments/64207.significant.rst b/airflow-core/newsfragments/64207.significant.rst deleted file mode 100644 index 3254fa20a54..00000000000 --- a/airflow-core/newsfragments/64207.significant.rst +++ /dev/null @@ -1 +0,0 @@ -OTel timer and timing metrics now use Histogram instead of Gauge, preserving count, sum, and bucket distribution across recordings. diff --git a/airflow-core/newsfragments/64232.feature.rst b/airflow-core/newsfragments/64232.feature.rst deleted file mode 100644 index 393a668ba82..00000000000 --- a/airflow-core/newsfragments/64232.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Allow customizing gray, black, and white color tokens in AIRFLOW__API__THEME in addition to brand. diff --git a/airflow-core/newsfragments/64503.bugfix.rst b/airflow-core/newsfragments/64503.bugfix.rst deleted file mode 100644 index 0358708ea1f..00000000000 --- a/airflow-core/newsfragments/64503.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix scheduler crashing with ``StaleDataError`` when a task instance is completed or removed by another session between ``verify_integrity`` loading task instances and ``session.flush()`` persisting them. Now caught and rolled back like the existing ``IntegrityError`` path. diff --git a/airflow-core/newsfragments/64552.improvement.rst b/airflow-core/newsfragments/64552.improvement.rst deleted file mode 100644 index ae70554cd22..00000000000 --- a/airflow-core/newsfragments/64552.improvement.rst +++ /dev/null @@ -1 +0,0 @@ -Allow UI theme config with only CSS overrides, icon only, or empty ``{}`` to restore OSS defaults. The ``tokens`` field is now optional in the theme configuration. diff --git a/airflow-core/newsfragments/64571.significant.rst b/airflow-core/newsfragments/64571.significant.rst deleted file mode 100644 index d9adc38f1d7..00000000000 --- a/airflow-core/newsfragments/64571.significant.rst +++ /dev/null @@ -1,61 +0,0 @@ -AIP-76: Asset Partitioning - -Airflow 3.2 introduces asset partitioning (time-based), allowing Dags to produce and consume -partitioned asset events identified by a ``partition_key`` (e.g. ``2026-03-10T09`` for -an hourly partition). - -**New timetables**: - -- ``CronPartitionTimetable`` — schedules a producer Dag on a cron expression and emits a - partitioned asset event on each run. -- ``PartitionedAssetTimetable`` — schedules a consumer Dag when the expected partitioned - asset events have arrived. - -**Partition mappers** transform upstream partition keys before passing them to downstream Dags: - -- ``IdentityMapper`` — passes the key through unchanged (default). -- ``StartOfHourMapper``, ``StartOfDayMapper``, ``StartOfWeekMapper``, ``StartOfMonthMapper``, - ``StartOfQuarterMapper``, ``StartOfYearMapper`` — normalize a datetime partition key to a - coarser time granularity (e.g. ``StartOfDayMapper`` maps ``2026-03-10T09:30:00`` → ``2026-03-10``). -- ``ProductMapper`` — applies a separate mapper to each ``|``-delimited segment of a - composite key (e.g. ``ProductMapper(IdentityMapper(), StartOfDayMapper())`` maps - ``"us|2026-03-10T09:30:00"`` → ``"us|2026-03-10"``). -- ``ChainMapper`` — applies mappers sequentially, passing each step's output to the next (e.g., - ``ChainMapper(StartOfHourMapper(), StartOfDayMapper(input_format="%Y-%m-%dT%H"))`` maps - ``"2026-03-10T09:30:00"`` → ``"2026-03-10"``). -- ``AllowedKeyMapper`` — validates the key is in a fixed allowlist and passes it through - unchanged, raising ``ValueError`` otherwise (e.g. - ``AllowedKeyMapper(["us", "eu", "ap"])`` accepts ``"us"`` but rejects ``"cn"``). - -Mappers can be set globally on a ``PartitionedAssetTimetable`` or overridden per upstream asset via ``partition_mapper_config``. - -**Rollup (one downstream run per window of upstream partitions)**: - -- ``RollupMapper`` — wraps a ``upstream_mapper`` (which normalises the upstream key to the - downstream granularity) with a ``window`` describing how many upstream partitions form one - downstream run. The downstream Dag run is held until every expected upstream partition for - the window has arrived (e.g. all 24 hourly partitions before firing a daily summary). -- Window types: ``HourWindow``, ``DayWindow``, ``WeekWindow``, ``MonthWindow``, - ``QuarterWindow``, ``YearWindow`` — enumerate the upstream partition keys that compose one - downstream window. ``MonthWindow``/``QuarterWindow``/``YearWindow`` iterate from the - ``upstream_mapper``'s emitted period start, so fiscal calendars are handled transparently when - the upstream mapper emits non-1st period starts. -- Typical use: ``default_partition_mapper=RollupMapper(upstream_mapper=StartOfDayMapper(), - window=DayWindow())`` on a ``PartitionedAssetTimetable`` whose ``assets`` are the hourly - upstream. - -Within the task context, the ``partition_key`` is available as ``dag_run.partition_key``. It can also be provided when manually triggering a Dag run via the REST API (``POST /dags/{dag_id}/dagRuns``). - -**Known limitations**: - -- ``DayWindow`` with a local-timezone upstream mapper is unsatisfiable on spring-forward days: - the DST gap (e.g. 02:00 ET skips to 03:00) means one of the 24 expected upstream keys is - never emitted by producers, so the rollup window can never be fully satisfied. -- On fall-back days the extra hour's upstream events are not included in the expected set and - do not contribute to any rollup. -- **Mitigation**: use UTC ``input_format`` (e.g. ``%Y-%m-%dT%H%z``) and ensure upstream - producers emit UTC partition keys to avoid local-clock ambiguity. - -* Migration rules needed - - * None — asset partitioning is a new feature. Existing Dags and assets are unaffected. diff --git a/airflow-core/newsfragments/64751.feature.rst b/airflow-core/newsfragments/64751.feature.rst deleted file mode 100644 index 41d647f143d..00000000000 --- a/airflow-core/newsfragments/64751.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Allow DeadlineAlert intervals to be dynamically resolved at Deadline evaluation using objects such as VariableInterval. diff --git a/airflow-core/newsfragments/64822.significant.rst b/airflow-core/newsfragments/64822.significant.rst deleted file mode 100644 index 348700d4d1f..00000000000 --- a/airflow-core/newsfragments/64822.significant.rst +++ /dev/null @@ -1 +0,0 @@ -Users who only have read access to DAGs will no longer be able to fetch data from the ``/dags`` endpoint, as it now requires additional permissions (``DagAccessEntity.RUN``, ``DagAccessEntity.HITL_DETAIL``, and ``DagAccessEntity.TASK_INSTANCE``). This change was made because the endpoint returns aggregated data from these multiple entities. Please update your custom user roles to include read access for DAG Runs, Task Instances, and HITL Details if those users should still have access to [...] diff --git a/airflow-core/newsfragments/65269.significant.rst b/airflow-core/newsfragments/65269.significant.rst deleted file mode 100644 index c5f68aa9457..00000000000 --- a/airflow-core/newsfragments/65269.significant.rst +++ /dev/null @@ -1 +0,0 @@ -Synchronous deadline callbacks (``SyncCallback``) can now access Connections and Variables from the Airflow metadata database. diff --git a/airflow-core/newsfragments/65346.significant.rst b/airflow-core/newsfragments/65346.significant.rst deleted file mode 100644 index bdfc390c31c..00000000000 --- a/airflow-core/newsfragments/65346.significant.rst +++ /dev/null @@ -1,3 +0,0 @@ -The SMTP STARTTLS upgrade performed by ``airflow.utils.email.send_email`` now validates the SMTP server's certificate against the system's trusted CA bundle by default. Previously the ``starttls()`` call was made without an SSL context, so any certificate was accepted. - -Deployments that intentionally point Airflow at an SMTP server with a self-signed or otherwise non-validating certificate and need to preserve the previous behaviour must set ``email.ssl_context = "none"`` in ``airflow.cfg``. The ``"default"`` value (now also the default when the option is unset) uses :func:`ssl.create_default_context`. Previously this option applied only to the ``SMTP_SSL`` path; it now applies to the STARTTLS path as well. diff --git a/airflow-core/newsfragments/65422.bugfix.rst b/airflow-core/newsfragments/65422.bugfix.rst deleted file mode 100644 index efb484da723..00000000000 --- a/airflow-core/newsfragments/65422.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix scheduler crash on asset-triggered DagRuns by eager-loading ``AssetEvent.source_aliases`` in ``SchedulerJobRunner.process_executor_events``. diff --git a/airflow-core/newsfragments/65590.feature.rst b/airflow-core/newsfragments/65590.feature.rst deleted file mode 100644 index 840a15c6f04..00000000000 --- a/airflow-core/newsfragments/65590.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Add ``on_kill()`` hook to ``BaseTrigger`` so trigger authors can cancel external jobs (BigQuery, Databricks, etc.) when a user marks a deferred task as failed, clears it, or marks it succeeded. The hook is a no-op by default and is only invoked for user-initiated cancellations — triggerer restarts and redistributions do not trigger it. diff --git a/airflow-core/newsfragments/66015.significant.rst b/airflow-core/newsfragments/66015.significant.rst deleted file mode 100644 index d7060a4c814..00000000000 --- a/airflow-core/newsfragments/66015.significant.rst +++ /dev/null @@ -1,3 +0,0 @@ -In #64963, the Airflow UI switched from full-match ``*_pattern`` REST API query parameters to the new index-friendly ``*_prefix_pattern`` parameters on list endpoints. This is a behavioral change for search-as-you-type filters in the UI: matches are prefix-based (``LIKE 'term%'`` via a range scan) instead of substring-based (``ILIKE '%term%'``), which means the database can use B-tree indexes and search stays fast on large deployments. The REST API itself keeps both forms: existing ``*_p [...] - -In #66015, a per-searchbar "Match anywhere" toggle was added so users who relied on the previous substring behavior can opt back into it from the UI. Each search input and each text filter pill now has a small regex-icon toggle next to the value; flipping it on switches that input from ``*_prefix_pattern`` to ``*_pattern``. diff --git a/airflow-core/newsfragments/66030.feature.rst b/airflow-core/newsfragments/66030.feature.rst deleted file mode 100644 index c6382dc0979..00000000000 --- a/airflow-core/newsfragments/66030.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Add ``FanOutMapper`` for one-to-many partition mapping (e.g. one weekly upstream key to seven daily downstream Dag runs). It composes ``upstream_mapper + window + downstream_mapper``, mirroring the shape of ``RollupMapper`` and reusing the existing ``Window`` classes (``DayWindow``, ``WeekWindow``, ``MonthWindow``, ``QuarterWindow``, ``YearWindow``). A new ``[scheduler] partition_mapper_max_downstream_keys`` config caps the number of downstream keys produced per upstream event by any ``P [...] diff --git a/airflow-core/newsfragments/66161.significant.rst b/airflow-core/newsfragments/66161.significant.rst deleted file mode 100644 index 79b7313bb29..00000000000 --- a/airflow-core/newsfragments/66161.significant.rst +++ /dev/null @@ -1,12 +0,0 @@ -Provider example DAGs are exposed as dedicated bundles - -Example DAGs shipped by provider distributions are now discovered via -``ProvidersManager`` and registered as their own DAG bundles, one per -provider, named ``apache-airflow-providers-<distribution>-example-dags`` -(or ``<distribution>-example-dags`` for third-party providers). The -``[core] load_examples`` option still gates whether they are registered. - -REST API clients that filtered ``bundle_name`` by ``"dags-folder"`` for -provider-shipped example DAGs (e.g. ``example_python_operator``) must -update to the new per-provider bundle names. DAG identifiers are -unchanged. diff --git a/airflow-core/newsfragments/66210.bugfix.rst b/airflow-core/newsfragments/66210.bugfix.rst deleted file mode 100644 index 8d2176ac9e1..00000000000 --- a/airflow-core/newsfragments/66210.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix slow and incomplete trigger cleanup in scheduler. Unreferenced triggers are now reliably deleted instead of accumulating without bound, and the periodic cleanup no longer scans the full ``task_instance`` table on every cycle. diff --git a/airflow-core/newsfragments/66412.significant.rst b/airflow-core/newsfragments/66412.significant.rst deleted file mode 100644 index 22b06e7eeb0..00000000000 --- a/airflow-core/newsfragments/66412.significant.rst +++ /dev/null @@ -1,26 +0,0 @@ -Fix triggerer race condition and deadlock that caused deferred tasks to stall indefinitely - -Triggers that call synchronous SDK methods (e.g. ``get_task_states`` used by -``safe_to_cancel`` in several Google provider operators) could crash the triggerer's -internal subprocess. The triggerer would then continue to heartbeat normally — -appearing healthy to the scheduler — while silently processing zero triggers, causing -every deferred task to time out. This was first reported in :github-issue:`64620`; a -partial fix shipped in Airflow 3.2.1 (:github-pr:`64882`) but introduced a new deadlock -with the same visible symptom under load. - -Both issues are fixed by replacing the lock-based serialisation with response -multiplexing: each request now carries a unique ID and the response is routed back to -the correct caller, so concurrent requests from trigger threads no longer contend or -deadlock regardless of how many triggers are running or what SDK methods they call. - -**New: triggerer subprocess watchdog** - -Even with the race fixed, a trigger that blocks the event loop (e.g. by calling -``time.sleep()`` or performing blocking I/O directly in ``async def run()``) would -previously leave the triggerer appearing healthy indefinitely. - -A new ``[triggerer] runner_health_check_threshold`` config option (default: 30 seconds) -adds a watchdog: if the triggerer subprocess goes silent for longer than the threshold, -the parent process stops updating the heartbeat so the scheduler can detect the hang and -reassign triggers rather than waiting for them to individually time out. Set the option -to ``0`` to disable the watchdog. diff --git a/airflow-core/newsfragments/66499.significant.rst b/airflow-core/newsfragments/66499.significant.rst deleted file mode 100644 index 5b321588876..00000000000 --- a/airflow-core/newsfragments/66499.significant.rst +++ /dev/null @@ -1,14 +0,0 @@ -Tighten ``[core] allowed_deserialization_classes_regexp`` to require full-string matches - -Patterns in ``[core] allowed_deserialization_classes_regexp`` are now matched -against the entire classname using ``re.fullmatch()`` instead of ``re.match()``. -Previously a pattern such as ``airflow\.models\.Variable`` admitted not only -the intended class but also names that started with it -(e.g. ``airflow.models.Variable_Malicious``), because ``re.match`` only anchors -at the start of the string. - -The default value of this option is empty, so out-of-the-box deployments are -unaffected. Deployments that configured this option with patterns relying on -prefix-match semantics — for example ``airflow\.models\.`` to mean "any class -under ``airflow.models``" — must add ``.*`` to the pattern -(``airflow\.models\..*``) to retain the previous behaviour. diff --git a/airflow-core/newsfragments/66584.feature.rst b/airflow-core/newsfragments/66584.feature.rst deleted file mode 100644 index e8a547d10ba..00000000000 --- a/airflow-core/newsfragments/66584.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Sibling ``BaseEventTrigger`` instances on different ``AssetWatcher`` s can now share a single underlying poll loop in the triggerer by overriding ``shared_stream_key``, ``open_shared_stream``, and ``filter_shared_stream``. Triggers that opt out (the default) keep their existing independent ``run()`` loop behavior. diff --git a/airflow-core/newsfragments/66737.significant.rst b/airflow-core/newsfragments/66737.significant.rst deleted file mode 100644 index 18f0e39331a..00000000000 --- a/airflow-core/newsfragments/66737.significant.rst +++ /dev/null @@ -1 +0,0 @@ -Custom deadline reference classes must now be registered via the new ``deadline_references`` attribute on ``AirflowPlugin``, matching the existing pattern for custom timetables and custom partition mappers. To use a custom ``DeadlineReference`` subclass, register it in a plugin's ``deadline_references`` list. Custom references that are not registered will raise ``DeadlineReferenceNotRegistered`` at deserialization. diff --git a/airflow-core/newsfragments/66807.feature.rst b/airflow-core/newsfragments/66807.feature.rst deleted file mode 100644 index 623aecef3d2..00000000000 --- a/airflow-core/newsfragments/66807.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Add ``dagrun.first_task_start_delay`` timer that measures the elapsed time between a Dag run's ``queued_at`` and the start time of its first task. This separates executor-pickup latency from the scheduler-enqueue latency captured by ``dagrun.first_task_scheduling_delay``. diff --git a/airflow-core/newsfragments/66848.feature.rst b/airflow-core/newsfragments/66848.feature.rst deleted file mode 100644 index 1d1029310f7..00000000000 --- a/airflow-core/newsfragments/66848.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Add ``wait_policy`` to ``RollupMapper`` — ``WaitForAll()`` (default) fires when all keys arrive; ``MinimumCount(n)`` fires on partial windows. diff --git a/airflow-core/newsfragments/67056.significant.rst b/airflow-core/newsfragments/67056.significant.rst deleted file mode 100644 index 266cb2c24e7..00000000000 --- a/airflow-core/newsfragments/67056.significant.rst +++ /dev/null @@ -1,101 +0,0 @@ -Decouple remote logging resolution from ``airflow.logging_config`` - -Remote task log handler resolution is now owned by the shared -``airflow_shared.logging.factory`` module and applies a single, well-defined -precedence rule. ``airflow.logging_config.load_logging_config`` is deprecated: -loading the ``[logging] logging_config_class`` dict and resolving the remote -handler are now two independent steps. - -**Resolution order (``resolve_remote_task_log``):** - -1. **User-defined ``[logging] logging_config_class``** — if the user has set - ``logging_config_class`` to a custom module path, and that module exports a - ``REMOTE_TASK_LOG`` (or ``DEFAULT_REMOTE_CONN_ID``) attribute, those values - win. Existing custom logging configs keep working unchanged. -2. **ProvidersManager scheme dispatch** — the scheme of ``[logging] - remote_base_log_folder`` (e.g. ``s3``, ``gs``, ``wasb``) is looked up in the - provider yaml ``remote-logging:`` registry. The matching ``RemoteLogIO`` - class is imported and instantiated via its ``from_config()`` classmethod. - The connection id comes from ``[logging] remote_log_conn_id`` (set - explicitly by the user; providers needing a backend default can read it - inside ``from_config``). -3. **Legacy attr-path fallback** — if neither of the above produced a handler, - the resolver imports the default logging module - (``airflow.config_templates.airflow_local_settings``) and reads its - ``REMOTE_TASK_LOG`` / ``DEFAULT_REMOTE_CONN_ID`` attributes. This is the - per-scheme ``if/elif`` chain in ``airflow_local_settings.py`` and is - transitional — it will be removed once every in-tree provider exposes - ``from_config`` in Airflow 4.0. - -**``RemoteLogIO.from_config`` contract:** - -Provider remote-log handler classes opting into provider dispatch must expose -a ``from_config`` classmethod. The shape is:: - - class MyRemoteLogIO(LoggingMixin): - @classmethod - def from_config(cls) -> "MyRemoteLogIO": - from airflow.providers.common.compat.sdk import conf - - return cls( - base_log_folder=conf.get("logging", "base_log_folder"), - remote_base=conf.get("logging", "remote_base_log_folder"), - delete_local_copy=conf.getboolean("logging", "delete_local_logs"), - # backend-specific keys live in the provider's own config section - ..., - ) - -Key properties: - -- Takes no arguments — the shared factory calls ``cls.from_config()`` with no - inputs. Providers read ``airflow.providers.common.compat.sdk.conf`` themselves and pick - the keys they care about. -- Returns a fully instantiated ``RemoteLogIO`` (or ``RemoteLogStreamIO``). -- Failures inside ``from_config`` are logged and treated as "no remote - handler" (the factory returns ``None`` and the legacy fallback runs); under - ``PYTEST_CURRENT_TEST`` the exception is re-raised so tests fail loudly. -- Providers that don't yet implement ``from_config`` continue to work via the - legacy ``airflow_local_settings.py`` chain (step 3). - -**``airflow.logging_config`` API changes:** - -- ``_get_logging_config()`` — new private helper that imports and validates - the ``[logging] logging_config_class`` dict only. Does not touch remote - logging state. -- ``_load_logging_config()`` — new private helper that calls - ``resolve_remote_task_log`` and caches the result on - ``_ActiveLoggingConfig``. Used lazily by ``get_remote_task_log`` and - ``get_default_remote_conn_id``. -- ``load_logging_config()`` — deprecated. Emits ``DeprecationWarning`` and - delegates to both helpers; still returns ``(logging_config_dict, - logging_class_path)`` so existing callers keep working. - -**Behaviour changes:** - -- ``configure_logging`` no longer eagerly resolves the remote handler. - Resolution is now lazy and happens on the first call to - ``get_remote_task_log()`` / ``get_default_remote_conn_id()``. -- Providers that registered a ``remote-logging:`` block but did not implement - ``from_config`` will be skipped with a warning; the legacy fallback path - takes over. - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [ ] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [x] Code interface changes - -* Migration rules needed - - * Replace direct calls to ``airflow.logging_config.load_logging_config()`` - with ``_get_logging_config()`` (for the logging dict) and/or - ``_load_logging_config()`` (to prime the remote-handler cache). - * Provider remote-log handler classes should implement a no-argument - ``from_config`` classmethod that reads ``airflow.providers.common.compat.sdk.conf`` - and returns a configured instance. Until they do, resolution falls - through to the legacy ``airflow_local_settings.py`` chain. diff --git a/airflow-core/newsfragments/67164.significant.rst b/airflow-core/newsfragments/67164.significant.rst deleted file mode 100644 index 283ac07d7af..00000000000 --- a/airflow-core/newsfragments/67164.significant.rst +++ /dev/null @@ -1,34 +0,0 @@ -SDK temporal partition mappers: ``timezone`` kwarg and keyword-only constructor - -``StartOfHourMapper``, ``StartOfDayMapper``, ``StartOfWeekMapper``, -``StartOfMonthMapper``, ``StartOfQuarterMapper``, and ``StartOfYearMapper`` -(imported from ``airflow.sdk``) now accept a ``timezone`` keyword argument, -matching the core ``_BaseTemporalMapper`` signature. The constructor is now -keyword-only. - -**Behaviour changes:** - -- ``input_format`` and ``output_format`` are no longer accepted positionally. - Callers that relied on ``StartOfDayMapper("%Y-%m-%dT%H:%M:%S")`` (valid in - ``task-sdk`` 1.2.1) must switch to - ``StartOfDayMapper(input_format="%Y-%m-%dT%H:%M:%S")``. -- A string ``timezone`` is now resolved via ``parse_timezone`` at - construction, so unknown names raise ``pendulum.tz.exceptions.InvalidTimezone`` - immediately instead of being stored verbatim and failing later (or, in some - paths, being silently dropped during serialization). - -**Migration:** - -- Update any ``StartOf*Mapper(...)`` call sites to pass ``input_format`` and - ``output_format`` by name. - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [ ] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [x] Code interface changes diff --git a/airflow-core/newsfragments/67184.feature.rst b/airflow-core/newsfragments/67184.feature.rst deleted file mode 100644 index 759e6921d51..00000000000 --- a/airflow-core/newsfragments/67184.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Add ``max_downstream_keys`` parameter to ``PartitionMapper`` to override ``[scheduler] partition_mapper_max_downstream_keys`` per mapper instance. diff --git a/airflow-core/newsfragments/67288.improvement.rst b/airflow-core/newsfragments/67288.improvement.rst deleted file mode 100644 index 03293e4ffa2..00000000000 --- a/airflow-core/newsfragments/67288.improvement.rst +++ /dev/null @@ -1 +0,0 @@ -Speed up ``TaskGroup.topological_sort`` across Dag shapes (chain, diamond, layered, reverse-chain); benchmarks show roughly 2-8x faster on large groups. diff --git a/airflow-core/newsfragments/67475.feature.rst b/airflow-core/newsfragments/67475.feature.rst deleted file mode 100644 index ea0b227b0d5..00000000000 --- a/airflow-core/newsfragments/67475.feature.rst +++ /dev/null @@ -1 +0,0 @@ -``Window`` subclasses accept a ``direction`` keyword — ``Window.Direction.FORWARD`` (default) fans out the period starting at the upstream key (forward in time); pass ``direction=Window.Direction.BACKWARD`` (e.g. ``WeekWindow(direction=Window.Direction.BACKWARD)``) to fan out the trailing period ending at the upstream key instead. diff --git a/airflow-core/newsfragments/67494.significant.rst b/airflow-core/newsfragments/67494.significant.rst deleted file mode 100644 index 93c56980e5e..00000000000 --- a/airflow-core/newsfragments/67494.significant.rst +++ /dev/null @@ -1,43 +0,0 @@ -Fix ``jwt_audience`` for the public API being read from two different config sections - -Earlier 3.x releases contained a bug in ``BaseAuthManager``: the JWT signer read -``jwt_audience`` from the ``[api]`` section while the validator read it from the documented -``[api_auth]`` section. Only ``[api_auth] jwt_audience`` is declared in ``config.yml``; -``[api] jwt_audience`` was never part of the schema. Default deployments were unaffected -because both sides fell back to ``apache-airflow``, but a deployment that set a custom -audience under the documented ``[api_auth]`` location had its tokens signed with the -default value and silently rejected by the validator. Some operators worked around the -bug by setting the audience under the undocumented ``[api]`` section instead, which made -the signer emit the configured value but left the validator on the default. - -Both the signer and validator now read ``jwt_audience`` from ``[api_auth]``. - -**What you should do:** - -- If you set ``[api_auth] jwt_audience`` (env ``AIRFLOW__API_AUTH__JWT_AUDIENCE``): - nothing — this is the documented location and continues to work. -- If you worked around the original bug by setting ``[api] jwt_audience`` (env - ``AIRFLOW__API__JWT_AUDIENCE``): move the value to ``[api_auth] jwt_audience`` - (env ``AIRFLOW__API_AUTH__JWT_AUDIENCE``). The value under ``[api]`` is still - honoured for backwards compatibility and emits a ``DeprecationWarning``; support - for it will be removed in a future release. -- If you set both: the documented ``[api_auth]`` value wins; remove the ``[api]`` - one to silence the deprecation warning. - -**Behaviour changes:** - -- Deployments that set ``[api_auth] jwt_audience`` will now also have their tokens - signed with that value (previously only validated against it). If signer and - validator were on different audiences because of this bug, expect tokens - generated after upgrade to be accepted again. - -* Types of change - - * [ ] Dag changes - * [x] Config changes - * [ ] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [ ] Code interface changes diff --git a/airflow-core/newsfragments/67523.feature.rst b/airflow-core/newsfragments/67523.feature.rst deleted file mode 100644 index f5320d97bfd..00000000000 --- a/airflow-core/newsfragments/67523.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Shared-stream triggers now support a producer-side ack channel so message-broker upstreams (Kafka, SQS, Pub/Sub, Service Bus) can commit, delete, or ack on the broker handle after all subscribers have accepted an event and the resulting trigger events have been persisted to the metadata database. diff --git a/airflow-core/newsfragments/67586.significant.rst b/airflow-core/newsfragments/67586.significant.rst deleted file mode 100644 index 4a1c2afb999..00000000000 --- a/airflow-core/newsfragments/67586.significant.rst +++ /dev/null @@ -1,4 +0,0 @@ -Add a new **Deadlines** page under the Browse menu. - -The page is accessible to any role that already has ``can_read`` and -``menu_access`` on ``DAG Runs``. diff --git a/airflow-core/newsfragments/67684.bugfix.rst b/airflow-core/newsfragments/67684.bugfix.rst deleted file mode 100644 index 9cf4ce96c85..00000000000 --- a/airflow-core/newsfragments/67684.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix ``ONE_FAILED``/``ONE_SUCCESS``/``ONE_DONE`` trigger rules inside a mapped task group being evaluated against every upstream instance instead of the upstream instance sharing the task's own map index, which wrongly triggered the rule for every expanded instance when only one upstream had failed (or succeeded). diff --git a/airflow-core/newsfragments/67688.improvement.rst b/airflow-core/newsfragments/67688.improvement.rst deleted file mode 100644 index d2a641ac00c..00000000000 --- a/airflow-core/newsfragments/67688.improvement.rst +++ /dev/null @@ -1 +0,0 @@ -Further optimize ``TaskGroup.topological_sort`` for reverse-declared DAGs via pass-number traversal; dramatically improves the O(N²) worst-case for adversarial shapes (e.g., reverse-insertion chains). diff --git a/airflow-core/newsfragments/67716.feature.rst b/airflow-core/newsfragments/67716.feature.rst deleted file mode 100644 index 63daeda113c..00000000000 --- a/airflow-core/newsfragments/67716.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Add ``FixedKeyMapper`` and ``SegmentWindow`` for categorical asset-partition rollup. ``FixedKeyMapper`` collapses any upstream partition key onto a single fixed downstream key, and ``SegmentWindow`` enumerates a fixed categorical segment set the scheduler waits for. Composing ``RollupMapper(FixedKeyMapper(...), SegmentWindow(...))`` expresses a categorical rollup, mirroring the temporal rollup shape, and ``SegmentWindow`` also composes with ``FanOutMapper`` for categorical scatter. Both [...] diff --git a/airflow-core/newsfragments/67868.bugfix.rst b/airflow-core/newsfragments/67868.bugfix.rst deleted file mode 100644 index 55535ed4b4f..00000000000 --- a/airflow-core/newsfragments/67868.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -The ``kwargs`` field of trigger objects returned by the REST API (for example in the ``trigger`` of a task-instance response) no longer exposes the decrypted trigger keyword arguments. Those kwargs can contain credentials a deferred operator hands to its trigger (an API key, a token, …), so the field is now always returned empty, as ``"{}"``. The field is retained in the response schema for backwards compatibility — and is now marked ``deprecated`` there so consumers are nudged off it — [...] diff --git a/airflow-core/newsfragments/67873.bugfix.rst b/airflow-core/newsfragments/67873.bugfix.rst deleted file mode 100644 index 5d691367860..00000000000 --- a/airflow-core/newsfragments/67873.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix ``none_failed_min_one_success`` trigger rule evaluation when no upstream task succeeds, including mapped tasks whose upstream instances are all ``removed``. diff --git a/airflow-core/newsfragments/68028.feature.rst b/airflow-core/newsfragments/68028.feature.rst deleted file mode 100644 index fa9db5b21d8..00000000000 --- a/airflow-core/newsfragments/68028.feature.rst +++ /dev/null @@ -1 +0,0 @@ -Human-in-the-loop tasks now wait in a new ``awaiting_input`` task state managed by the scheduler instead of deferring onto the triggerer, so the triggerer can scale to zero while tasks await a human response. Waiting tasks resume directly on a response or on the scheduler's response-timeout sweep. On Airflow versions before 3.3 the operator falls back to the previous trigger-based path. diff --git a/airflow-core/newsfragments/68175.significant.rst b/airflow-core/newsfragments/68175.significant.rst deleted file mode 100644 index 4c9f7b4dffc..00000000000 --- a/airflow-core/newsfragments/68175.significant.rst +++ /dev/null @@ -1,24 +0,0 @@ -Airflow CLI commands are moving to talk to the API server - -The CLI is being migrated to reach Airflow through the API server (via the ``airflowctl`` -client) instead of the metadata database directly. Migrated so far: ``dags trigger``, -``dags delete``, ``pools`` (list/get/set/delete/import/export), and ``assets materialize``; -this fragment is updated as more commands migrate rather than adding new ones. - -These commands now require a reachable API server and mint a short-lived token in memory -(set ``AIRFLOW_CLI_TOKEN`` for auth managers that cannot mint locally, or for remote servers). -``airflow.api.client`` is removed — use ``airflow.cli.api_client.get_cli_api_client``. - -Each migrated command emits a ``RemovedInAirflow4Warning`` and will be removed in a future -Airflow release; use the equivalent ``airflowctl`` command instead. - -* Types of change - - * [ ] Dag changes - * [ ] Config changes - * [ ] API changes - * [ ] CLI changes - * [x] Behaviour changes - * [ ] Plugin changes - * [ ] Dependency changes - * [x] Code interface changes diff --git a/airflow-core/newsfragments/68266.bugfix.rst b/airflow-core/newsfragments/68266.bugfix.rst deleted file mode 100644 index 0d41b654186..00000000000 --- a/airflow-core/newsfragments/68266.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Asset-triggered partitioned Dag runs now set ``partition_date`` when the consumer's partition mapper is temporal (directly, or wrapped in ``RollupMapper`` / ``FanOutMapper`` / ``ChainMapper``). Non-temporal mappers leave ``partition_date`` unset. diff --git a/airflow-core/newsfragments/continuous-optional-start-date.improvement.rst b/airflow-core/newsfragments/continuous-optional-start-date.improvement.rst deleted file mode 100644 index c001c620933..00000000000 --- a/airflow-core/newsfragments/continuous-optional-start-date.improvement.rst +++ /dev/null @@ -1 +0,0 @@ -The ``schedule="@continuous"`` parameter now works without requiring a ``start_date``, and any DAGs with this schedule will begin running immediately when unpaused. diff --git a/reproducible_build.yaml b/reproducible_build.yaml index a316f288e96..7ad5ef6575d 100644 --- a/reproducible_build.yaml +++ b/reproducible_build.yaml @@ -1,2 +1,2 @@ -release-notes-hash: e0c22e0ffced13f516e2cbe7f85fbabc -source-date-epoch: 1780037694 +release-notes-hash: c0417e88d3ce0e117ed80e605b8940a3 +source-date-epoch: 1781638709
