bramhanandlingala opened a new pull request, #67430:
URL: https://github.com/apache/airflow/pull/67430
## Changes
### 1. Remove DagRun back-fill from task-emitted outlet events
Previously, if a task emitted outlet events and all of them used the same
`partition_key`, that key was silently written back to `DagRun.partition_key`
(if the run did not already have one). This was wrong, the DagRun key
represents *why the run was started* (set at trigger time), not *what the task
happened to produce*. Tasks should not change the DagRun's identity.
**Removed:** the `runtime_pks` accumulator and the back-fill block in
`register_asset_changes_in_db` in `taskinstance.py`.
### 2. Extra-only outlet events now inherit `dag_run.partition_key`
When a task emits an outlet event with only `extra` data and no explicit
`partition_key`, the event now inherits the DagRun-level `partition_key`.
Before this fix, those events stored `None` even when the DagRun had a key,
making the event's provenance invisible for "extra-only" notifications.
### 3. Warn when `add_partitions()` is called on an asset alias accessor
`OutletEventAccessor.add_partitions()` on an alias outlet has no effect,
aliases do not route partition keys to downstream assets. The key just sits in
the alias event's `partition_keys` set and goes nowhere.
A `UserWarning` is now raised to steer authors toward calling
`add_partitions()` on the target asset accessor directly.
### 4. Reject `partition_key` at the REST API trigger endpoint for
non-partitioned DAGs
`POST /dags/{dag_id}/dagRuns` now returns **HTTP 400** if `partition_key` is
supplied but the DAG's timetable has `partitioned = False`. Storing a
`partition_key` on a run that can never use it leads to confusing query results
and silent data loss.
## Tests
- `test_runtime_partition_key_does_not_backfill_dag_run` — verifies the
DagRun key is NOT written back by task emission
- `test_extra_only_event_inherits_dag_run_partition_key` — verifies
extra-only events pick up the DagRun key
- `test_runtime_partition_key_mixed_events_for_same_asset` — updated
expected set to reflect inheritance
- `test_add_partitions_on_alias_emits_warning` — checks the `UserWarning`
fires and keys still accumulate locally
- `test_should_respond_400_if_partition_key_on_non_partitioned_dag` —
verifies the REST API rejects the invalid combination
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
