devnp2007 commented on issue #76997:
URL: https://github.com/apache/airflow/issues/76997#issuecomment-6847272963
Hi @cmarteepants, I would love to work on this feature and help implement
asset-level deadline alerts!
Before opening a PR, I want to align on the design and API details so the
implementation fits cleanly into the existing `DeadlineReference` architecture
(#44003) and AIP-86:
### Proposed Plan of Action
1. **API & Serialization (`AssetDeadline`):**
- Add an optional `deadline: AssetDeadline | None = None` parameter on
`Asset` (supporting both `cron` and `interval` thresholds plus `callback`) and
`run_deadline` on `PartitionedAssetTimetable`.
- Serialize deadline definitions during Dag parsing so the
scheduler/triggerer can evaluate them without re-importing Dag files.
2. **Producer vs. Consumer & Deduplication:**
- Distinguish producer contracts vs. consumer expectations based on
whether the declaring Dag registers the asset as an `outlet` or in `schedule`
(with standalone/watcher-fed assets treated at the asset level).
- Deduplicate identical declarations across Dags using a deterministic
signature (e.g., hash of asset URI + threshold + callback reference) so shared
asset definitions trigger a single callback while still surfacing all declaring
Dags in the Asset UI.
3. **Evaluation & Partitioned Assets:**
- Extend the deadline evaluation loop to monitor missing asset events on
a cadence (including per-partition key checks derived from the producer's
partition timetable for time-based partitions, passing the missing key(s) into
the callback context).
4. **UI & Tests:**
- Surface declared deadlines on the Asset view (marking the producer's
deadline as the contract) alongside full unit/scheduler tests.
Could you assign this to me, and let me know if this breakdown—especially
around the `AssetDeadline` API shape and deduplication behavior—matches what
you have in mind?
-this was written with the help of AI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
