devnp2007 opened a new pull request, #77023:
URL: https://github.com/apache/airflow/pull/77023
### Summary
Implements Asset Deadlines across `task-sdk`, `airflow-core` serialization,
scheduler lifecycle management, database persistence, and the Core REST API:
- **`task-sdk` Definitions**:
- Added `AssetDeadline(cron=..., interval=..., callback=..., name=...)`
with mutual-exclusivity validation (`cron` XOR `interval`), cron syntax and
positive `timedelta` checks, and value-based equality/hashing for cross-Dag
deduplication.
- Added `deadline: AssetDeadline | None = None` on `Asset` and
`run_deadline: AssetDeadline | None = None` on `AssetTriggeredTimetable`.
- **Serialization**:
- Added `encode_asset_deadline` / `decode_asset_deadline` and wired
`deadline` / `run_deadline` through `SerializedAsset` and
`AssetTriggeredTimetable`.
- **Database & Lifecycle Management**:
- Added Alembic migration `0157_3_5_0_add_asset_deadlines.py` extending
`deadline_alert` (`asset_id`, `is_producer`, `cron`) and `deadline`
(`asset_id`, `partition_key`).
- Added `Deadline.schedule_asset_deadlines()` for proactive scheduling on
Dag serialization and scheduler ticks.
- Added `Deadline.prune_asset_deadlines()` (partition-aware) wired into
`AssetManager.register_asset_change()` so arriving `AssetEvent`s prune
satisfied pending deadlines and schedule the next interval window.
- Enriched `Deadline.handle_miss()` callback context with `asset_id`,
`asset_name`, `asset_uri`, `partition_key`, and `is_producer`.
- Deduplicated identical `AssetDeadline` definitions across
producer/consumer Dags in `SerializedDagModel._sync_asset_deadlines()`.
- **Core REST API & OpenAPI**:
- Added `GET /api/v2/assets/{asset_id}/deadlines`
(`AssetDeadlinesResponse`) and updated `v2-rest-api-generated.yaml`.
### Testing
- Unit tests in `task-sdk/tests/task_sdk/definitions/test_deadline.py` (44
tests)
- Lifecycle, partitioning, deduplication, and REST API tests in
`airflow-core/tests/unit/models/test_asset_deadline.py` and existing
deadline/serialization suites (64 tests)
- Verified `ruff check`, `ruff format`, `mypy`, and `airflow db
check-migrations`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
