1fanwang commented on issue #43989: URL: https://github.com/apache/airflow/issues/43989#issuecomment-2478545575
Sharing some of my early thoughts on the points raised here and from our offline discussion > "Where should deadlines be stored?" Overall +1 to have it in a new/its own table for clear separation of concerns, this way we can also optimize for querying specifically for deadline, also this gives us the flexibility to support `identifier` that's mentioned here. > Must be able to sort/search by "need-by" We can have a dedicated index for `need-by` > "Does every possible future identifier type have a name format where we can infer what it is?" Maybe we can make this safe and explicit by having an `identifier_type` column instead of inferring from format? > Must be able to remove by "dagrun_id" Something like this? `DELETE FROM deadline_entry WHERE identifier_type = 'dagrun' AND identifier = 'dagrun_123'` > Must be able to allow alternative remove options for future work Assuming this is saying we might query based on different identifier types `DELETE FROM deadline_entry WHERE dag_id = 'dag1' AND task_id = 'task1'` > "1:1 mapping for identifier:deadline" We can enforce unique constraint on (identifier_type, identifier) WDYT -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
