Re: [I] Clearing a running dag run does not reset the queued_at timestamp [airflow]
ferruzzi commented on issue #59220: URL: https://github.com/apache/airflow/issues/59220#issuecomment-3638863462 Started an email thread on the dev list to get community thoughts. My proposal is that we should always kick the run back to the queue and update the queued_at regardless of the current state. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Clearing a running dag run does not reset the queued_at timestamp [airflow]
ferruzzi commented on issue #59220: URL: https://github.com/apache/airflow/issues/59220#issuecomment-3633731322 Reading through the [original Issue](https://github.com/apache/airflow/issues/30124) that spawned that change, the request was not to change the start_date which I agree with, but I feel like the queued_at time should reset -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Clearing a running dag run does not reset the queued_at timestamp [airflow]
henry3260 commented on issue #59220: URL: https://github.com/apache/airflow/issues/59220#issuecomment-3633707504 I suspect this change was made for performance optimizations or to simplify the state transitions. That said, I agree the current behavior feels unintuitive. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Clearing a running dag run does not reset the queued_at timestamp [airflow]
ferruzzi commented on issue #59220: URL: https://github.com/apache/airflow/issues/59220#issuecomment-3633660756 Hm, I wonder if that's still the desired behavior. I can do an easy deadline-specific workaround by adding a check before the `if dr.state in State.finished_dr_states: ` but I wonder if it's something we want to change. What are the advantages of this way instead of just always flagging it QUEUED? Are we saying that if it's currently running when a user clears it, that it just restarts instead of getting tossed back into the queue? That doesn't seem like the expected behavior, but maybe I'm wrong. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Clearing a running dag run does not reset the queued_at timestamp [airflow]
tirkarthi commented on issue #59220: URL: https://github.com/apache/airflow/issues/59220#issuecomment-3632274911 Clearing a dagrun in running state doesn't move it to queued to be moved to running state by scheduler. The dagrun remains in running state. Related commit that made this change in Airflow 2.7.0 before which clearing will always set the dagrun state to queued. Perhaps for deadlines use case the `dr.queued_at` could be updated irrespective of the dagrun's current state during clearing but that's still semantically little different because dagrun never moves to queued state to update `queued_at` and then to later become running. Ref commit : https://github.com/apache/airflow/commit/070ecbd87c5ac067418b2814f554555da0a4f30c -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Clearing a running dag run does not reset the queued_at timestamp [airflow]
tirkarthi commented on issue #59220: URL: https://github.com/apache/airflow/issues/59220#issuecomment-3632248904 I tried debugging this. The docstring for the method has a note on this. The `start_date`, `clear_number` and `queued_at` are updated only for dagruns in finished state with `dr.state in State.finished_dr_states` being false for a running dagrun. https://github.com/apache/airflow/blob/7218cf045812a5a7135d7f119cdde0aed817b132/airflow-core/src/airflow/models/taskinstance.py#L200-L206 https://github.com/apache/airflow/blob/7218cf045812a5a7135d7f119cdde0aed817b132/airflow-core/src/airflow/models/taskinstance.py#L281-L307 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Clearing a running dag run does not reset the queued_at timestamp [airflow]
henry3260 commented on issue #59220: URL: https://github.com/apache/airflow/issues/59220#issuecomment-3629921791 Can I work on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
