easontm commented on issue #18843: URL: https://github.com/apache/airflow/issues/18843#issuecomment-939400375
I'm experiencing a similar issue. If a task is in the `scheduled` state and the DAG code is temporarily removed (e.g. part of DAG CICD), the DAG processor will delete the associated information in `serialized_dag` while the task still exists. Then if the scheduler tries to transition the task from `scheduled` to `queued` before the new code is serialized, it will crash the scheduler. Upon reboot, one of the first things the scheduler tries to do is adopt orphan tasks, and adoption is attempted before DAG serialization, resulting in a crashloop. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
