potiuk commented on issue #25585: URL: https://github.com/apache/airflow/issues/25585#issuecomment-1207783225
Yeah. Let's move it to discussion. I think this one is really coming from a race condition that should be occasional and some of them are really unavoidable until we introduce [DAG versioning](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-36+DAG+Versioning). Currently there are pleny of assumptions that our DAGs are in the same state in Scheduler, Webserver and workers and we do not even have a notion of "changing" DAG - there is no way to simply see that a DAG was "there" at a given time but was "missing" a moment later or the other way round. So if it changes in the middle, things like that can happen. As long as it is self-recoverable, this is not a "huge" issue and DAG versioning is I think the only way to solve it "well" (or at least have a chance to be solved in a consistent way). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
