lukas-at-harren commented on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-824894652
@ashb The setup works like this (I am @overbryd, just a different account.) * There are dynamic DAGs that are part of the repository, and that gets pulled in regularly with a side-car container on the scheduler. * The worker has an init container that pulls the latest state of the repository. * Secondly I have DAGs that are built (very quickly) based on database query results. This scenario opens up the following edge cases/problems: * A DAG can go missing, after it has been queued when the entry is removed from the database, rare but happens. * A DAG can go missing, after it has been queued when a git push happens, which change the DAGs, before the worker started. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
