GitHub user potiuk added a comment to the discussion: DagNotFound - Dag already exists and and it failed one time with dag_id not found
This is not airflow issue - this is at most a discussion where you need help with your problem. Most likely the solution you chose for DAG synchronisation is not perfect. There seem to be some race condition that your worker does not have the dag file present in the folder it expects. For example if you have custom DAG syncing solution, it might be that you delete files before you recreate them. Or maybe the syncing in a new pod is not complete when your task starts. This - in k8s environment is usually handled by init container that syncs the files before starting the task. Generally - look at your DAG syncing deployment choice you made and fix it so that you can be 100% sure that DAG files are present when tasks are running. GitHub link: https://github.com/apache/airflow/discussions/43728#discussioncomment-11164142 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
