GitHub user pykenny added a comment to the discussion: Need help understanding total number of Dags oscillating on UI
Sorry, I seemed to made the wrong statement about the behavior earlier -- DAG processor updates the metadata database, and tries to infer state from database as well, not communicating to scheduler directly. When running standalone parser, scheduler look up the timestamp and stale duration to decide if some DAGs are out-of-sync. So perhaps there are some job overlapping between processor and scheduler, but I'm not certain about the assumption/idea behind the design (for instance, should scheduler deactivate all the DAGs when DAG processor's down?). GitHub link: https://github.com/apache/airflow/discussions/44495#discussioncomment-12235149 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
