easontm commented on issue #19901: URL: https://github.com/apache/airflow/issues/19901#issuecomment-997355672
> yes, just another single run I think I'm experiencing a similar circumstance (on 2.2.2), however I usually get more than one run after causing a re-process. It's not specific to any particular DAG, but there is some kind of condition (that I can't replicate on-demand) which results in the scheduler no longer creating new DAGruns. It always happens with DAGs whose `max_active_runs` is easy to hit, in the 1-3 range. IIRC every time I've seen this, the column `DAG.next_dagrun_create_after` is empty for that DAG -- but not every DAG that has this field empty has this issue. Another correlated symptom is that affected DAGs no longer get any scheduler/processing logs (in `$AIRFLOW_HOME/logs/scheduler/<DATE>/...`). This happens to me maybe 2-3 times a month and I have all my (`INFO`) scheduler logs, although I haven't seen any anomalies myself. If anyone with better insight on the scheduler's state machine can tell me what to look for in the logs I can search deeper. Or I can even add some log points on my installation to collect until the anomaly occurs again. @alexakra do your blocked DAGs have processing logs in the above path? And just wondering, what's your setting for `schedule_after_task_execution`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
