massamany commented on pull request #19528: URL: https://github.com/apache/airflow/pull/19528#issuecomment-977753237
Hi, I'm not 100% sure, but I think that I'm experiencing a bug related to this issue. I currently use Airflow 2.2.0 to schedule several DAGs. I've one which is scheduled to run every 2 minutes. In this DAG, I'm using the latest_only operator with the catchup activated. The start date of the DAG is computed before launching Airflow and is valued with the start date of the previous scheduling period. Description of my reproduction case : - At startup, the database is empty. - The DAG is paused at first launch by default. - After Airflow is up, I wait several minutes and then activate the DAG. In 2.2.0 : - Airflow up at 11:11 - Start date computed to be 11:08 as expected - when the DAG is activated at 11:18, the missed execution are launched, but skipped thanks to the latest_only. - the last execution runs OK. - then the DAG is correctly scheduled every 3 minutes. - conclusion : everything is OK See screeshots of the DAG details :  See screenshots of the Tree view after several minutes (last execution at 11:22 for the scheduling period [11:20-11:22]) :  In 2.2.1 AND 2.2.2 : - Airflow up at 11:24 - Start date computed to be 11:22 as expected - when the DAG is activated at 11:32, the first missed execution (for 11:22) is launched, but skipped thanks to the latest_only. - then 2 minutes later (at 11:34), the second execution (for 11:24) is launched, but skipped thanks to the latest_only. - then the DAG is stuck in this state, the "next run" field still indicates 11:24 and no execution will never be scheduled anymore. - conclusion : may-be I'm doing something wrong but it seems to me that there might be a bug here related to this current issue. See screeshots of the DAG details :  See screenshots of the full Tree view screen after several minutes (taken at 11:41) :  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
