potiuk commented on issue #15588: URL: https://github.com/apache/airflow/issues/15588#issuecomment-1529408801
> Did we get any solution for this? We are also running into this problem now > We have the same problem, has anyone got a solution for this? It'd be great if after restarting the scheduler, either it picked up the same Databricks job or at least it failed. In our case it just keeps creating new tasks on every restart. @syedatifakhtar @JoseMLopezSanz - please try to act in the way that might lead to aither yourself helping yourselves or someone else who might want to help you have at least a chance to help you. If you also use Airflow 2.0.0 (as the orignal author did) - because you have completely forgottent to mention it - then I recommend you to upgrade to latest released version of Airflow. There is a high chance that the issue observed in 2.0.0 has been fixed because 2.0.0 has been released almost 3 years ago and there were ~ 20 releases or so with likely 1000s of bugfixes. So before commenting on old issue like that I highly recommend to upgrade to latest Airflow and try it there. If it is still there (this or similar issue) reporting the issue against the new version of Airflow is the best way to go @JoseMLopezSanz @syedatifakhtar - and in those NEW issues that you will open (if it happens that you have some problem) please describe your configuration, deployment, executor, version of airlfow, provide the logs of around the thing it happens. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
