christine-le commented on issue #15607: URL: https://github.com/apache/airflow/issues/15607#issuecomment-911984237
Just wanted to chime in to say we ran into the exact same error under pretty similar set up as @neilharia7 with a set of dynamic dags. Setting processor_poll_interval to 30 seconds (up from the default of 1 second) resolved the error for us. We may play around with this number a bit and see if we can reduce this number a bit. We have a couple thousand dags, many of which are dynamically generated, so may just need more time for parsing. Details of our set up: Python 3.6 Airflow 2.0.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
