GitHub user potiuk added a comment to the discussion: Airflow with CeleryExecutor becomes extremely slow when executing a long-running task
Looks like your long running task takes all the memory - remaining processes start swapping out and slow down to a crawl and they start to compete for swap. Look for memory leak on whatever task you have or but more memory if your task need more GitHub link: https://github.com/apache/airflow/discussions/44430#discussioncomment-11399399 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
