0x26res commented on issue #40435: URL: https://github.com/apache/airflow/issues/40435#issuecomment-2274310683
@wolfier does each deferred task start a new process? It wouldn't surprised me, if there are a lot of deferred tasks starting/running at the same time that the overhead of each process causes some of them to be killed because OMM / lack of memory. Does the scheduler throttle / limit the number of deferred tasks that are running at the same time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
