potiuk commented on issue #8722: URL: https://github.com/apache/airflow/issues/8722#issuecomment-1160175121
This is not how it works (and it will never work like that). It's mixing the responsibilities of Airflow with Deployment @astolle Only K8S knows why the POD has been killed and when the POD is killed because of OOM, it has no chance to even know and react on it. Making sure that resources are well allocated and monitoring it, is not an Airflow "task". It's the job of monitoring deployment - you should monitor all OOMs and react to it. K8S is the "monitor" of the pods in this case not Airflow. Just make sure, monitoring of the K8S cluster is in place, the same way as you'd monitor any other application on K8S. Airflow is no different ther @astolle @Sinsin1367. I am turning it into a discussion, but I believe (unless we change Airflow into resource management platform as well) that this something we will never change and it should be handled by proper cluster monitoring. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
