GitHub user JinRenNg closed the discussion with a comment: Kubernetes scheduler liveness probe fails during OOM conditions preventing automatic restart
I will proceed to close this discussion for the moment as we are currently adopting airflow 3.x. We found out that the architecture have been greatly enhanced, especially where there is a new dag_processor component which have offloaded the DAG parsing job from the scheduler. It would greatly help to reduce the overwhelming issue on the schedueler in airflow 2.x. Hence, instead of working on pre airflow 3.x issue, we would love to move forward with the new architecture. GitHub link: https://github.com/apache/airflow/discussions/53662#discussioncomment-13907208 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
