GitHub user rcrchawla added a comment to the discussion: Airflow task failed but spark kube app is running
Okay currently we have de-scaled our scheduler from 2 to 1 will it help getting some performance improve. We do not have that kind of minimal environment where there is huge traffic of jobs. Currently task_instance_heartbeat_timeout -- 900 and worker concurrency is 16 with 2 workers running earlier also we used to have 16 concurrency in airflow version 2.2.2 and we have upgraded to airflow 3.0.4. Will de-scaling of scheduler server help ? GitHub link: https://github.com/apache/airflow/discussions/63298#discussioncomment-16092588 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
