Raul824 commented on issue #35107: URL: https://github.com/apache/airflow/issues/35107#issuecomment-1774401817
@RNHTTR First thing I checked was memory because every time this similar issue occurred it was due to memory. kubectl top nodes - shows 30% memory in use and 70% free. our jobs are passing all the load to databricks and are doing only request get to get the status every 30 seconds. We are not getting OOMkilled on any of the pods as we have provided a pretty large size clusters to Airflow. Could you please help me with the doubt as why the airflow is trying to get the status of job from different worker whereas celery is running in on a different worker. Logs and snippets are in original post, please let me know if any more details I can add which will help. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
