deepchand commented on issue #19192: URL: https://github.com/apache/airflow/issues/19192#issuecomment-1224015314
@potiuk i have debugged more on this issue and found like updating the task state in same scheduler loop significantly increase the total time of self.executor.heartbeat() function which is causing this problem `"The scheduler does not appear to be running. Last heartbeat was received X minutes ago"` To be specific https://github.com/apache/airflow/blob/9ac742885ffb83c15f7e3dc910b0cf9df073407a/airflow/executors/celery_executor.py#L312 the state which we are passing in this function is kind of blocking and if we stop updating task state from here significantly decrease the scheduling loop time and also not giving this warning `"The scheduler does not appear to be running. Last heartbeat was received X minutes ago" ` Please have a look and let me know if any other info is needed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
