tirkarthi commented on issue #24171: URL: https://github.com/apache/airflow/issues/24171#issuecomment-2296062775
I just want to add that we have been using yarn rest API to fetch application status in our custom deferrable operators related to spark on yarn and it has been working fine. We had also implemented the logic to kill the application when the task gets cleared or marked as success/fail from UI. https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerRest.html#Application_API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
