timarmstrong commented on pull request #34245: URL: https://github.com/apache/spark/pull/34245#issuecomment-940360769
This looks correct from the point of view of avoiding races. It looks like there were some concerns about deadlocks mentioned in https://github.com/apache/spark/pull/24699#discussion_r287408826 . Did you take a look at that? It's definitely easy to have subtle deadlocks with any kind of synchronisation in callbacks. I took a look myself and it looks OK to me but I am new to the code so may be missing something - TaskContextImpl.markTask{Completed,Failed} both seem to drop the TaskContext lock before invoking the listeners. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
