gengliangwang commented on PR #36424: URL: https://github.com/apache/spark/pull/36424#issuecomment-1119226177
@tanvn Thanks for trying the new patch. Yes the current approach doesn't fix all the race condition. I take a look at the description again. The method `isProcessing` should only return false when all the updates are done. I wonder why the filtering doesn't work in https://github.com/apache/spark/blob/v3.2.1/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala#L593 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
