Ngone51 commented on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-656187044
Hi all, I'm recalling this PR because I just find that there's another developer @sarutak meet the same issue(SPARK-32236). It has less impact for end-users but still occasionally caught by developers. So I think it's still better to fix it. And since we've already experienced many discussions here, so I think it may be better to continue here(thanks sarutak!) Base on all the above dissuasions, I decide to move this PR back to the original solution, which simply sleeps one more second in the stop of `LocalSparkCluster`. Both sarutak and I think it's the simplest compromise for the issue. On the other hand, as you know, any possible deterministic solutions me and @gerashegalov have tried can introduce overhead changes. The simple solution won't introduce any side effects. Besides, it works most times practically, though it's not deterministic theoretically. cc: @jiangxb1987 @dongjoon-hyun ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
