jiangxb1987 commented on issue #24467: [SPARK-27568][CORE] Fix readLock leak while calling take()/first() on a cached rdd URL: https://github.com/apache/spark/pull/24467#issuecomment-489827803 AFAIK this shall not lead to any job failure because the config "spark.storage.exceptionOnPinLeak" is normally turned off. However this is really a issue when people submit jobs from python side, and I submitted https://github.com/apache/spark/pull/24542 to catch the AssertionError. To me the fix proposed in this PR is acceptable, but I'm not sure whether we shall still fix this since now it shall not cause critical issues and the fix itself is kind of hacky.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
