Ngone51 edited a comment on issue #24467: [SPARK-27568][CORE] Fix readLock leak while calling take()/first() on a cached rdd URL: https://github.com/apache/spark/pull/24467#issuecomment-490136001 > For this particular case, I think using a task completion listener is good enough? Block level read/write lock mechanism has a basic assumption that *all block locks should be released when a task finished*. And that's why we check the leaked locks after the task finished. As task completion listener also would be triggered after task finished, so I think using it may not take big difference. Actually, *The process of checking leaked locks*(by calling `releaseAllLocksForTask(taskId)`) is just like a `close` operation of database iterator, which releases all unlocked blocks for the task. So, I agree with @jiangxb1987 and prefer not to fix it if we could not have a better way against to this hacky way by now.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
