viirya commented on issue #23272: [SPARK-26265][Core] Fix deadlock in BytesToBytesMap.MapIterator when locking both BytesToBytesMap.MapIterator and TaskMemoryManager URL: https://github.com/apache/spark/pull/23272#issuecomment-446067993 If you worry that the page is to be freed by the other consumer using the map iterator and also the map iterator itself, because I am not in front of laptop so I can't check it. But I guess freePage should already cover it. On Tue, Dec 11, 2018, 11:34 Wenchen Fan <[email protected] wrote: > *@cloud-fan* commented on this pull request. > ------------------------------ > > In core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java > <https://github.com/apache/spark/pull/23272#discussion_r240463496>: > > > @@ -283,6 +290,9 @@ private void advanceToNextPage() { > } > } > } > + if (pageToFree != null) { > + freePage(pageToFree); > > the MapIterator.spill will be called by BytesToBytesMap.spill which will > be called by other consumers. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/23272#discussion_r240463496>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AAEM91AsLKsA5zS0gXEhip3GuUcVjJ7-ks5u3yfAgaJpZM4ZK2_Y> > . >
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
