viirya commented on issue #23272: [SPARK-26265][Core] Fix deadlock in BytesToBytesMap.MapIterator when locking both BytesToBytesMap.MapIterator and TaskMemoryManager URL: https://github.com/apache/spark/pull/23272#issuecomment-446067447 Oh, you meant that the page is freed by other using this map or iterator. Is it a problem? I think it should not be a case that more than one consumers free the same page at the same time. On Tue, Dec 11, 2018, 11:34 Wenchen Fan <[email protected] wrote: > *@cloud-fan* commented on this pull request. > ------------------------------ > > In core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java > <https://github.com/apache/spark/pull/23272#discussion_r240463496>: > > > @@ -283,6 +290,9 @@ private void advanceToNextPage() { > } > } > } > + if (pageToFree != null) { > + freePage(pageToFree); > > the MapIterator.spill will be called by BytesToBytesMap.spill which will > be called by other consumers. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/23272#discussion_r240463496>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AAEM91AsLKsA5zS0gXEhip3GuUcVjJ7-ks5u3yfAgaJpZM4ZK2_Y> > . >
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
