tgravescs commented on pull request #31152: URL: https://github.com/apache/spark/pull/31152#issuecomment-758703519
so I'm a little confused by this. so I went back to the history. This seems to be used by Spillable, HashRelation, and RowQueue. It looks like at the time those were added, the MemoryConsumer didn't support a memory mode. It looks like when that memory mode was added spilling was only support to on heap by Spillable (looking at the comment ing): https://github.com/apache/spark/commit/8fb1d1c7f3ed1b62625052a532b7388ebec71bbf Is that not the case now? Looking at: https://github.com/apache/spark/commit/8fb1d1c7f3ed1b62625052a532b7388ebec71bbf#diff-cc874d683910c317079b6544d47ac8bb1c5d7a35faaa4dc7c65951c6dd283508L110 it seems like its only on heap. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
