FelixYBW commented on issue #4392: URL: https://github.com/apache/incubator-gluten/issues/4392#issuecomment-3392921591
> [@FelixYBW](https://github.com/FelixYBW) Yes, for BHJ, we need to store the raw data for building the hash table in on-heap memory, and then have the driver distribute it to each executor. Even if we follow Spark's implementation approach of building the hash table only once on the driver side, we still need to store the hash table in on-heap memory before distributing it. Do you know for hash join, if worker build the hash table in onheap memory or is able to build in offheap? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
