Dandandan commented on issue #4093: URL: https://github.com/apache/arrow-datafusion/issues/4093#issuecomment-1302092719
Hi - what parts exactly do you want to compress? Hash collisions are handled when matching the rows, they don't need to be kept in memory across batches. For joins that need to remember visited rows (left joins / left semi/left anti joins I believe) we are already using a bitmap based on Arrow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
