andygrove commented on pull request #7951: URL: https://github.com/apache/arrow/pull/7951#issuecomment-673768514
@alamb How would you feel about us merging this, and then you follow up with an improved approach? I did have a quick attempt at using Rayon but you would probably be able to do this faster and more correctly than I could right now. With this and the other pending PRs, I can run TPC-H query 1 against a 100 GB data set with 240 partitions with reasonable performance (about the same as Apache Spark). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
