Rich-T-kid commented on PR #26315: URL: https://github.com/apache/datafusion/pull/26315#issuecomment-6688946519
Closing this WIP because the end-to-end benefit does not justify the added configuration and maintenance surface. I retested the relevant TPC-H SF1 queries with the Arrow dictionary row-cache experiment: Q4 was not significantly faster than the default Utf8View path, while Q12, Q20, and Q22 remained 13.3%, 11.9%, and 13.4% slower. The Arrow cache only produced a sub-millisecond incremental gain on Q4 and did not change the overall result. We can revisit an NDV cap if future Arrow/DataFusion changes make dictionary reads consistently competitive with the default representation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
