tustvold commented on issue #1542: URL: https://github.com/apache/arrow-rs/issues/1542#issuecomment-1270140912
Coming back to this, I'm not sure this is such a good idea anymore. The take kernel results in a substantial amount of codegen, as specialized implementations are generated for every possible array. Just compiling the current take kernels for all the different types of dictionary key represents a substantial amount of codegen and compilation time, multiplying this by all the various different types of iterator, is likely to make this orders of magnitude worse. I also found when implementing the specialized filter kernels that it was oftentimes faster to materialize the indices than to use an iterator, as it was easier for the compiler to reason about correctly. TLDR I think this would likely explode compilation times, and bloat the final executables, without necessarily yielding any performance wins -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
