mbutrovich commented on PR #2991: URL: https://github.com/apache/datafusion-comet/pull/2991#issuecomment-3774494706
> > I'm also not convinced the dependency on the `memchr` crate and `Finder` is the real performance win here, but rather avoiding the array generation similar to [apache/datafusion#19529](https://github.com/apache/datafusion/pull/19529). Have we measured both changes independently, because the DataFusion changes shows good speedup on its own. I'd like to avoid maintaining possibly unnecessary dependencies and/or code. > > Should we have comparison with latest DataFusion version ? We already seeing improvements with this implementation at least in current version. We need numbers to isolate the changes. I suspect the `memchr` is not the huge win here, as I mentioned above. Changes should be tested independently, especially something like a specific SIMD implementation (which Arrow kernels have moved away from). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
