alamb commented on PR #23032: URL: https://github.com/apache/datafusion/pull/23032#issuecomment-5114703275
> As an end user I don't know what others are using; when you say "I am not convinced that the solution is not over-specialized to your use case" I am confused because I don't have any other frame of reference. From my point of view, my queries are no longer crashing, runtime memory is significantly reduced, and all the tests are passing. I don't know what "a deeper performance analysis" entails. > > You mention "it raises a legitimate performance concern", but what specifically is the metric being measured that is a concern? I look at the benchmarks above and I see some faster and some slower (which makes sense), I don't know which are unacceptably slower. I don't know what tradeoffs are acceptable. I don't know which benchmarks are more important than the others. Are all performance impacting changes unacceptable? > > (maybe cc @alamb for help) In my mind, changes that result in some queries getting faster and some getting slower is problematic because it may be experienced as a regression by some community members My reading of the benchmark results on this PR were inconclusive -- I am not sure after reviewing the results if this PR actually makes some query cases slower or if the results are just noise. So in terms of concrete steps forward I would suggest: 1. Try and reproduce the results that show this branch slower locally -- if you can't that is important details 2. If you can, do some profiling and see if we can find some other place to squeeze performance out -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
