hhhizzz commented on PR #23182: URL: https://github.com/apache/datafusion/pull/23182#issuecomment-4806653729
> > I remember 5% ish noise is likely for short-running queries > > Yes, my point is that q23 was only about **1.14x faster on SF1**, but on my server, running **SF10 produced around a 3x speedup**. This makes me wonder whether we should require at least **SF10+ runs** before merging performance-related PRs. I reran the benchmarks on my machine with SF1, and it looks like there's essentially no difference. It seems TPC-DS SF1 just isn't suitable for evaluating this kind of optimization. In hindsight, it also wouldn't have been able to reveal the impact of your previous changes. That said, I did notice a **+42.81x** improvement on ClickBench, so the optimization still appears to be effective.😁 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
