korowa commented on issue #5504: URL: https://github.com/apache/arrow-datafusion/issues/5504#issuecomment-2021270695
Just some thoughts based on #9800 -- at this moment benchmark results may have significant fluctuations (which is expected), but on the other side, basing on example from #9461 (https://github.com/splitgraph/arrow-datafusion/pull/1) shows that is fine to use them for tracking some significant performance regressions. Maybe we should increase 5% "no change" threshold for these benchmarks? It won't be able to show majority of improvements, but it still will be able to highlight if changes are literally breaking in terms of performance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
