adriencaccia commented on issue #6149: URL: https://github.com/apache/arrow-rs/issues/6149#issuecomment-2255525620
> I have a question about `codspeed`: Where do these performance tests take place? Are they run on GitHub Actions or on a dedicated environment? They are run on GitHub Actions default runners, using https://github.com/CodSpeedHQ/action/. > How can we ensure stable performance metrics and avoid false positives? Thanks to the action, the benchmarks are run on a simulated CPU and thus the measurements are stable (you can read more about it in [our docs](https://docs.codspeed.io/features/understanding-the-metrics/#execution-speed-measurement)). On the existing benchmarks, you can see that the results are quite consistent: - On a sample PR https://codspeed.io/CodSpeedHQ/arrow-rs/branches/fork-6139-Rafferty97/simplify-struct-array - On a specific benchmark ([link to the page](https://codspeed.io/CodSpeedHQ/arrow-rs/benchmarks/arrow-array/benches/fixed_size_list_array.rs::benches::criterion_benchmark::fixed_size_list_array(len:%204096%2C%20value_len:%201)?branchId=66a462bfb8fd4eed6f2da335)):  The links above is the CodSpeed integration of a forked `arrow-rs`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
