HippoBaro commented on PR #11267: URL: https://github.com/apache/arrow-rs/pull/11267#issuecomment-6085737613
@alamb @etseidl I’ve done some work on the benchmark bot to support arbitrarily large benchmark suites through sharding. The failed run used `shards: 1`, so the entire suite ran on a single runner. See: https://github.com/adriangb/datafusion-benchmarking/pull/40, https://github.com/adriangb/datafusion-benchmarking/pull/41 and https://github.com/adriangb/datafusion-benchmarking/pull/44 My :2c: is that the missing piece is some configuration in this repo declaring that, for example, the `arrow_writer` suite needs at least 4 shards to finish within the 2h deadline. As of right now you must specify it on each and every invocation. My reasoning is that benchmarks are only useful if we actually run them systematically. Narrowing the suite through filters implicitly assumes we already know where performance will change, which risks hiding regressions in places we did not anticipate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
