alamb commented on PR #11267: URL: https://github.com/apache/arrow-rs/pull/11267#issuecomment-6084618957
> Not just this one. arrow_reader is not far behind. I'd be in favor of splitting these huge ones into more targeted sets. We could run the full suite as separate jobs to hunt for hidden regressions at some interval, but only run the targeted benches during development. This is what we do now with the bench filter, but sometimes it's hard to know what filter to use. Yeah, I agree The long each individual benchmark runs the more chance of noise in the benchmark as well (as there is more time between when a benchmark is run on the merge-base and the PR, so there is more chance that the other load on the machine has changed as well) Maybe we can have a `arrow_writer` test suite with some critical benchmarks, and then an `arrow_writer_full` for all of them Another thing I have seen is update the benchmark binary to look for an environment variable to opt in to some tests so like ```shell cargo bench --bench arrow_writer ``` Would run the "default" tests and you would do something like this to run all of them: ```shell ARROW_WRITER_FULL=true cargo bench --bench arrow_writer ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
