On Tue, 2 Feb 2021 01:40:09 GMT, Martin Buchholz <mar...@openjdk.org> wrote:
>> @cl4es I agree pruning is a good idea. I settled on 3 data points with 16x >> separations as good enough to clearly show the difference between O(1) O(N) >> O(N^2) and O(2^N) (although O(2^N) would "run forever"). >> >> (although ... please tell me you're not actually running these benchmarks in >> an automated fashion ... too expensive, and needs a human to interpret the >> results) > >> A manual exploration of a new set of micros would naturally start with >> the default config, so if such a config runs forever, that would be poor >> ergonomics IMHO. I don't think such configurations should be checked in >> in an active state. > > We're actually in agreement. There's no actual O(2^N) operation here, and > jmh is similar to jtreg in having timeouts indicating failure. > > Although I've been using shell loops as included in the class comments, I'll > make sure running the tests using all the defaults gives sensible results. > > I'm surprised to see jmh use the same number (5) of warmup and measurement > iterations. Unless you're looking for very small effects, one warmup run > should be sufficient. I added annotations for sensible (faster) default, including @Warmup(iterations = 1) ------------- PR: https://git.openjdk.java.net/jdk/pull/1940
