On Wed, 20 Aug 2025 15:21:39 GMT, Magnus Ihse Bursie <i...@openjdk.org> wrote:
>> Leo Korinth has updated the pull request incrementally with one additional >> commit since the last revision: >> >> after suggestion from Daniel > > It seems to me like there is a need to automatically collect normal test run > times automatically, so we can match the timeout given to any individual test > with the normal execution time. After all, the purpose of any timeout on > tests is to allow the test to execute normally, but not wait too long in case > of a problem that causes the test to take too long (or forever) to run. > > I realize that this is highly hardware dependent, but test times tend to be > Pareto distributed, so a very quick test maybe takes 1 second on fast > machines and 10 on slow, and very slow tests maybe take 15 minutes on fast > machines and 40 minutes on slow. In the first case, anything above 15 seconds > is probably sus, and in the second case, anything above 60 is probably not > good either (I'm just adding 50% to the max time). > > Right now it seems engineers is spending their valuable time giving > guesstimates for each individual test. That seems like poorly used time and > resources. > @magicus unfortunately that is often not the case in practice. We can see > many tests that normally run very quickly but occasionally run very slow - > minutes versus seconds. That is surprising and a bit disappointing. I guess it is not worth the effort to try and figure out why this is the case; it could probably vary from test to test and be difficult to track for little gain. Still, it makes you wonder. ------------- PR Comment: https://git.openjdk.org/jdk/pull/26749#issuecomment-3210365303
