On 6/27/19 1:20 PM, Severin Gehwolf wrote: > On Thu, 2019-06-27 at 11:58 +0200, Aleksey Shipilev wrote: >> On 5/22/19 6:34 PM, Severin Gehwolf wrote: >> hotspot: >> >> *) Test exclusion question. Do we really want to exclude these tests from >> tier1? I would rather see >> them included and make them fail in regular runs. This would put pressure on >> fixing them for 8u... >> >> 150 # Right now tier1 runs all hotspot tests except RTM tests, >> 151 # and one serviceability test which seems flakey. >> 152 hotspot_tier1 = \ >> 153 :jdk \ >> 154 -compiler/rtm \ >> 155 -serviceability/sa/jmap-hashcode/Test8028623.java > > I'm not sure about it. We could exclude nothing and go from there. > However, the more tests we have failing from the get-go the less useful > it is to enforce tier1 tests to always pass for JDK 8u (bad > signal/noise ratio). I'd be OK for Test8028623.java, but the > compiler/rtm tests, I'm not so sure... Thoughts?
From my perspective, new test profile would inevitably show some (new) failures. We would need to work to fix those failures before we can rely on low SNR in tier1. Ignoring tests may skip over the tests that need attention during that transitional period. We don't ignore the tests that are failing right now in fastdebug, right? Same thing here. In other words, I would rather see tier1 suite to land, and then work on fixing/ignoring tests once we understand how/if they are fixable. -- Thanks, -Aleksey
