On Mon, 31 Mar 2025 16:01:39 GMT, Kevin Rushforth <k...@openjdk.org> wrote:
>> The intent is not to annotate each test, but rather to use this tool to >> debug the issues in an intermittently failing test. > > Hmm. That isn't what I thought we were doing. I thought the idea was to > annotate most (if not all all) screen capture tests, qualified by a system > property, enable that property in our CI test runs, so that when we do get > intermittent failures, we'll be able to take a look at them. > > We _could_ to what you propose, but in that case I wouldn't want _any_ tests > annotated as part of this or any other PR. Once we have this annotation on > any test in our repo, then my objection about not having it on by default > needs to be addressed, at which point you might as well annotate more tests, > since we have seen occasional failures on many of them. There is more than one way to sk^H^H pet the cat. We could use a property to disable (or rather, enable) the screenshots, and only enable the capture during the debugging session. This will prevent us from catching those hard-to-reproduce intermittent tests that fail only occasionally. The other concern is that in the case of some infrastructure misconfiguration (for example, a sudden loss of screen capture permission) would result in log file size explosion, and I don't think we have `tailwatch`-like facility to halt the test run when this happens. I think the debug-only annotation is a meaningful compromise, but I would very much welcome other ideas. (I am ok with removing the annotation from two tests that currently use it). ------------- PR Review Comment: https://git.openjdk.org/jfx/pull/1746#discussion_r2021406569
