Good point. The raw numbers are available in the test run output. See https://builds.apache.org/view/A-D/view/Beam/view/PostCommit/job/beam_PreCommit_Java_Cron/2718/testReport/ for the "skipped" column. And you get the same on console or Gradle Scan: https://scans.gradle.com/s/ml3jv5xctkrmg/tests?collapse-all This would be good to review periodically for obvious trouble spots.
But I think you mean something more detailed. Some report with columns: Test Suite, Test Method, Jira, Date Ignored, Most Recent Update I think we can get most of this from Jira, if we just make sure that each ignored test has a Jira and they are all labeled in a consistent way. That would be the quickest way to get some result, even though it is not perfectly automated and audited. Kenn On Tue, May 5, 2020 at 2:41 PM Jan Lukavský <je...@seznam.cz> wrote: > Hi, > > it seems we are accumulating test cases (see discussion in [1]) that are > marked as @Ignored (mostly due to flakiness), which is generally > undesirable. Associated JIRAs seem to be open for a long time, and this > might generally cause that we loose code coverage. Would anyone have > idea on how to visualize these Ignored tests better? My first idea would > be something similar to "Beam dependency check report", but that seems > to be not the best example (which is completely different issue :)). > > Jan > > [1] https://github.com/apache/beam/pull/11614 > >
