Yongjun Zhang created HADOOP-11045: -------------------------------------- Summary: Introducing a tool to detect flaky tests of hadoop jenkins test job Key: HADOOP-11045 URL: https://issues.apache.org/jira/browse/HADOOP-11045 Project: Hadoop Common Issue Type: Improvement Components: build, tools Reporter: Yongjun Zhang Assignee: Yongjun Zhang
File this jira to introduce a tool to detect flaky tests of hadoop jenkins test jobs. I developed the tool on top of some initial work [~tlipcon] did. We find it quite useful. With Todd's agreement, I'd like to push it to upstream so all of us can share (thanks Todd for the initial work and support). I hope you find the tool useful. This is a tool for hadoop contributors rather than hadoop users. Thanks [~tedyu] for the advice to put to dev-support dir. Description of the tool: # # Given a jenkins test job, this script examines all runs of the job done # within specified period of time (number of days prior to the execution # time of this script), and reports all failed tests. # # The output of this script includes a section for each run that has failed # tests, with each failed test name listed. # # More importantly, at the end, it outputs a summary section to list all failed # tests within all examined runs, and indicate how many runs a same test # failed, and sorted all failed tests by how many runs each test failed in. # # This way, when we see failed tests in PreCommit build, we can quickly tell # whether a failed test is a new failure or it failed before, and it may just # be a flaky test. # # Of course, to be 100% sure about the reason of a failed test, closer look # at the failed test for the specific run is necessary. # -- This message was sent by Atlassian JIRA (v6.3.4#6332)