[jira] [Commented] (HBASE-18540) Ad-hoc job for checking if a test is flaky
[ https://issues.apache.org/jira/browse/HBASE-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118915#comment-16118915 ] Jonathan Hsieh commented on HBASE-18540: I use this script on my lappy. Making it job sounds like it could be useful. http://github.mtv.cloudera.com/jon/bin/blob/master/loopfail.sh > Ad-hoc job for checking if a test is flaky > -- > > Key: HBASE-18540 > URL: https://issues.apache.org/jira/browse/HBASE-18540 > Project: HBase > Issue Type: New Feature > Components: test >Reporter: Sean Busbey > > It would be great if we had an ad-hoc job that could run a given test a bunch > of times and tell me if it is flaky. > I'm trying to evaluate the state of our new nightly jobs, and I'm not sure if > hte failures I'm seeing are a) things I need to get fixed soon, b) things > that are flaky but previously unrecognized, c) things that are flaky but > missed because our current flaky list is only based on the behavior of the > master branch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18540) Ad-hoc job for checking if a test is flaky
[
https://issues.apache.org/jira/browse/HBASE-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118914#comment-16118914
]
Mike Drob commented on HBASE-18540:
---
Last time I had to do this, I used basically this recipe:
{code}
mvn clean package -DskipTests
for i in {1..100}; do
mvn test -q -Dtest=$TEST
echo $? >> results
done
echo $TEST is $(grep -c 1 results) % flaky.
{code}
> Ad-hoc job for checking if a test is flaky
> --
>
> Key: HBASE-18540
> URL: https://issues.apache.org/jira/browse/HBASE-18540
> Project: HBase
> Issue Type: New Feature
> Components: test
>Reporter: Sean Busbey
>
> It would be great if we had an ad-hoc job that could run a given test a bunch
> of times and tell me if it is flaky.
> I'm trying to evaluate the state of our new nightly jobs, and I'm not sure if
> hte failures I'm seeing are a) things I need to get fixed soon, b) things
> that are flaky but previously unrecognized, c) things that are flaky but
> missed because our current flaky list is only based on the behavior of the
> master branch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18540) Ad-hoc job for checking if a test is flaky
[ https://issues.apache.org/jira/browse/HBASE-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118803#comment-16118803 ] Sean Busbey commented on HBASE-18540: - /cc [~mdrob], because I remember him talking about this kind of issue before so maybe I've just filed a dup. > Ad-hoc job for checking if a test is flaky > -- > > Key: HBASE-18540 > URL: https://issues.apache.org/jira/browse/HBASE-18540 > Project: HBase > Issue Type: New Feature > Components: test >Reporter: Sean Busbey > > It would be great if we had an ad-hoc job that could run a given test a bunch > of times and tell me if it is flaky. > I'm trying to evaluate the state of our new nightly jobs, and I'm not sure if > hte failures I'm seeing are a) things I need to get fixed soon, b) things > that are flaky but previously unrecognized, c) things that are flaky but > missed because our current flaky list is only based on the behavior of the > master branch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
