Re: [DISCUSS] Automatic rerunning of failed tests in Hive Pre-commit
Hive has >24hours of tests - in case of automated reruns... I wonder how a patch which breaks almost all tests should be handled? I believe we already have a process to step up against these things: if you encounter a flaky test - it should be checked by using: http://ci.hive.apache.org/job/hive-flaky-check/ ..and/or fix the underlying issue... we had a discussion about it on the mailing list a long time ago. I see that quite a few flaky tests have creeped in... http://ci.hive.apache.org/job/hive-precommit/job/PR-4372/ most of these should be fixed...or cleared off the radar cheers, Zoltan OpenPGP_signature Description: OpenPGP digital signature
Re: [DISCUSS] Automatic rerunning of failed tests in Hive Pre-commit
I’ll second the above. We already have a number of flaky tests that would go undetected and just increase the build time.
Re: [DISCUSS] Automatic rerunning of failed tests in Hive Pre-commit
Hello, I tend to agree with Sai; if we can run on demand the failed tests that would be a cool feature. If we just rerun everything that fails without questions asked we may do more harm than good. Taking into account the fact that nobody runs all tests locally before submitting a PR (cause it is not feasible) there is a very high chance that things go really bad in CI and rerunning will just keep wasting resources. Best, Stamatis On Thu, Jun 8, 2023 at 7:56 PM Sai Hemanth Gantasala wrote: > > Hello everyone, > > My personal preference is that the option to rerun *only* the failed test > suites should be manual. The reason is, there might be frequent intentional > test failures (failures due to our own patch) and the tests being flaky is > less likely. So giving the option to the user to rerun the failed test > suites wastes fewer resources if the failures are not flaky. > If we were to automatically retry running failed test suites, then I prefer > the value of "rerunFailingTestsCount" to be set to 1, since there is very > less probability that the tests will be flaky in the consecutive runs. > > Thanks, > Sai. > > On Thu, Jun 8, 2023 at 12:07 AM Ayush Saxena wrote: > > > +1 from me as well to rerun the failing tests. > > The oracle docker is also a pain, it is one of the main reasons for > > retriggers, these retriggers wastes a lot of resources and increases the > > time to get build results for genuine runs. > > > > -Ayush > > > > On Thu, 8 Jun 2023 at 12:31, Butao Zhang wrote: > > > > > +1. I often have to rerun whole pre-commit job due to individual unstable > > > test, ant it is too time-consuming. It would be much better if we can set > > > maven config to retry automatically. > > > > > > > > > > > > Thanks, > > > > > > Butao Zhang > > > > > > Replied Message > > > | From | r12 t45 | > > > | Date | 6/8/2023 14:52 | > > > | To | | > > > | Subject | [DISCUSS] Automatic rerunning of failed tests in Hive > > > Pre-commit | > > > Hi All, > > > > > > It often happens that Hive unit tests fail during pre-commit which > > requires > > > rerunning the whole pre-commit job and creates hours of delays. > > > What if we set Maven config to retry failed tests automatically X times? > > > There is "rerunFailingTestsCount" property in maven-surefire-plugin which > > > can be used for that. > > > I would like to hear the feedback and if it is positive I could open a > > JIRA > > > ticket and work on it. > > > > > > Thanks, > > > Dmitriy > > > > >
Re: [DISCUSS] Automatic rerunning of failed tests in Hive Pre-commit
Hello everyone, My personal preference is that the option to rerun *only* the failed test suites should be manual. The reason is, there might be frequent intentional test failures (failures due to our own patch) and the tests being flaky is less likely. So giving the option to the user to rerun the failed test suites wastes fewer resources if the failures are not flaky. If we were to automatically retry running failed test suites, then I prefer the value of "rerunFailingTestsCount" to be set to 1, since there is very less probability that the tests will be flaky in the consecutive runs. Thanks, Sai. On Thu, Jun 8, 2023 at 12:07 AM Ayush Saxena wrote: > +1 from me as well to rerun the failing tests. > The oracle docker is also a pain, it is one of the main reasons for > retriggers, these retriggers wastes a lot of resources and increases the > time to get build results for genuine runs. > > -Ayush > > On Thu, 8 Jun 2023 at 12:31, Butao Zhang wrote: > > > +1. I often have to rerun whole pre-commit job due to individual unstable > > test, ant it is too time-consuming. It would be much better if we can set > > maven config to retry automatically. > > > > > > > > Thanks, > > > > Butao Zhang > > > > Replied Message > > | From | r12 t45 | > > | Date | 6/8/2023 14:52 | > > | To | | > > | Subject | [DISCUSS] Automatic rerunning of failed tests in Hive > > Pre-commit | > > Hi All, > > > > It often happens that Hive unit tests fail during pre-commit which > requires > > rerunning the whole pre-commit job and creates hours of delays. > > What if we set Maven config to retry failed tests automatically X times? > > There is "rerunFailingTestsCount" property in maven-surefire-plugin which > > can be used for that. > > I would like to hear the feedback and if it is positive I could open a > JIRA > > ticket and work on it. > > > > Thanks, > > Dmitriy > > >
Re: [DISCUSS] Automatic rerunning of failed tests in Hive Pre-commit
+1 from me as well to rerun the failing tests. The oracle docker is also a pain, it is one of the main reasons for retriggers, these retriggers wastes a lot of resources and increases the time to get build results for genuine runs. -Ayush On Thu, 8 Jun 2023 at 12:31, Butao Zhang wrote: > +1. I often have to rerun whole pre-commit job due to individual unstable > test, ant it is too time-consuming. It would be much better if we can set > maven config to retry automatically. > > > > Thanks, > > Butao Zhang > > Replied Message > | From | r12 t45 | > | Date | 6/8/2023 14:52 | > | To | | > | Subject | [DISCUSS] Automatic rerunning of failed tests in Hive > Pre-commit | > Hi All, > > It often happens that Hive unit tests fail during pre-commit which requires > rerunning the whole pre-commit job and creates hours of delays. > What if we set Maven config to retry failed tests automatically X times? > There is "rerunFailingTestsCount" property in maven-surefire-plugin which > can be used for that. > I would like to hear the feedback and if it is positive I could open a JIRA > ticket and work on it. > > Thanks, > Dmitriy >
Re: [DISCUSS] Automatic rerunning of failed tests in Hive Pre-commit
+1. I often have to rerun whole pre-commit job due to individual unstable test, ant it is too time-consuming. It would be much better if we can set maven config to retry automatically. Thanks, Butao Zhang Replied Message | From | r12 t45 | | Date | 6/8/2023 14:52 | | To | | | Subject | [DISCUSS] Automatic rerunning of failed tests in Hive Pre-commit | Hi All, It often happens that Hive unit tests fail during pre-commit which requires rerunning the whole pre-commit job and creates hours of delays. What if we set Maven config to retry failed tests automatically X times? There is "rerunFailingTestsCount" property in maven-surefire-plugin which can be used for that. I would like to hear the feedback and if it is positive I could open a JIRA ticket and work on it. Thanks, Dmitriy
[DISCUSS] Automatic rerunning of failed tests in Hive Pre-commit
Hi All, It often happens that Hive unit tests fail during pre-commit which requires rerunning the whole pre-commit job and creates hours of delays. What if we set Maven config to retry failed tests automatically X times? There is "rerunFailingTestsCount" property in maven-surefire-plugin which can be used for that. I would like to hear the feedback and if it is positive I could open a JIRA ticket and work on it. Thanks, Dmitriy
