Here is a tally of run-time (in minutes) of one run of the hive regression.
TEST1 1 TEST2 1 TEST3 3 TEST5 7 TEST6 1 TEST9 4 TEST15 5 TEST17 5 TEST18 6 TEST21 1 TEST30 1 TEST31 1 TEST33 4 TEST34 2 TEST35 3 TEST36 1 The total time used is about 46 minutes. My vote will be to include a subset of from the above list what are fast to run (say all 1 minute ones). It probably will be a good idea to keep them still in hive tests. Thanks --Qifan On Tue, Jul 26, 2016 at 5:37 PM, Selva Govindarajan < [email protected]> wrote: > Thanks Steve for resurrecting this discussion. > > Hive tests have been stabilized to a greater extent that we shouldn't have > false failures now. Recently, there has been a quite amount of contribution > coming in the area related to hive in Trafodion. Hence I would vote +1 for > adding hive tests to check PR. > > Selva > > -----Original Message----- > From: Steve Varnau [mailto:[email protected]] > Sent: Tuesday, July 26, 2016 1:12 PM > To: [email protected] > Subject: RE: Proposal to add hive regression tests to check-PR tests > > Hi, > > I wanted to revisit this discussion to come to resolution. There was a > digression into the idea of dynamically choosing tests, but I'd like to > come > back to original proposal of adding an extra suite to the check tests. > > As I read the thread, there were several responses in support of the > proposal, and a couple of reservations. The reservations include increasing > the chance for false failures, which already can be a headache. Also the > concern of adding long running tests that are included in hive versus maybe > adding a few more small tests to core. Or perhaps using "extra tests" as > needed, which is available on request. > > I'm willing to add another test job if that is what the community wants, > but > might it make more sense to more small tests to core or move some from hive > to core? > > --Steve > > > -----Original Message----- > > From: Selva Govindarajan [mailto:[email protected]] > > Sent: Monday, July 18, 2016 10:28 PM > > To: [email protected] > > Subject: RE: Proposal to add hive regression tests to check-PR tests > > > > I totally agree with Steve to use a simple and predictable mechanism > > to do check PR tests, If my memory serves me right, prior to Trafodion > > becoming an Apache incubating project, hive tests were part of > > check-PR. Because of unpredictable state of hive regressions then it > > was decided to suspend > > running hive regressions as part of check-in. Based on the current > state > > of Trafodion, and the fact that the hive regressions have been > > stabilized to a greater extent, it is important that this stability is > > maintained by the future contributions. Recently many contributions > > have come in hive-related area of the code. > > > > Adding hive regressions as part of check-PR should not increase the > > overall time to complete the check-PR, but it would require additional > > resources,. > > Hence, Trafodion Jenkins infrastructure would incur additional cost. > > > > I am expecting the Trafodion Release Manager of R2.1 will help us to > > determine with the community input what is the best option to go with. > > > > Selva > > -----Original Message----- > > From: Steve Varnau [mailto:[email protected]] > > Sent: Monday, July 18, 2016 12:27 PM > > To: [email protected] > > Subject: RE: Proposal to add hive regression tests to check-PR tests > > > > The current test process looks at which files have been modified and > > puts it into a bucket, which is used to determine what tests to run. > > However, the only buckets that now exist are DOC and NONDOC. > > > > So if the change consists only of things in the docs/ tree, then it > > only does static check and a docs build. If there are non-docs > > changes, it assumes it needs to run all the build and tests. > > > > It is pretty conservative, but the more heuristics we put in to > > customize the tests, the more chance that it will miss something. I > > can imagine a connectivity only change not running the jobs that don't > > exercise connectivity. But figuring out what things might affect hive > > tests seems much harder. > > > > There are many things (installer, executor,...) that might affect any > > of our tests. Seems safer to keep the test heuristics very simple and > > predictable, and change the content of the test suites to what ought > > to be in check versus nightly. > > > > --Steve > > > > > -----Original Message----- > > > From: Qifan Chen [mailto:[email protected]] > > > Sent: Monday, July 18, 2016 9:43 AM > > > To: dev <[email protected]> > > > Subject: Re: Proposal to add hive regression tests to check-PR tests > > > > > > The author just honestly describes the changes, and the tool picks > > > the right tests. Thanks --Qifan > > > > > > On Mon, Jul 18, 2016 at 11:29 AM, Sean Broeder > > > <[email protected]> > > > wrote: > > > > > > > I'd prefer not to leave it up to authors to select which tests are > > > > appropriate. Sometimes we get it right and others we are horribly > > > > wrong. > > > > > > > > Thanks, > > > > Sean > > > > > > > > -----Original Message----- > > > > From: Qifan Chen [mailto:[email protected]] > > > > Sent: Monday, July 18, 2016 9:20 AM > > > > To: dev <[email protected]> > > > > Subject: Re: Proposal to add hive regression tests to check-PR > > > > tests > > > > > > > > I agree with Sandhya and wonder if we can enhance check-PR tests > > > > (hive > > > for > > > > example, in question) with the following twist. > > > > > > > > 1. Randomly select several (say 3) tests from regression/hive. The > > > > rational is that we only need to sanity check the changes and > > > > a full daily > > > > build with test will follow the merge. > > > > 2. Before the check-in, we always run the full regression test, > > > > and I do > > > > not see the value to run full Hive again in check-PR. > > > > 3. In the future, we could find the most appreciate tests for > > > > check-PR > > > > (instead of randomly select, or select the full set). The > > > > author can point > > > > out the nature of the change and the check-in tool does the > > > > selection. > > > > For > > > > example, a change in DoP for Hbase tables will select some > > > > tests from > > > > regress/seabase, but not from regress/hive. > > > > > > > > Thanks > > > > > > > > --Qifan > > > > > > > > On Mon, Jul 18, 2016 at 10:46 AM, Sandhya Sundaresan < > > > > [email protected]> wrote: > > > > > > > > > +0 for me. > > > > > I am not sure of the need to add the whole test suite to check > > > > > tests. > > > > > The hive regressions do run nightly anyway so failures should be > > > > > clear on each nightly run on a daily basis. > > > > > My concern is that long running tests like hive/TEST018 are more > > > > > to > > > > test > > > > > features like bulkload/unload and since we already have the > > > > > option to run "extra tests" in Jenkins, I'm not sure bringing in > > > > > entire test suites into check tests is the right approach or > > > > > trend going forward and adding time and resources to what is > > > > > supposed to be a sanity test > > > > for > > > > > every single PR. > > > > > > > > > > Sandhya > > > > > > > > > > -----Original Message----- > > > > > From: Selva Govindarajan [mailto:[email protected]] > > > > > Sent: Monday, July 18, 2016 7:22 AM > > > > > To: [email protected] > > > > > Subject: RE: Proposal to add hive regression tests to check-PR > > > > > tests > > > > > > > > > > Hive regressions takes little less than an hour. As I said > > > > > before, the time is not a factor because the regressions are run > > > > > in parallel in different VMs. Seabase regressions which is run > > > > > as part of check-PR takes around 1 hour and 40 mins. Hence hive > > > > > regressions shouldn't add more time for check-PR to complete, > > > > > but of course it would need another VM. > > > > > > > > > > Selva > > > > > > > > > > -----Original Message----- > > > > > From: Jin, Jian (Seth) [mailto:[email protected]] > > > > > Sent: Sunday, July 17, 2016 7:31 PM > > > > > To: [email protected] > > > > > Subject: RE: Proposal to add hive regression tests to check-PR > > > > > tests > > > > > > > > > > How long will it take for Hive regression? > > > > > > > > > > Br, > > > > > > > > > > Seth > > > > > > > > > > -----Original Message----- > > > > > From: Liu, Ming (Ming) [mailto:[email protected]] > > > > > Sent: 2016年7月16日 9:16 > > > > > To: [email protected] > > > > > Subject: RE: Proposal to add hive regression tests to check-PR > > > > > tests > > > > > > > > > > +1 to this > > > > > > > > > > -----Original Message----- > > > > > From: Selva Govindarajan [mailto:[email protected]] > > > > > Sent: Saturday, July 16, 2016 9:08 AM > > > > > To: [email protected] > > > > > Subject: Proposal to add hive regression tests to check-PR tests > > > > > > > > > > If you have subscribed to Trafodion Daily Build, you would have > > > > > noticed that the daily build has been failing for some days. > > > > > Most often, it is due the failure in hive regression tests run > > > > > as part of the daily build. > > > > > Lately, there has been some conscious effort made successfully > > > > > to ensure that the hive regression tests can be run reliably. To > > > > > maintain the Trafodion daily build in that state, I am proposing > > > > > to include hive regressions to check-PR tests. It shouldn’t add > > > > > the overall time taken to regressions tests because tests are > > > > > run in parallel on different VMs, though it would consume more > > > > > resources. > > > > > > > > > > > > > > > > > > > > - Selva > > > > > > > > > > > > > > > > > > > > > -- > > > > Regards, --Qifan > > > > > > > > > > > > > > > > -- > > > Regards, --Qifan > -- Regards, --Qifan
