Hi Eric, Standard procedure is to modify the EXPECTEDaaa file. If you are satisfied with your new results you could simply copy the LOGaaa file as the new EXPECTEDaaa file. There are filters in the test suite that will hide things like timestamp from the final comparison. DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass while a fix is being worked on for a particular failure.
Thanks Suresh On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <[email protected]> wrote: > Thanks Sandhya, very useful! > BTW, if I want to change a test to take into account valid changes that > translate into false failure, should I modify the comparison file, or the > file that logs known legitimate differences? > Thanks again for the help :-), > Eric > > -----Original Message----- > From: Sandhya Sundaresan [mailto:[email protected]] > Sent: Tuesday, August 4, 2015 1:19 PM > To: [email protected] > Subject: RE: Where are the logs for the first set of regression tests? > > RE: Where are the logs for the first set of regression tests? > > There are logs and diffs. I notice I added the instructions on the wiki > only > for the individual "pull request" automated jobs. I will add them for the > official test results as well. > > Here they are : > > Click on the link in the daily build email : > > *https://jenkins02.trafodion.org/job/Check-Daily/48/* > <https://jenkins02.trafodion.org/job/Check-Daily/48/> > > ==> Click on "Console Output" for any test suite you want to check > > ==> Around the 5th or 6th line you will see a link to detailed logs . Click > that. > > > > 2015-08-04 08:30:13 Started by upstream project "Check-Daily" build > number 48 > > 2015-08-04 08:30:13 originally caused by: > > 2015-08-04 08:30:13 Started by timer > > 2015-08-04 08:30:13 [EnvInject] - Loading node environment variables. > > 2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75 > (redhat_ahw2.2) in workspace > /home/jenkins/workspace/core-regress-fullstack2-ahw2.2 > > 2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh > /tmp/hudson2056470402644332566.sh > > 2015-08-04 08:30:14 Detailed logs: > * > http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/* > < > http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/ > > > > == > Navigate to sql-regress-log > > You will see a directory for the component test suite – eg > “executor” and a full log – “Regress.log”. > > If you navigate to the component directory eg executor, you > will see all the DIFFs and LOG files . > > Sandhya > > -----Original Message----- > From: Eric Owhadi [mailto:[email protected] <[email protected]>] > Sent: Tuesday, August 4, 2015 11:06 AM > To: [email protected] > Subject: RE: Where are the logs for the first set of regression tests? > > Hi Sandhya, > > Thanks for the offer, I'll first do a re-run with the feature off and get > familiar with the official results, that'll give me some learning > opportunity :-). > > There is no "diff" file on the official results automatic daily run? So > that > it is easy to compare with the failure one get running the test manually? > > Eric > > -----Original Message----- > > From: Sandhya Sundaresan [mailto:[email protected] > <[email protected]>] > > Sent: Tuesday, August 4, 2015 12:50 PM > > To: [email protected] > > Subject: RE: Where are the logs for the first set of regression tests? > > Hi Eric, > > They are run on a nightly basis. Attached are the results form last night > and previous night. As you can see 2 failures last night and both are being > investigated. Otherwise we have been having good clean runs. So any other > failures you see need to be investigated. If you can give me VNC access to > your machine, I can take a quick look. > > I have documented how to read the official results on the wiki : > > > https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checking_Test_Results > > Sandhya > > -----Original Message----- > > From: Eric Owhadi [mailto:[email protected] <[email protected]>] > > Sent: Tuesday, August 4, 2015 10:40 AM > > To: [email protected] > > Subject: RE: Where are the logs for the first set of regression tests? > > Out of curiosity, anyone have recently run the regression test all on > master > branch? I am getting some failure, and I wonder if they are linked with my > changes or if the current regression suit is having known temporary issues? > > Eric > > -----Original Message----- > > From: Dave Birdsall [mailto:[email protected] > <[email protected]>] > > Sent: Tuesday, August 4, 2015 11:26 AM > > To: [email protected] > > Subject: RE: Where are the logs for the first set of regression tests? > > Right. Start with the simple stuff I think. Eric's e-mail trail is valuable > as it is fresh newbie experience; if we could answer Eric's questions on > the > wiki I think that would be a great start. > > -----Original Message----- > > From: Sandhya Sundaresan [mailto:[email protected] > <[email protected]>] > > Sent: Tuesday, August 4, 2015 9:24 AM > > To: [email protected] > > Subject: RE: Where are the logs for the first set of regression tests? > > Hi Eric, Dave, > > I can add a page to the wiki for this. We did have the steps documented > in our old internal wiki for those starting on Trafodion for the first > time. > > Also Roberta had a comprehensive document describing the whole regress > environment back in MX days , I think. Not sure if anyone has that saved. > > Either way, for the wiki, perhaps a smaller page would be sufficient. > > Thanks > > Sandhya > > -----Original Message----- > > From: Eric Owhadi [mailto:[email protected] <[email protected]>] > > Sent: Tuesday, August 4, 2015 9:03 AM > > To: [email protected] > > Subject: RE: Where are the logs for the first set of regression tests? > > I didn't want to ask, but indeed that would be fantastic and time saving. > > Also how to write and/or modify when tests are complaining, but we know > that > this is just because regression is based on delta, and cannot recognize > legitimate expected delta (due to know feature we just implemented), and > real regression. > > Eric > > -----Original Message----- > > From: Dave Birdsall [mailto:[email protected] > <[email protected]>] > > Sent: Tuesday, August 4, 2015 10:21 AM > > To: [email protected] > > Subject: RE: Where are the logs for the first set of regression tests? > > Do we have a page on our Apache Trafodion wiki that describes how to run > the > regression tests and how to interpret and deal with the results? If someone > would like to volunteer to write such a page, it would be very useful to > newbies, I think. > > -----Original Message----- > > From: Eric Owhadi [mailto:[email protected] <[email protected]>] > > Sent: Tuesday, August 4, 2015 7:45 AM > > To: [email protected] > > Subject: RE: Where are the logs for the first set of regression tests? > > Thanks Suresh, that's it indeed :-). Was driving me crazy. > > Eric > > -----Original Message----- > > From: Suresh Subbiah [mailto:[email protected] > <[email protected]>] > > Sent: Tuesday, August 4, 2015 9:41 AM > > To: [email protected] > > Subject: Re: Where are the logs for the first set of regression tests? > > Hi Eric, > > I think the "first" set of tests is the same as the last set. This is an > artifact of how runallsb prints out results. At the end of every suite a > summary of that suite is printed out (with no header what suite it refers > to > since we have been running that suite for the last X minutes). Then at the > end of a run it concatenates the summary of every suite, this time with a > header for each suite. So if you kick off regressions now and come back > several hours later there will be a screen with a summary for all suites > and > then before the summary for core there will be list of results with no > suite > name. This is the same as the last suite in the summary (the hive test > suite). > > Thanks > > Suresh > > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <[email protected]> wrote: > > > Hi Eric, > > > > > > The results should be in one of the directories under $rundir. For > > > example, under $rundir/core, the result for TEST001 (under > > > $scriptsdir/core) is LOG001. > > > > > > In my ~/.bashrc file, I have these two variables defined. > > > > > > export rundir=/mnt/qfc/rundir > > > > > > export scriptsdir=$trafodionRoot/sql/regress > > > > > > > > > After mkdir $rundir, make sure $rundir contains directories, one for > > > each category in $scriptsdir. Once it is setup, you can run > > > regressions as many times as you want :-) > > > > > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <[email protected]> > wrote: > > > > > > > That may be an obvious question, but I can’t find the logs and diffs > > > > for the first set of test showing up in the result of the runallsb: > > > > > > > > I can see all the logs and diffs for categories like core, > > > > compGeneral > > > etc, > > > > but the first set of test labelled: > > > > > > > > TEST001 > > > > > > > > TEST002 > > > > > > > > TEST003 > > > > > > > > TEST004 > > > > > > > > TEST005 > > > > > > > > TEST006 > > > > > > > > TEST015 > > > > > > > > TEST017 > > > > > > > > TEST018 > > > > > > > > TEST020 > > > > > > > > > > > > > > > > are not associated with a category, and they don’t have a > > > > subdirectory > > > with > > > > categoryname where I can find the other test results. > > > > > > > > Thanks in advance for the help. > > > > > > > > > > > > > > > > As a side note, if a test fails because of the compare, but the > > > > delta is legit, what should one do? Update known diffs? Or set a new > > > > compare file? > > > > > > > > > > > > > > > > Eric > > > > > > > > > > > > > > > > -- > > > Regards, --Qifan > > > >
