On 03/29/2017 09:47 AM, Brady Allen Johnson wrote: > > Ursicio, > > Awesome work coming up with a solution so quick. I think this will have to go > in for Boron SR4, and just document the > problems with SR3.
we don't have branches or branch locking in Int/Test, so no worries about putting test patches at any time. JamO > Thanks, > > Brady > > -----Original Message----- > *From*: Ursicio Javier Martin <[email protected] > <mailto:ursicio%20javier%20martin%20%[email protected]%3e>> > *To*: Diego Jesus Granados Lopez <[email protected] > <mailto:diego%20jesus%20granados%20lopez%20%[email protected]%3e>>, > Jamo Luhrsen <[email protected] > <mailto:jamo%20luhrsen%20%[email protected]%3e>>, Colin Dixon > <[email protected] > <mailto:colin%20dixon%20%[email protected]%3e>>, > [email protected] <[email protected] > <mailto:%[email protected]%22%20%[email protected]%3e>> > *Subject*: Re: [sfc-dev] SFC Boron-SR3 test failure checkoff > *Date*: Wed, 29 Mar 2017 15:33:35 +0000 > > Hi Jamo, > > Here you have the proposal in integration/test to overcome this Boron issue: > > https://git.opendaylight.org/gerrit/#/c/54034/ > > BR, > > Ursicio > > > -----Original Message----- > From: Diego Jesus Granados Lopez > Sent: miércoles, 29 de marzo de 2017 15:13 > To: Jamo Luhrsen <[email protected] <mailto:[email protected]>>; Colin > Dixon <[email protected] <mailto:[email protected]>>; Ursicio Javier > Martin <[email protected] > <mailto:[email protected]>>; [email protected] > <mailto:[email protected]> > Subject: RE: [sfc-dev] SFC Boron-SR3 test failure checkoff > > Jamo, > > Thanks for the explanation. Still, this lack of branching in integration/test > repo is something very hard to understand to me, especially when that fact is > translating into the need of adding branching intelligence into the > testcases. When branches are needed, it is version control where they belong. > It's not only that: in my experience, when you require from testcases to be > valid for different software tracks, at the end you're required that the > testcases are verified on all those tracks, a requirement that is really hard > on individuals contributing new tests, because of how hard it is to change > tracks during development. As I said, I experienced a similar situation in my > organization, and this level of demand translated into the dysfunction of > people actively avoiding to contribute tests (because of not having the time > to verify them in all tracks). > > Also in the spirit of trying to improve things, , it was me who contributed > four of those testcases which are failing in Boron (they support a correction > which is meant only for Carbon), having no clue that they would be run vs all > active tracks. It would be really nice if this "not trivial to anticipate" > behavior was easier to stumble upon in documentation (at least I couldn't > find anything about it; can totally be my fault), or at least if this > expectation was an explicit comment in code reviews for people starting to > contribute tests as I am myself, at least when that branching-support logic > is clearly not present in the new/modified tests. > > I'm sure there is a background that lead to this decision of not branching > the integration/test repo, and I'd really appreciate if you could point me to > that info so I can understand if there was a reasoning that I'm not getting > > Back to the original point, it is true that we have eight testcases that we > didn't notice were failing in Boron. As my colleague Ursicio said, we'll be > working to fix them, but in the meanwhile, is it ok that we simply > acknowledge the failure as not blocking in the spreadsheet Colin shared? > (I'll be doing it right away) > > Best regards, > Diego > > -----Original Message----- > From: Jamo Luhrsen [mailto:[email protected]] > Sent: martes, 28 de marzo de 2017 23:24 > To: Colin Dixon <[email protected] <mailto:[email protected]>>; Diego > Jesus Granados Lopez <[email protected] > <mailto:[email protected]>>; Ursicio Javier Martin > <[email protected] > <mailto:[email protected]>>; [email protected] > <mailto:[email protected]> > Subject: Re: [sfc-dev] SFC Boron-SR3 test failure checkoff > > first thing is we should be noticing these things well before release people > are noticing them as we try to get an SR out the door. I have a hunch this > started happening before > 2017 started. > > secondly, there is some confusion here: > > our test code comes from the integration/test repo and we only have a master > branch. There is no such thing as a boron test branch. The git snippets you > gave are from integration/test so it makes sense that you only see master. > > our jobs have the branch of the *controller* in their name. So, for the job > we are looking at here (sfc-csit-3node-rest-basic-all-boron), it is > downloading a distribution .zip made from the boron branch. It's not building > it, just downloading from nexus. > > SFC csit has dealt with this in the past where CSIT needs to be smart about > knowing which ODL branch it's testing against. Other projects deal with this > as well. It looks like SFC is now only assuming master in it's suites: > > https://github.com/opendaylight/integration-test/blob/master/csit/suites/sfc/SFC_Basic/010__sfc_service_functions.robot#L116-L118 > > here's a patch for sfc csit that came in a few months ago playing in that > same area. > > https://git.opendaylight.org/gerrit/#/c/45819 > > integration-dev can help with reviews of patches that come in to get this > cleaned up. > > hope it helps, > JamO > > > On 03/28/2017 06:27 AM, Colin Dixon wrote: >> Thanks for looking into this. We've heard other people say the same things >> too. --Colin On Tue, Mar 28, 2017 at 8:57 AM >> Diego Jesus Granados Lopez <[email protected] >> <mailto:[email protected]> >> <mailto:[email protected]>> wrote: I'm adding more >> evidence. From one of the failed tasks [1]: >> GIT_URL ssh://[email protected] >> <mailto:[email protected]>:29418/integration/test >> <http://[email protected]:29418/integration/test> >> GIT_BRANCH origin/master GIT_COMMIT >> 9f3c006279ca73a063f923e64a0c74aebf809a9d At execution logs: 04:52:41 > git >> fetch --tags --progress >> ssh://[email protected] >> <mailto:[email protected]>:29418/integration/test >> <http://[email protected]:29418/integration/test> master >> 04:52:41 > git rev-parse FETCH_HEAD^{commit} # >> timeout=10 04:52:41 Checking out Revision >> 9f3c006279ca73a063f923e64a0c74aebf809a9d (origin/master) 04:52:41 > git >> config >> core.sparsecheckout # timeout=10 04:52:41 > git checkout -f >> 9f3c006279ca73a063f923e64a0c74aebf809a9d That commit being used >> (9f3c) is the carbon tip (not in boron). So the wrong version of >> integration/test is being used for running the test. I'm >> not sure of which project is in charge of that script / task configuration >> in order to forward this info to them. Is it >> releng/builder? [1] >> https://jenkins.opendaylight.org/releng/job/sfc-csit-3node-rest-basic- >> all-boron/334 -----Original >> Message----- From: Ursicio Javier Martin Sent: martes, 28 de marzo de 2017 >> 12:55 To: Diego Jesus Granados Lopez >> <[email protected] >> <mailto:[email protected]> >> <mailto:[email protected]>>; Jamo Luhrsen >> <[email protected] <mailto:[email protected]> >> <mailto:[email protected]>>; [email protected] >> <mailto:[email protected]> >> <mailto:[email protected]> Cc: Colin Dixon >> <[email protected] <mailto:[email protected]> >> <mailto:[email protected]>> Subject: RE: [sfc-dev] SFC Boron-SR3 test >> failure checkoff Hi Jamo, After looking at the >> execution logs I agree with Diego, for some reason tests devoted to Carbon >> are being executed in Boron scope. Maybe a >> Releng issue ... BR, Ursicio -----Original Message----- From: >> [email protected] >> <mailto:[email protected]> >> <mailto:[email protected]> >> [mailto:[email protected] >> <mailto:[email protected]>] On Behalf Of Diego Jesus >> Granados Lopez Sent: martes, 28 de marzo de 2017 10:29 To: Jamo Luhrsen >> <[email protected] <mailto:[email protected]> >> <mailto:[email protected]>>; [email protected] >> <mailto:[email protected]> >> <mailto:[email protected]> Cc: Colin Dixon >> <[email protected] <mailto:[email protected]> >> <mailto:[email protected]>> Subject: Re: [sfc-dev] SFC Boron-SR3 test >> failure checkoff Hi, I checked the failing job and >> most (not sure if all, but probably) of the failing tests shouldn't be >> executing in boron checks. E.g. there are 4 tests >> consistently failing that shouldn't be executing in Boron (Service Path >> validation: added by me) they were merged to carbon >> only one month ago. Any chance the Jenkins job is misconfigured and using >> master branch instead of boron-something for >> running that jobs? I looked at the job config and it contains a >> "GERRIT_BRANCH" parameter set to "stable/boron", but as I >> said if it was being used, tests added to carbon only shouldn't be being run >> BR, Diego -----Original Message----- From: >> [email protected] >> <mailto:[email protected]> >> <mailto:[email protected]> >> [mailto:[email protected] >> <mailto:[email protected]>] On Behalf Of Jamo Luhrsen >> Sent: lunes, 27 de marzo de 2017 22:57 To: >> [email protected] <mailto:[email protected]> >> <mailto:[email protected]> Cc: Colin >> Dixon <[email protected] <mailto:[email protected]> >> <mailto:[email protected]>> Subject: Re: [sfc-dev] SFC >> Boron-SR3 test failure checkoff SFC, I took a look at one of your failing >> jobs from the tracking sheet and it does have >> seemingly consistent failures now. But, the same job back in the SR2 >> timeframe was passing 100%. Maybe there is a >> regression? job from SR2-ish: >> https://logs.opendaylight.org/releng/jenkins092/sfc-csit-3node-rest-ba >> sic-all-boron/212/archives/log.html.gz Thanks, JamO > tl;dr > Your project >> has test failures in the Boron-SR3 distribution >> tests. > Please look at them and mark them as OKAY in the spreadsheet if > >> they're not blocking or start a conversation >> with us _now_ if they are. > > > Full details: > The SFC project has at >> least one test failure in the distribution test > >> from the proposed Boron-SR3 release here: > >> https://wiki.opendaylight.org/view/Simultaneous_Release:Boron_Release_ > >> Plan#Boron_SR3_Download > > To find the test failures, go to the >> distribution test report link at > the page above or >> directly here: > >> https://jenkins.opendaylight.org/releng/view/distribution/job/integrat > >> ion-distribution-test-boron/173/ >> > > Then look for yellow or red balls after tests that start with your > >> > > project's short name and click on the number to go >> to the actual run > that produced the failures. > > Please check to see if >> the failures are a blocking issue for releasing >> > Boron-SR3 and if so, start the discussion and if not, update the > >> > spreadsheet here with OK in front of the issues: > >> https://docs.google.com/spreadsheets/d/1zImtd764e-hOgJAxoJKl85fxHCPu2a > >> gLfqsBtf13zQY/edit#gid=1776065453 > > Ideally, >> also follow up here or on this thread: > >> https://lists.opendaylight.org/pipermail/release/2017-March/009712.htm > l > >> > >> Cheers, > --Colin _______________________________________________ sfc-dev >> mailing list [email protected] >> <mailto:[email protected]> >> <mailto:[email protected]> >> https://lists.opendaylight.org/mailman/listinfo/sfc-dev >> _______________________________________________ sfc-dev mailing >> list [email protected] <mailto:[email protected]> >> <mailto:[email protected]> >> https://lists.opendaylight.org/mailman/listinfo/sfc-dev > > _______________________________________________ > sfc-dev mailing list > [email protected] <mailto:[email protected]> > https://lists.opendaylight.org/mailman/listinfo/sfc-dev > _______________________________________________ sfc-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/sfc-dev
