On 03/29/2017 02:58 AM, Ursicio Javier Martin wrote: > Hi Jamo, > > Thanks for the explanation. > > Then we could say the "branch" mechanism is implemented by the test cases > themselves in one way or another which sounds a > bit strange for us; but I'm sure you have already discussed about this and > conclude that this is the best way.
Yeah, this was a big conversation a long time ago (summer of 2015). I can probably dig up emails or meeting notes, but the tl;dr is that Int/Test (and Int/Packaging) is not part of the simultaneous release and we don't want/need any of the overhead that goes with it. Some of that overhead was maintaining/locking branches, etc. That coupled with the fact that our master branch was always nearly an identical copy of the older branches made it easy to decide we only needed one branch. I know SFC has encountered this (needing different test code (maybe changing APIs or data models to do the same functionality?)) several times now, and it's not alone, it is the exception in ODL so far. so it's a trade off right now. I assume if lots of projects have this pain, we would decide to go back to the branching method. > For this particular SFC matter we are planning to move the test cases > specific of carbon to another directory and > reestablish the "boron" version of the failed test cases. After this change > test cases specific of carbon will be executed > just in a Jenkins job connected to carbon and not to boron. I saw the patch, and I'm not following. Is that patch a revert to get back to the old test code that works with Boron? if so, I only see three patches in csit/suites/sfc that came in since things were 100% passing. can we just revert those instead? The new patch you gave is large and still references master as the ODL distro it's working with. these are the three patches I'm referring to: https://git.opendaylight.org/gerrit/#/c/48813/ https://git.opendaylight.org/gerrit/#/c/48899/ https://git.opendaylight.org/gerrit/#/c/50843/ Thanks, JamO > BR, > > Ursicio > > > -----Original Message----- From: Jamo Luhrsen [mailto:[email protected]] > Sent: martes, 28 de marzo de 2017 23:24 To: > Colin Dixon <[email protected]>; Diego Jesus Granados Lopez > <[email protected]>; Ursicio Javier > Martin <[email protected]>; [email protected] > Subject: Re: [sfc-dev] SFC Boron-SR3 test > failure checkoff > > first thing is we should be noticing these things well before release people > are noticing them as we try to get an SR out > the door. I have a hunch this started happening before 2017 started. > > secondly, there is some confusion here: > > our test code comes from the integration/test repo and we only have a master > branch. There is no such thing as a boron > test branch. The git snippets you gave are from integration/test so it makes > sense that you only see master. > > our jobs have the branch of the *controller* in their name. So, for the job > we are looking at here > (sfc-csit-3node-rest-basic-all-boron), it is downloading a distribution .zip > made from the boron branch. It's not building > it, just downloading from nexus. > > SFC csit has dealt with this in the past where CSIT needs to be smart about > knowing which ODL branch it's testing against. > Other projects deal with this as well. It looks like SFC is now only assuming > master in it's suites: > > https://github.com/opendaylight/integration-test/blob/master/csit/suites/sfc/SFC_Basic/010__sfc_service_functions.robot#L116-L118 > > here's a patch for sfc csit that came in a few months ago playing in that > same area. > > https://git.opendaylight.org/gerrit/#/c/45819 > > integration-dev can help with reviews of patches that come in to get this > cleaned up. > > hope it helps, JamO > > > On 03/28/2017 06:27 AM, Colin Dixon wrote: >> Thanks for looking into this. We've heard other people say the same things >> too. >> >> --Colin >> >> On Tue, Mar 28, 2017 at 8:57 AM Diego Jesus Granados Lopez >> <[email protected] >> <mailto:[email protected]>> wrote: >> >> I'm adding more evidence. From one of the failed tasks [1]: >> >> GIT_URL ssh://[email protected]:29418/integration/test >> <http://[email protected]:29418/integration/test> >> GIT_BRANCH origin/master GIT_COMMIT >> 9f3c006279ca73a063f923e64a0c74aebf809a9d >> >> At execution logs: >> >> 04:52:41 > git fetch --tags --progress >> ssh://[email protected]:29418/integration/test >> <http://[email protected]:29418/integration/test> master >> 04:52:41 > git rev-parse FETCH_HEAD^{commit} >> # timeout=10 04:52:41 Checking out Revision >> 9f3c006279ca73a063f923e64a0c74aebf809a9d (origin/master) 04:52:41 > git >> config core.sparsecheckout # timeout=10 04:52:41 > git checkout -f >> 9f3c006279ca73a063f923e64a0c74aebf809a9d >> >> That commit being used (9f3c) is the carbon tip (not in boron). So the wrong >> version of integration/test is being used >> for running the test. >> >> I'm not sure of which project is in charge of that script / task >> configuration in order to forward this info to them. >> Is it releng/builder? >> >> [1] https://jenkins.opendaylight.org/releng/job/sfc-csit-3node-rest-basic- >> all-boron/334 >> >> -----Original Message----- From: Ursicio Javier Martin Sent: martes, 28 de >> marzo de 2017 12:55 To: Diego Jesus Granados >> Lopez <[email protected] >> <mailto:[email protected]>>; Jamo Luhrsen >> <[email protected] <mailto:[email protected]>>; >> [email protected] >> <mailto:[email protected]> Cc: Colin Dixon >> <[email protected] <mailto:[email protected]>> Subject: >> RE: [sfc-dev] SFC Boron-SR3 test failure checkoff >> >> Hi Jamo, >> >> After looking at the execution logs I agree with Diego, for some reason >> tests devoted to Carbon are being executed in >> Boron scope. >> >> Maybe a Releng issue ... >> >> BR, >> >> Ursicio >> >> -----Original Message----- From: [email protected] >> <mailto:[email protected]> >> [mailto:[email protected] >> <mailto:[email protected]>] On Behalf Of Diego >> Jesus Granados Lopez Sent: martes, 28 de marzo de 2017 10:29 To: Jamo >> Luhrsen <[email protected] >> <mailto:[email protected]>>; [email protected] >> <mailto:[email protected]> Cc: Colin Dixon >> <[email protected] <mailto:[email protected]>> Subject: Re: [sfc-dev] >> SFC Boron-SR3 test failure checkoff >> >> Hi, >> >> I checked the failing job and most (not sure if all, but probably) of the >> failing tests shouldn't be executing in boron >> checks. E.g. there are 4 tests consistently failing that shouldn't be >> executing in Boron (Service Path validation: >> added by me) they were merged to carbon only one month ago. >> >> Any chance the Jenkins job is misconfigured and using master branch instead >> of boron-something for running that jobs? I >> looked at the job config and it contains a "GERRIT_BRANCH" parameter set to >> "stable/boron", but as I said if it was >> being used, tests added to carbon only shouldn't be being run >> >> BR, Diego >> >> -----Original Message----- From: [email protected] >> <mailto:[email protected]> >> [mailto:[email protected] >> <mailto:[email protected]>] On Behalf Of Jamo >> Luhrsen Sent: lunes, 27 de marzo de 2017 22:57 To: >> [email protected] >> <mailto:[email protected]> Cc: Colin Dixon >> <[email protected] <mailto:[email protected]>> Subject: >> Re: [sfc-dev] SFC Boron-SR3 test failure checkoff >> >> SFC, >> >> I took a look at one of your failing jobs from the tracking sheet and it >> does have seemingly consistent failures now. >> But, the same job back in the SR2 timeframe was passing 100%. Maybe there is >> a regression? >> >> job from SR2-ish: >> >> >> https://logs.opendaylight.org/releng/jenkins092/sfc-csit-3node-rest-ba >> sic-all-boron/212/archives/log.html.gz >> >> >> Thanks, JamO >> >>> tl;dr Your project has test failures in the Boron-SR3 distribution tests. >>> Please look at them and mark them as OKAY in >>> the spreadsheet if they're not blocking or start a conversation with us >>> _now_ if they are. >>> >>> >>> Full details: The SFC project has at least one test failure in the >>> distribution test from the proposed Boron-SR3 >>> release here: >>> https://wiki.opendaylight.org/view/Simultaneous_Release:Boron_Release_ >>> Plan#Boron_SR3_Download >>> >>> To find the test failures, go to the distribution test report link at the >>> page above or directly here: >>> https://jenkins.opendaylight.org/releng/view/distribution/job/integrat >>> ion-distribution-test-boron/173/ >>> >>> Then look for yellow or red balls after tests that start with your >>> project's short name and click on the number to go >>> to the actual run that produced the failures. >>> >>> Please check to see if the failures are a blocking issue for releasing >>> Boron-SR3 and if so, start the discussion and >>> if not, update the spreadsheet here with OK in front of the issues: >>> https://docs.google.com/spreadsheets/d/1zImtd764e-hOgJAxoJKl85fxHCPu2a >>> gLfqsBtf13zQY/edit#gid=1776065453 >>> >>> Ideally, also follow up here or on this thread: >>> https://lists.opendaylight.org/pipermail/release/2017-March/009712.htm l >>> >>> Cheers, --Colin >> _______________________________________________ sfc-dev mailing list >> [email protected] >> <mailto:[email protected]> >> https://lists.opendaylight.org/mailman/listinfo/sfc-dev >> _______________________________________________ sfc-dev mailing list >> [email protected] >> <mailto:[email protected]> >> https://lists.opendaylight.org/mailman/listinfo/sfc-dev >> _______________________________________________ sfc-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/sfc-dev
