Hi Jaya, We are sending status report of openflow service being operational to infrautils diagstatus service, as and when we receive systemBootReady notification. This happened in 2 mins from the start of controller. We also report the status to be ERROR in case of some issues. As I am not able to see any such logs, issue has to be checked at infrautils diagstatus service, whether it received status report properly.
Regards, Somashekhar -----Original Message----- From: Jaya Priyadarshini Sent: Tuesday, April 2, 2019 4:46 PM To: Jamo Luhrsen <[email protected]>; SOMASHEKHAR MANOHARA JAVALAGI <[email protected]>; Faseela K <[email protected]>; [email protected]; [email protected]; D Arunprakash <[email protected]> Cc: openflowplugin-dev <[email protected]> Subject: RE: [integration-dev] [openflowplugin-dev] Netvirt Apex jobs in jenkins - openflow stuck in starting state Hi Som, As per info which u asked, https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/71/console-timestamp.log.gz ODL is started at 07:22:10 2019-03-31 07:22:10,517 INFO: Starting Opendaylight on node overcloud-controller-0.opnfvlf.org And it is checking if all bundles came up at around this time. 08:32:41 ============================================================================== 08:32:41 01_l2 :: Test suite to verify packet flows between vm instances. 08:32:41 ============================================================================== Regards Jaya -----Original Message----- From: Jamo Luhrsen <[email protected]> Sent: Monday, April 1, 2019 9:34 PM To: SOMASHEKHAR MANOHARA JAVALAGI <[email protected]>; Faseela K <[email protected]>; Jaya Priyadarshini <[email protected]>; [email protected]; [email protected]; D Arunprakash <[email protected]> Cc: openflowplugin-dev <[email protected]> Subject: Re: [integration-dev] [openflowplugin-dev] Netvirt Apex jobs in jenkins - openflow stuck in starting state something is broken. Looks like the failures are only there in sodium. here is a karaf log from job #71 failure: https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/71/controller_sos/sosreport-controllerreport-20190331083711/opt/opendaylight/data/log/karaf.log.gz you can see some WARN messages in TracingBroker which seem to just repeat. the controller is up for at least 10m before the actual failure is finally noted. In a neon job I just checked, the same validation is completed in 3s. JamO On 4/1/19 4:44 AM, SOMASHEKHAR MANOHARA JAVALAGI wrote: > Hi Faseela, > > Karaf is started at Mar 31, 2019 7:23:59 AM. > > Once all the bundles are up, we are registering to SystemReadyMonitor > by 2019-03-31T07:25:31,172 > > After 16 seconds we are receiving onSystemBootready notification at > 2019-03-31T07:25:47,219 > > Immediately within a second we are marking openflow service status as > operational at 2019-03-31T07:25:48,709 > > As bundles coming up is taking more time, we are finding delay in operational > status change of openflow service. > > Regards, > > Somashekhar > > *From:* [email protected] > <[email protected]> *On Behalf Of > *Faseela K > *Sent:* Monday, April 1, 2019 12:43 PM > *To:* Jaya Priyadarshini <[email protected]>; > [email protected]; > [email protected]; D Arunprakash > <[email protected]> > *Cc:* openflowplugin-dev <[email protected]> > *Subject:* Re: [openflowplugin-dev] Netvirt Apex jobs in jenkins - > openflow stuck in starting state > > Arun, > > Any idea when this can happen? > > System Ready State says all bundles have come up, but OFP is still in > starting state. > > Thanks, > > Faseela > > *From:* [email protected] > <mailto:[email protected]> > <[email protected] > <mailto:[email protected]>> *On Behalf Of > *Jaya Priyadarshini > *Sent:* Monday, April 1, 2019 12:41 PM > *To:* [email protected] > <mailto:[email protected]>; > [email protected] > <mailto:[email protected]> > *Subject:* [netvirt-dev] Apex jobs in jenkins > > Here is the apex jobs status. > > Success > > > > 100% > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-fluorin > e/lastBuild> > > > > netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack- > fluorine > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-fluorin > e/> > > > > 8 hr 19 min - #142 > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-fluorin > e/lastSuccessfulBuild/> > > > > 18 days - #123 > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-fluorin > e/lastFailedBuild/> > > > > 1 hr 0 min > > > > 25 / 25 passed > https://jenkins.opendaylight.org/releng/plugin/robot/robot.png > > > > Success > > > > 100% > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-neon/la > stBuild> > > > > netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack- > neon > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-neon/> > > > > 15 hr - #158 > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-neon/la > stSuccessfulBuild/> > > > > 11 days - #148 > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-neon/la > stFailedBuild/> > > > > 52 min > > > > 25 / 25 passed > https://jenkins.opendaylight.org/releng/plugin/robot/robot.png > > > > In progress > > > > 0% > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/ > lastBuild> > > > > netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack- > sodium > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/ > > > > > > 23 hr - #71 > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/ > lastSuccessfulBuild/> > > > > 16 days - #40 > <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt > -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/ > lastFailedBuild/> > > > > 1 hr 36 min > > > > 0 / 25 passed > https://jenkins.opendaylight.org/releng/plugin/robot/robot.png > > > > 3^rd failure is due to following reason > > > > [?1l>[?2004lTimestamp: Sun Mar 31 08:33:15 UTC 2019 Node IP Address: > /127.0.0.1 System is operational: false System ready state: ACTIVE > > OPENFLOW : STARTING (INITIALIZING) <<<<<<<<<<<<<<<<<<<<<<<<<<<<<It never > comes to operational. > > IFM : OPERATIONAL > > ITM : OPERATIONAL > > ELAN : OPERATIONAL (Service started) > > OVSDB : OPERATIONAL (OVSDB initialization complete) > > DATASTORE : OPERATIONAL [?1h=[90m~[0m > > So Suite setup itself has failed and hence all failures. > > Now how do we go ahead with apex, what should be next step? > > Regards > > Jaya > > > _______________________________________________ > integration-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/integration-dev > _______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
