Hi Som,
As per info which u asked,
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/71/console-timestamp.log.gz
ODL is started at
07:22:10 2019-03-31 07:22:10,517 INFO: Starting Opendaylight on node
overcloud-controller-0.opnfvlf.org
And it is checking if all bundles came up at around this time.
08:32:41
==============================================================================
08:32:41 01_l2 :: Test suite to verify packet flows between vm instances.
08:32:41
==============================================================================
Regards
Jaya
-----Original Message-----
From: Jamo Luhrsen <[email protected]>
Sent: Monday, April 1, 2019 9:34 PM
To: SOMASHEKHAR MANOHARA JAVALAGI <[email protected]>;
Faseela K <[email protected]>; Jaya Priyadarshini
<[email protected]>; [email protected];
[email protected]; D Arunprakash <[email protected]>
Cc: openflowplugin-dev <[email protected]>
Subject: Re: [integration-dev] [openflowplugin-dev] Netvirt Apex jobs in
jenkins - openflow stuck in starting state
something is broken.
Looks like the failures are only there in sodium. here is a karaf log from job
#71 failure:
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/71/controller_sos/sosreport-controllerreport-20190331083711/opt/opendaylight/data/log/karaf.log.gz
you can see some WARN messages in TracingBroker which seem to just repeat.
the controller is up for at least 10m before the actual failure is finally
noted. In a neon job I just checked, the same validation is completed in 3s.
JamO
On 4/1/19 4:44 AM, SOMASHEKHAR MANOHARA JAVALAGI wrote:
> Hi Faseela,
>
> Karaf is started at Mar 31, 2019 7:23:59 AM.
>
> Once all the bundles are up, we are registering to SystemReadyMonitor
> by 2019-03-31T07:25:31,172
>
> After 16 seconds we are receiving onSystemBootready notification at
> 2019-03-31T07:25:47,219
>
> Immediately within a second we are marking openflow service status as
> operational at 2019-03-31T07:25:48,709
>
> As bundles coming up is taking more time, we are finding delay in operational
> status change of openflow service.
>
> Regards,
>
> Somashekhar
>
> *From:* [email protected]
> <[email protected]> *On Behalf Of
> *Faseela K
> *Sent:* Monday, April 1, 2019 12:43 PM
> *To:* Jaya Priyadarshini <[email protected]>;
> [email protected];
> [email protected]; D Arunprakash
> <[email protected]>
> *Cc:* openflowplugin-dev <[email protected]>
> *Subject:* Re: [openflowplugin-dev] Netvirt Apex jobs in jenkins -
> openflow stuck in starting state
>
> Arun,
>
> Any idea when this can happen?
>
> System Ready State says all bundles have come up, but OFP is still in
> starting state.
>
> Thanks,
>
> Faseela
>
> *From:* [email protected]
> <mailto:[email protected]>
> <[email protected]
> <mailto:[email protected]>> *On Behalf Of
> *Jaya Priyadarshini
> *Sent:* Monday, April 1, 2019 12:41 PM
> *To:* [email protected]
> <mailto:[email protected]>;
> [email protected]
> <mailto:[email protected]>
> *Subject:* [netvirt-dev] Apex jobs in jenkins
>
> Here is the apex jobs status.
>
> Success
>
>
>
> 100%
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-fluorin
> e/lastBuild>
>
>
>
> netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-
> fluorine
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-fluorin
> e/>
>
>
>
> 8 hr 19 min - #142
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-fluorin
> e/lastSuccessfulBuild/>
>
>
>
> 18 days - #123
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-fluorin
> e/lastFailedBuild/>
>
>
>
> 1 hr 0 min
>
>
>
> 25 / 25 passed
> https://jenkins.opendaylight.org/releng/plugin/robot/robot.png
>
>
>
> Success
>
>
>
> 100%
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-neon/la
> stBuild>
>
>
>
> netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-
> neon
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-neon/>
>
>
>
> 15 hr - #158
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-neon/la
> stSuccessfulBuild/>
>
>
>
> 11 days - #148
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-neon/la
> stFailedBuild/>
>
>
>
> 52 min
>
>
>
> 25 / 25 passed
> https://jenkins.opendaylight.org/releng/plugin/robot/robot.png
>
>
>
> In progress
>
>
>
> 0%
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/
> lastBuild>
>
>
>
> netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-
> sodium
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/
> >
>
>
>
> 23 hr - #71
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/
> lastSuccessfulBuild/>
>
>
>
> 16 days - #40
> <https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt
> -csit-1node-0cmb-1ctl-2cmp-apex-queens-upstream-snat-conntrack-sodium/
> lastFailedBuild/>
>
>
>
> 1 hr 36 min
>
>
>
> 0 / 25 passed
> https://jenkins.opendaylight.org/releng/plugin/robot/robot.png
>
>
>
> 3^rd failure is due to following reason
>
>
>
> [?1l>[?2004lTimestamp: Sun Mar 31 08:33:15 UTC 2019 Node IP Address:
> /127.0.0.1 System is operational: false System ready state: ACTIVE
>
> OPENFLOW : STARTING (INITIALIZING) <<<<<<<<<<<<<<<<<<<<<<<<<<<<<It never
> comes to operational.
>
> IFM : OPERATIONAL
>
> ITM : OPERATIONAL
>
> ELAN : OPERATIONAL (Service started)
>
> OVSDB : OPERATIONAL (OVSDB initialization complete)
>
> DATASTORE : OPERATIONAL [?1h=[90m~[0m
>
> So Suite setup itself has failed and hence all failures.
>
> Now how do we go ahead with apex, what should be next step?
>
> Regards
>
> Jaya
>
>
> _______________________________________________
> integration-dev mailing list
> [email protected]
> https://lists.opendaylight.org/mailman/listinfo/integration-dev
>
_______________________________________________
openflowplugin-dev mailing list
[email protected]
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
