Hi Luis, Thank you for sharing the great information.
The information matches with my investigation. I've written the outcome of my investigation so far into the bug report. https://bugs.opendaylight.org/show_bug.cgi?id=6595#c2 I think the OpenFlow plugin failed to process port status message, because it failed to get a WriteTransaction. And, the OpenFlow plugin failed to get the WriteTransaction, because it was before it started the service as MASTER for the siwtch. Best Regards, Hideyuki Tai ________________________________________ From: [email protected] <[email protected]> on behalf of Luis Gomez <[email protected]> Sent: Thursday, September 1, 2016 7:17 PM To: Shuva Jyoti Kar Cc: [email protected] Subject: Re: [openflowplugin-dev] Blocking bugs In the case of cluster the port status ERROR seem to happen just after a member becomes SLAVE for a device: https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3node-clustering-only-boron/606/archives/odl3_karaf.log.gz While in VTN Manger the issue seems to happen because of longer Master selection the first time switch connects to controller: 2016-08-31 23:18:01,788 | INFO | entLoopGroup-5-2 | ConnectionAdapterImpl | 173 - org.opendaylight.openflowjava.openflow-protocol-impl - 0.8.0.Boron-RC2 | Hello received / branch 2016-08-31 23:18:02,428 | WARN | entLoopGroup-5-3 | DeviceContextImpl | 183 - org.opendaylight.openflowplugin.impl - 0.3.0.Boron-RC2 | Error processing port status message: 2016-08-31 23:18:02,553 | INFO | lt-dispatcher-20 | LifecycleServiceImpl | 183 - org.opendaylight.openflowplugin.impl - 0.3.0.Boron-RC2 | ========== Start-up clustering MASTER services for node openflow:1 was SUCCESSFUL ========== In successive switch connections the Master resolution is very fast: 2016-08-31 23:27:28,551 | INFO | entLoopGroup-5-6 | ConnectionAdapterImpl | 173 - org.opendaylight.openflowjava.openflow-protocol-impl - 0.8.0.Boron-RC2 | Hello received / branch 2016-08-31 23:27:28,582 | INFO | lt-dispatcher-21 | LifecycleServiceImpl | 183 - org.opendaylight.openflowplugin.impl - 0.3.0.Boron-RC2 | ========== Start-up clustering MASTER services for node openflow:1 was SUCCESSFUL ========== BR/Luis > On Sep 1, 2016, at 10:33 AM, Luis Gomez <[email protected]> wrote: > > Sure, will do that today. > > >> On Sep 1, 2016, at 10:11 AM, Shuva Jyoti Kar <[email protected]> >> wrote: >> >> Luis/Venkat, >> >> Have pushed a gerrit get more information on the port whose status is being >> missed: >> https://git.opendaylight.org/gerrit/#/c/45024/ >> >> Please try it out with this and share the logs to analyse the failure to >> process the port status message. >> We need to establish that the tx chain is activated while we are trying to >> update the port status >> >> Thanks >> Shuva >> >> From: Luis Gomez [mailto:[email protected]] >> Sent: Thursday, September 01, 2016 2:40 PM >> To: Shuva Jyoti Kar >> Cc: Venkatrangan G - ERS, HCL Tech; Andrej Leitner; >> [email protected] >> Subject: Re: [openflowplugin-dev] Blocking bugs >> >> FYI the "Error processing port status message" is not seen very often in >> openflow test suites for 1 node but it is very present in 3 node cluster >> test: >> >> >> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3node-clustering-only-boron/604/archives/odl1_karaf.log.gz >> >> >> BR/Luis >> >> >> >> >> On Sep 1, 2016, at 1:21 AM, Shuva Jyoti Kar <[email protected]> >> wrote: >> >> Venkat, >> >> I see add flow failing with “Device reported error type BADACTION code >> BADSETARGUMENT”, “Device disconnected” and >> “java.util.concurrent.CancellationException: Task was cancelled” , which are >> genuine. >> Do we know what is the flow that is being rejected as a BADACTION , or why >> is there a flow push when the device is disconnected or why was the task >> cancelled ? >> >> Thanks >> Shuva >> >> From: Venkatrangan G - ERS, HCL Tech [mailto:[email protected]] >> Sent: Thursday, September 01, 2016 1:14 PM >> To: Shuva Jyoti Kar; Luis Gomez; Andrej Leitner >> Cc: [email protected] >> Subject: RE: [openflowplugin-dev] Blocking bugs >> >> Shuva, >> It fails very frequently since 23rd this month. Every time it fails all >> the ping tests in OF10 fail, which makes it unusable for Openflow 1.0. >> Ref: >> https://logs.opendaylight.org/releng/jenkins092/vtn-csit-1node-coordinator-only-boron/195/archives/karaf.log.gz >> >> Remove Flows RPC has resulted in a failure, Add Flow has resulted in a >> failure, Statistics has resulted in a failure, Due to these failures the >> flow entries were not installed. >> >> >> Regards, >> Venkat G >> >> From: Shuva Jyoti Kar [mailto:[email protected]] >> Sent: Wednesday, August 31, 2016 11:31 PM >> To: Venkatrangan G - ERS, HCL Tech <[email protected]>; Luis Gomez >> <[email protected]>; Andrej Leitner <[email protected]> >> Cc: [email protected] >> Subject: RE: [openflowplugin-dev] Blocking bugs >> >> Venkat, >> >> Could you please explain your last statement? Does it fail sporadically or >> consistently ? >> >> Also the statement for bug 6595 states “The latest regression runs for VTN >> CSIT jobs reports failure >> in normal flow installation when testing with OF 1.0 switches” >> Is that correct since the flow installation fails genuinely because of >> errors. >> >> Please let us know >> >> Thanks >> Shuva >> >> From: Venkatrangan G - ERS, HCL Tech [mailto:[email protected]] >> Sent: Thursday, September 01, 2016 11:57 AM >> To: Shuva Jyoti Kar; Luis Gomez; Andrej Leitner >> Cc: [email protected] >> Subject: RE: [openflowplugin-dev] Blocking bugs >> >> Shuva, >> >> We also see that the Port Statistics RPC has failed, which results in VTN >> concluding that ports are DOWN. VTN is a reactive implementation, hence VTN >> does not install entries considering the egress port is DOWN. >> The tests are not failing in every run, but every time it fails, OF1.0 >> failures are observed. >> >> >> Regards, >> Venkat G >> >> From: [email protected] >> [mailto:[email protected]] On Behalf Of >> Shuva Jyoti Kar >> Sent: Wednesday, August 31, 2016 11:18 PM >> To: Luis Gomez <[email protected]>; Andrej Leitner >> <[email protected]> >> Cc: [email protected] >> Subject: Re: [openflowplugin-dev] Blocking bugs >> >> Yes, but if I drill into the logs >> >> 1) >> https://logs.opendaylight.org/releng/jenkins092/vtn-csit-1node-coordinator-only-boron/209/archives/karaf.log.gz >> a. errors=Device disconnected (3times) >> 2) >> https://logs.opendaylight.org/releng/jenkins092/vtn-csit-1node-manager-only-boron/637/archives/karaf.log.gz >> a. errors=Device reported error type BADACTION code BADSETARGUMENT(19 >> times) >> b. errors=Device disconnected (10 times) >> c. reason java.util.concurrent.CancellationException: Task was >> cancelled(5 times) >> >> If you search for SalFlowServiceImpl. >> We do have 0f1.0 TCs for flowprovisioning , are those passing ? If yes then >> probably , vtn might need to drill down further >> >> Thanks >> Shuva >> >> >> >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Luis >> Gomez >> Sent: Thursday, September 01, 2016 11:35 AM >> To: Andrej Leitner >> Cc: [email protected] >> Subject: Re: [openflowplugin-dev] Blocking bugs >> >> The bug 6654 is happening sporadically in our system test but you can >> reproduce it very quickly if you stop and start mininet with no delay for >> few times. Thats what I did to confirm the issue. >> >> The bug 6595 is blocker because VTN has a regression in its OF10 system test >> using the new plugin. BTW I also see this exception very often in the >> openflow system test so it would be good to fix it for the Boron release. >> >> BR/Luis >> >> >> >> On Aug 31, 2016, at 8:43 PM, Andrej Leitner <[email protected]> >> wrote: >> >> Hi, >> I assume 6554 is not fixed yet. I've tried to get into it, but was unable to >> reproduce it locally. Today is public holiday on Slovakia so will continue >> on Friday. I don't know 6595, must be something new. >> From: Abhijit Kumbhare <[email protected]> >> Sent: Wednesday, August 31, 2016 9:09:54 PM >> To: [email protected] >> Subject: [openflowplugin-dev] Blocking bugs >> >> Hi folks, >> >> We seem to have two blocking bugs - that we should discuss tomorrow if not >> fixed already: >> >> https://bugs.opendaylight.org/show_bug.cgi?id=6554 >> >> https://bugs.opendaylight.org/show_bug.cgi?id=6595 >> >> Thanks, >> Abhijit >> AndrejLeitner >> Software Developer >> >> Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia >> R&D centrum / Janka Kráľa 9 / 974 01 Banská Bystrica / Slovakia >> / [email protected] >> reception: +421 2 206 65 114 / www.pantheon.sk >> >> >> _______________________________________________ >> openflowplugin-dev mailing list >> [email protected] >> https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev >> >> >> >> ::DISCLAIMER:: >> ---------------------------------------------------------------------------------------------------------------------------------------------------- >> The contents of this e-mail and any attachment(s) are confidential and >> intended for the named recipient(s) only. >> E-mail transmission is not guaranteed to be secure or error-free as >> information could be intercepted, corrupted, >> lost, destroyed, arrive late or incomplete, or may contain viruses in >> transmission. The e mail and its contents >> (with or without referred errors) shall therefore not attach any liability >> on the originator or HCL or its affiliates. >> Views or opinions, if any, presented in this email are solely those of the >> author and may not necessarily reflect the >> views or opinions of HCL or its affiliates. Any form of reproduction, >> dissemination, copying, disclosure, modification, >> distribution and / or publication of this message without the prior written >> consent of authorized representative of >> HCL is strictly prohibited. If you have received this email in error please >> delete it and notify the sender immediately. >> Before opening any email and/or attachments, please check them for viruses >> and other defects. >> ---------------------------------------------------------------------------------------------------------------------------------------------------- > _______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev _______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
