I am probably responding to this thread too late.. but my comments inline ..
On Wed, Mar 23, 2016 at 1:26 AM, Michal Rehak -X (mirehak - PANTHEON TECHNOLOGIES at Cisco) <[email protected]> wrote: > Hi guys, > > > regarding flows cleaning: in order to keep flows in DS/operational > consistent with device there is different logic for that in Li-design. > Requirements are - stay consistent in cluster even after leader change. So > keeping local cache of flows and computing which flow should be removed > from DS/operational after stats get processed might fail. But removing all > flows before writing fresh reported flows is simple and sufficient strategy > here. It is also faster then removing outdated flows one by one. > Michal, did we do any performance test with this strategy ? I see one issue here for the application those are listening to operational data store. When you will remove all the flows, application will get notification about the deleting the flow and then immediately you will write the flow back to data store, that will generate another notification for the applications. I think this is violating the contract with the application, that flow is present in the switch, but we are still notifying them that it's deleted. This can create lot of performance impact on application, if applications are reactive to the flows presence in the switch. As far as i understand, it's a broken functionality. > > But since ovs-2.4 we have huge data block in table element: > table-features. And by writing table with empty collection of flows these > big table-features are written too. By every statistics response. This is > causing the cpu load and renders controller unusable for more than like 63 > switches (topo=tree,6). > > > Switching table-features off via config-subsystem is just a workaround for > testing. > > There is better solution - move table-features out of table. But here we > need to ask downstream projects on how they use table-features. If the only > use case relies on rpc then we are good. If DS/config or DS/operational are > involved then we need to adapt these projects. That's why I did not answer > the second question - I have no idea about impact here. > Moving table features' out of the table looks a better approach to me. As far as i know, DIDM/NIC will require very minimal work to adopt these changes. > > Also it might be worth to think about evicting table-features out of DS > completely - if the only use case relies on rpc and there is no real > requirement on reading, writing or listening to table-features in DS. > As far as we write to data store, on the connection and whenever user changes the table features, it should not be a significant load. But I am open for the RPC approach as well, but then it's very confusing for the applications, because the information they requires will be scattered around data store and RPC. > > > So I guess we should discuss table-features with downstreamers as soon as > possible in order to have this fixed by SR2. > > > > > Regards, > > Michal > > > > ------------------------------ > *From:* Abhijit Kumbhare <[email protected]> > *Sent:* Tuesday, March 22, 2016 19:33 > *To:* Anil Vishnoi > *Cc:* Michal Rehak -X (mirehak - PANTHEON TECHNOLOGIES at Cisco); Luis > Gomez; An Ho; Jamo Luhrsen; Kamal Rameshan (kramesha); Jozef Bacigal -X > (jbacigal - PANTHEON TECHNOLOGIES at Cisco); Shuva Jyoti Kar; Muthukumaran > K; Alexis de Talhouët > > *Subject:* Re: Beryllium SR1 issue with OVS 2.4 - bug 5464 > > Sorry - our emails clashed mid air - asking the same question :). > Michal - will be great if you just respond on my last question on the > list. Reason: this is more of a technical discussion than a logistics one > that the email thread had started with (who will fix it). > > On Tue, Mar 22, 2016 at 11:18 AM, Anil Vishnoi <[email protected]> > wrote: > >> >> >> On Tue, Mar 22, 2016 at 2:36 AM, Michal Rehak -X (mirehak - PANTHEON >> TECHNOLOGIES at Cisco) <[email protected]> wrote: >> >>> Hi Luis, >>> >>> good question (1). >>> >>> In He design there is statistics manager writing only flows which >>> arrived from device and then removing the rest. Which means it is slow but >>> it is not touching table features. >>> >>> But in Li design this action is simplified by writing empty flow >>> collections to every table first and then putting all received flows+stats >>> in. And this might be the reason - in order to write empty flow lists we >>> need to read whole table, change flow collection and write it back. >>> >> Michal, what do you mean by empty flow collection? and why are we doing >> that, is it for performance improvement? >> >> >>> The instance of data tree is lazy but DS has to sync before exposing >>> lazy data tree. >>> >>> >>> >>> Regards, >>> >>> Michal >>> >>> >>> ------------------------------ >>> *From:* Luis Gomez <[email protected]> >>> *Sent:* Monday, March 21, 2016 17:23 >>> *To:* Michal Rehak -X (mirehak - PANTHEON TECHNOLOGIES at Cisco) >>> *Cc:* Abhijit Kumbhare; An Ho; Jamo Luhrsen; Kamal Rameshan (kramesha); >>> Anil Vishnoi; Jozef Bacigal -X (jbacigal - PANTHEON TECHNOLOGIES at Cisco); >>> Shuva Jyoti Kar; Muthukumaran K; Alexis de Talhouët >>> >>> *Subject:* Re: Beryllium SR1 issue with OVS 2.4 - bug 5464 >>> >>> Michal, I have 2 questions here: >>> >>> 1) In the case of He plugin, the OVS 2.4 table features impact the CPU >>> only at switch connection time when table features are exchanged, however >>> in Li plugin the CPU is high even after the switch connects. Is there a >>> difference in the Li implementation that explains this behavior? >>> >>> 2) I think there are couple of projects, Anil may remember, that are >>> using table features. We should probably talk to them if we plan to make >>> this feature user configurable, right? >>> >>> BR/Luis >>> >>> >>> On Mar 21, 2016, at 3:20 AM, Michal Rehak -X (mirehak - PANTHEON >>> TECHNOLOGIES at Cisco) <[email protected]> wrote: >>> >>> Hi all, >>> I have prepared a draft change for master branch adding a config >>> subsystem flag for disabling table features. This effectively skips sending >>> table features query to device and answer failure immediately (exactly as >>> OVS-2.3 did). >>> >>> @Abhijit: Is it ok to bind this change to >>> https://bugs.opendaylight.org/show_bug.cgi?id=5464 >>> or do we need a new bug filed for this? >>> >>> I am not sure about destiny of table features as it introduces >>> significant load to DS. So we might move it somewhere outside tables and >>> flows in order to speedup DS for frequent operations. Or table features >>> might get completely evicted from DS - there is rpc for reading and >>> changing it anyway. I see table features as a huge load of data which comes >>> handy only at the beginning - every change to table features wipes out all >>> the flow. >>> >>> >>> Regards, >>> Michal >>> >>> ------------------------------ >>> *From:* Abhijit Kumbhare <[email protected]> >>> *Sent:* Monday, March 21, 2016 05:52 >>> *To:* Luis Gomez >>> *Cc:* An Ho; Jamo Luhrsen; Kamal Rameshan (kramesha); Anil Vishnoi; >>> Jozef Bacigal -X (jbacigal - PANTHEON TECHNOLOGIES at Cisco); Shuva Jyoti >>> Kar; Muthukumaran K; Michal Rehak -X (mirehak - PANTHEON TECHNOLOGIES at >>> Cisco); Alexis de Talhouët >>> *Subject:* Re: Beryllium SR1 issue with OVS 2.4 - bug 5464 >>> >>> Thanks for the investigation Luis! Especially about the lot of >>> information returned by OVS 2.4 regarding the table features. >>> >>> On Sunday, March 20, 2016, Luis Gomez <[email protected]> wrote: >>> >>>> Actually the OVS 2.4 scalability issue is only happening with Li plugin >>>> (I double checked today and updated the bug) so we are good with Be SR1. >>>> >>>> BR/Luis >>>> >>>> >>>> >>>> On Mar 18, 2016, at 8:57 AM, Abhijit Kumbhare <[email protected]> >>>> wrote: >>>> >>>> No - I think we should fix this in SR2 rather than delay SR1. >>>> >>>> On Fri, Mar 18, 2016 at 8:47 AM, An Ho <[email protected]> wrote: >>>> >>>>> We may have fixed the problem in the distribution with these patches >>>>> [1] [2] and several projects (aaa, capwap, etc) are passing their >>>>> distribution tests [3]. >>>>> >>>>> Now we need to identify if we need to respin Beryllium again based on >>>>> Josef's input. >>>>> >>>>> Abhijit, have you heard from folks? >>>>> >>>>> Best Regards, >>>>> An Ho >>>>> >>>>> https://git.opendaylight.org/gerrit/#/c/36208/ >>>>> https://git.opendaylight.org/gerrit/#/c/36420/ >>>>> >>>>> https://jenkins.opendaylight.org/releng/view/autorelease/job/integration-distribution-test-beryllium/346/ >>>>> >>>>> https://jenkins.opendaylight.org/releng/view/autorelease/job/autorelease-release-beryllium/96/ >>>>> >>>>> -----Original Message----- >>>>> From: Jamo Luhrsen [mailto:[email protected]] >>>>> Sent: Thursday, March 17, 2016 4:59 PM >>>>> To: Abhijit Kumbhare; An Ho >>>>> Cc: Kamal Rameshan (kramesha); Anil Vishnoi; Jozef Bacigal -X >>>>> (jbacigal - PANTHEON TECHNOLOGIES at Cisco); Shuva Jyoti Kar; Muthukumaran >>>>> K; Luis Gomez Palacios; Michal Rehak -X (mirehak - Pantheon Technologies >>>>> SRO at Cisco); Alexis de Talhouët >>>>> Subject: Re: Beryllium SR1 issue with OVS 2.4 - bug 5464 >>>>> >>>>> >>>>> >>>>> On 03/17/2016 04:55 PM, Abhijit Kumbhare wrote: >>>>> > Jozef is already looking at this I understand (from the bug comment). >>>>> > Anil's first thought was this may be an OpenFlow Java problem - and >>>>> > that it may be taking too much time to process a connection. It will >>>>> be easier for Jozef to follow up with Michal Rehak & Michal Polkorab if >>>>> that is actually the case. >>>>> > I think this is probably not a blocker for SR1 & may be fixed in SR2 >>>>> - >>>>> > as the problem here is scalability (not the basic operation). Can we >>>>> > wait till the morning to get Jozef's input whether it can be fixed >>>>> fast before deciding whether to have the fix for SR1 or for SR2? >>>>> >>>>> >>>>> My guess is that we can wait, as we still don't know what's going on >>>>> with the autorelease distros having trouble coming up. >>>>> >>>>> JamO >>>>> >>>>> >>>>> > On Thu, Mar 17, 2016 at 4:16 PM, An Ho <[email protected] <mailto: >>>>> [email protected]>> wrote: >>>>> > >>>>> > Hi Kamal, Anil, Jozef,____ >>>>> > >>>>> > __ __ >>>>> > >>>>> > Is there an ETA for when a patch is available?____ >>>>> > >>>>> > __ __ >>>>> > >>>>> > Please help identify if the issue is a BLOCKER (we must delay >>>>> Beryllium) or if we can release without a fix >>>>> > (workaround exists).____ >>>>> > >>>>> > __ __ >>>>> > >>>>> > Best Regards,____ >>>>> > >>>>> > An Ho____ >>>>> > >>>>> > __ __ >>>>> > >>>>> > *From:*Abhijit Kumbhare [mailto:[email protected] <mailto: >>>>> [email protected]>] >>>>> > *Sent:* Thursday, March 17, 2016 1:40 PM >>>>> > *To:* Kamal Rameshan (kramesha); Anil Vishnoi; Jozef Bacigal -X >>>>> (jbacigal - PANTHEON TECHNOLOGIES at Cisco); Shuva >>>>> > Jyoti Kar; Muthukumaran K >>>>> > *Cc:* Luis Gomez Palacios; Michal Rehak -X (mirehak - Pantheon >>>>> Technologies SRO at Cisco); Jamo Luhrsen; Alexis de >>>>> > Talhouët; An Ho >>>>> > *Subject:* Beryllium SR1 issue with OVS 2.4 - bug 5464____ >>>>> > >>>>> > __ __ >>>>> > >>>>> > Kamal, Anil,____ >>>>> > >>>>> > __ __ >>>>> > >>>>> > Jozef is probably looking at it - but can you guys take a look >>>>> at it as well today? It may be a blocker for Be SR 1 >>>>> > (not determined yet).____ >>>>> > >>>>> > __ __ >>>>> > >>>>> > I believe this may be with actual Be release as well.____ >>>>> > >>>>> > __ __ >>>>> > >>>>> > Abhijit____ >>>>> > >>>>> > __ __ >>>>> > >>>>> > ---------- Forwarded message ---------- >>>>> > From: *Jamo Luhrsen* <[email protected] <mailto: >>>>> [email protected]>> >>>>> > Date: Thu, Mar 17, 2016 at 1:18 PM >>>>> > Subject: Re: [release] [OpenDaylight TSC] Beryllium SR1 Status >>>>> 3/16 >>>>> > To: Luis Gomez <[email protected] <mailto:[email protected]>>, >>>>> Colin Dixon <[email protected] >>>>> > <mailto:[email protected]>> >>>>> > Cc: "[email protected] <mailto: >>>>> [email protected]>" <[email protected] >>>>> > <mailto:[email protected]>>, "release ( >>>>> [email protected] >>>>> > <mailto:[email protected]>)" >>>>> > <[email protected] >>>>> > <mailto:[email protected]>> >>>>> > >>>>> > >>>>> > I know that the Apex installer project uses OVS 2.4. Is that >>>>> enough to push for >>>>> > a fix in SR1? >>>>> > >>>>> > JamO >>>>> > >>>>> > On 03/17/2016 11:49 AM, Luis Gomez wrote: >>>>> > > Can we also ask opnfv community if they use OVS 2.4? as we >>>>> identified critical bug [1] in openflowplugin with this OVS >>>>> > > version and I am not sure this is something we should fix for >>>>> Be SR1. >>>>> > > >>>>> > > [1] https://bugs.opendaylight.org/show_bug.cgi?id=5464 >>>>> > > >>>>> > > >>>>> > >> On Mar 17, 2016, at 12:46 AM, Colin Dixon < >>>>> [email protected] <mailto:[email protected]> >>>>> > <mailto:[email protected] <mailto:[email protected]>>> >>>>> wrote: >>>>> > >> >>>>> > >> That make sense. >>>>> > >> >>>>> > >> Since OPNFV was the major downstream user expecting and/or >>>>> depending on this, are they going to be significantly >>>>> > >> impacted by the delay? Have we talked to them? >>>>> > >> >>>>> > >> --Colin >>>>> > >> >>>>> > >> >>>>> > >> On Wed, Mar 16, 2016 at 4:11 PM, An Ho <[email protected] >>>>> <mailto:[email protected]> <mailto:[email protected] >>>>> > <mailto:[email protected]>>> wrote: >>>>> > >> >>>>> > >> Hi Colin,____ >>>>> > >> >>>>> > >> __ __ >>>>> > >> >>>>> > >> We initially targeted Beryllium SR1 Release on 3/17, but >>>>> our latest build [1] has a critical bug related to >>>>> > >> vpnservice [2] and test failures in the integration >>>>> distribution test job which are still under further >>>>> > >> investigation [3]. Given the situation, I would like to >>>>> propose that we do the following:____ >>>>> > >> >>>>> > >> __ __ >>>>> > >> >>>>> > >> After the test failure investigation is complete, we >>>>> schedule a respin of the build to incorporate the patches >>>>> > >> fixing the vpnservice critical bugs and the distribution >>>>> test failures, then have projects sign off on the newly >>>>> > >> respun build. This may have the potential of delaying >>>>> the release of Beryllium SR1.____ >>>>> > >> >>>>> > >> __ __ >>>>> > >> >>>>> > >> The alternative is to release with the test failures and >>>>> the critical bugs.____ >>>>> > >> >>>>> > >> __ __ >>>>> > >> >>>>> > >> Best Regards,____ >>>>> > >> >>>>> > >> An Ho____ >>>>> > >> >>>>> > >> __ __ >>>>> > >> >>>>> > >> [1] >>>>> https://wiki.opendaylight.org/view/Simultaneous_Release:Beryllium_Release_Plan#Beryllium_SR1_Download____ >>>>> > >> >>>>> > >> [2] >>>>> https://lists.opendaylight.org/pipermail/vpnservice-dev/2016-March/000176.html____ >>>>> > >> >>>>> > >> [3] >>>>> https://lists.opendaylight.org/pipermail/integration-dev/2016-March/006100.html____ >>>>> > >> >>>>> > >> __ __ >>>>> > >> >>>>> > >> >>>>> > >> _______________________________________________ >>>>> > >> TSC mailing list >>>>> > >> [email protected] <mailto:[email protected]> >>>>> <mailto:[email protected] >>>>> > <mailto:[email protected]>> >>>>> > >> https://lists.opendaylight.org/mailman/listinfo/tsc >>>>> > > >>>>> > > >>>>> > > >>>>> > > _______________________________________________ >>>>> > > TSC mailing list >>>>> > > [email protected] <mailto:[email protected]> >>>>> > > https://lists.opendaylight.org/mailman/listinfo/tsc >>>>> > > >>>>> > _______________________________________________ >>>>> > release mailing list >>>>> > [email protected] <mailto: >>>>> [email protected]> >>>>> > https://lists.opendaylight.org/mailman/listinfo/release____ >>>>> > >>>>> > __ __ >>>>> > >>>>> > >>>>> >>>> >>>> >>>> >>> >>> >> >> >> -- >> Thanks >> Anil >> > > -- Thanks Anil
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
