Removed the other lists temporarily: Luis>> Unfortunately last week we realized of a new issue in carbon: https://bugs.opendaylight.org/show_bug.cgi?id=7884
Jozef - is this something you have any idea why this bug would only be in Carbon not in Boron? On Sun, Mar 5, 2017 at 7:11 PM, Luis Gomez <[email protected]> wrote: > Hi Vratko, some update on OpenFlow cluster issues: > > 1) table miss flow only pushed by 1 instance (new bug): https://bugs. > opendaylight.org/show_bug.cgi?id=7770 > > > There is already a candidate fix. > > 2) restart of device owner in non-HA scenarios does not work (old bug): > https://bugs.opendaylight.org/show_bug.cgi?id=6459 > > > This issue will be addressed in this other bug: https://bugs. > opendaylight.org/show_bug.cgi?id=7763 > > 3) Openflow cluster performance issues (old bug): https://bugs. > opendaylight.org/show_bug.cgi?id=6755 > > > I opened bug to controller project to better understand the log ERRORs: > https://bugs.opendaylight.org/show_bug.cgi?id=7901 > > Unfortunately last week we realized of a new issue in carbon: > https://bugs.opendaylight.org/show_bug.cgi?id=7884 > > BR/Luis > > > On Feb 9, 2017, at 5:32 PM, Luis Gomez <[email protected]> wrote: > > Hi Vratko, > > I investigated the issue I commented to you and created a bug for it, > currently we have these cluster related bugs in OpenFlow identified by the > system test (there could be more): > > 1) table miss flow only pushed by 1 instance (new bug): https://bugs. > opendaylight.org/show_bug.cgi?id=7770 > 2) restart of device owner in non-HA scenarios does not work (old bug): > https://bugs.opendaylight.org/show_bug.cgi?id=6459 > 3) Openflow cluster performance issues (old bug): https://bugs. > opendaylight.org/show_bug.cgi?id=6755 > > As you said it is unclear whether openflow cluster issues are openflow or > cluster related, all bugs are now in openflow queue and I would expect > openflow devs to move to cluster queue if that is where they belong to. > > BR/Luis > > > On Feb 7, 2017, at 10:34 AM, Luis Gomez <[email protected]> wrote: > > > On Feb 7, 2017, at 10:03 AM, Vratko Polak -X (vrpolak - PANTHEON > TECHNOLOGIES at Cisco) <[email protected]> wrote: > > Two more questions. > > > https://jenkins.opendaylight.org/releng/view/ > CSIT-3node/job/openflowplugin-csit-3node-clustering-only-boron/ > > Cluster non HA test > > I just realized 1) and 2) are the same job. > I am not sure which of the six suites [1] > are you referring to. > > > Typo, this is the link for non-HA: https://jenkins. > opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-3node- > periodic-bulkomatic-clustering-daily-only-boron/ > > > >> but other tests are not, I will have to investigate this. > > > > Keep us informed. > > Do you have an ETA? > > > I would say in the next 2 weeks I will have something in place for cluster > scalability. > > > Vratko. > > [1] https://logs.opendaylight.org/releng/jenkins092/ > openflowplugin-csit-3node-clustering-only-carbon/470/archives/log.html.gz > > *From:* Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) > *Sent:* 7 February, 2017 15:05 > *To:* 'Luis Gomez' <[email protected]> > *Cc:* [email protected]; controller- > [email protected]; openflowplugin-dev <openflowplugin-dev@lists. > opendaylight.org> > *Subject:* RE: [integration-dev] Clustering acceptance tests > > Thanks Luis. > > > but other tests are not, I will have to investigate this. > > Keep us informed. > > > 3) & 4) is probably controller cluster limitation. > > Both jobs occasionally pass, > and I have opened a Bug [0] for exceptions in karaf log. > To me, it looks like an error in OpenflowPlugin > (as opposed to Controller) code. > > > writing very fast (REST or internal app) on a shard follower DS, and > reading on the other follower. > > We plan to expand controller-csit-3node-rest-clust-cars-perf-only-carbon, > not sure yet whether this scenario will be included. > > Vratko. > > [0] https://bugs.opendaylight.org/show_bug.cgi?id=7750 > > *From:* Luis Gomez [mailto:[email protected] <[email protected]>] > *Sent:* 7 February, 2017 08:35 > *To:* Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) < > [email protected]> > *Cc:* [email protected]; controller- > [email protected]; openflowplugin-dev <openflowplugin-dev@lists. > opendaylight.org> > *Subject:* Re: [integration-dev] Clustering acceptance tests > > Here is what I know from OpenFlow plugin (cc-ing ofplugin devs): > > > * Does your project have a test plan mentioning specific cluster scenarios? > > > Not written test plan but we are running a bunch of cluster tests. > > > > * Do you have any of such scenarios implemented as Robot suites? > > > 1) https://jenkins.opendaylight.org/releng/view/ > CSIT-3node/job/openflowplugin-csit-3node-clustering-only-boron/ -> > Cluster HA test (DPN connect to all nodes), it used to pass except for 1 > test (member isolation with iptables), now I see this test is stable but > other tests are not, I will have to investigate this. > > 2) https://jenkins.opendaylight.org/releng/view/ > CSIT-3node/job/openflowplugin-csit-3node-clustering-only-boron/ -> > Cluster non HA test (DPN connect to 1 node), failing because this old bug: > https://bugs.opendaylight.org/show_bug.cgi?id=6459. > > 3) https://jenkins.opendaylight.org/releng/view/ > CSIT-3node/job/openflowplugin-csit-3node-periodic- > bulkomatic-clustering-perf-daily-only-boron/ -> Max flows/sec using > bulk-o-matic DS on cluster setup. Not fully working because some cluster > backend limitation https://bugs.opendaylight.org/show_bug.cgi?id=6755 > > 4) https://jenkins.opendaylight.org/releng/view/ > CSIT-3node/job/openflowplugin-csit-3node-periodic-restconf- > clustering-perf-daily-only-boron/ -> Max flows/sec using NB REST on > cluster setup, this never worked very good because previous bug. > > > * Do the robot suites have failures, suspected to be caused by clustering > (as opposed to application logic, or mistakes in Robot code)? > > > So far I think issue in 2) is OpenFlow cluster implementation and issue in > 3) & 4) is probably controller cluster limitation. > > > > * Are there open Bugs corresponding to the clustering failures? > > > Yes, except for 1) that will require some analysis on the unstable tests. > > > > * Are you planning to implement more Robot 3node suites until Carbon > release? > > > I will probably replace 1 of the performance suites (no point to run 2 if > they do not work) by a cluster switch scalability test. > > > > * Are there scenarios you would like Controller team to cover using mock > apps? > > > I think issue in 3) & 4) could be reproduced in controller project by just > writing very fast (REST or internal app) on a shard follower DS, and > reading on the other follower. > > > On Feb 6, 2017, at 5:31 AM, Vratko Polak -X (vrpolak - PANTHEON > TECHNOLOGIES at Cisco) <[email protected]> wrote: > > Hello Test Contacts. > > In Controller project, our highest priority > for Carbon release is to make sure ODL clustering > is usable and stable. > > We are in the phase of formulating explicit acceptance criteria, > so we can create execution plan for turning them into Robot suites. > > Of course, clustering is not very useful just by itself, > it is used as a tool applications can use to reach their goals. > So real acceptance criteria for clustering should also > take into account whether ODL applications can work in cluster. > > Many projects are already running their 3node CSIT tests, > but on one hand, some important scenarios might be not covered yet, > and some suites might be too unstable to serve as acceptance tests. > > Controller team is small and busy, so we are asking for help. > Here is a set of quick questions for test contacts: > * Does your project have a test plan mentioning specific cluster scenarios? > * Do you have any of such scenarios implemented as Robot suites? > * Do the robot suites have failures, suspected to be caused by clustering > (as opposed to application logic, or mistakes in Robot code)? > * Are there open Bugs corresponding to the clustering failures? > * Are you planning to implement more Robot 3node suites until Carbon > release? > * Are there scenarios you would like Controller team to cover using mock > apps? > > Vratko (as a Controller test contact). > _______________________________________________ > integration-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/integration-dev > > > > > > _______________________________________________ > integration-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/integration-dev > >
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
