On 10/19/2016 11:26 PM, Sai MarapaReddy wrote: > Address tracker tries [1] to read any existing address from data store. My > guess is in the case of immediate restart of > mininet , addresses from data store are not cleaned and this is why we see > all addresses. > > In further attempt to verify this information I have a patch here [2]. I > tried running it with keyword "test-l2switch-all" > but no success. Could you please run it. This might help in troubleshooting > the issue further.
it's because we don't have a patch test job for l2switch yet. It's something projects are adding on an as-needed basis. I gave you one here: https://git.opendaylight.org/gerrit/#/c/47219/ give it a +1 and we can try to get it merged. Thanks, JamO > [1] > https://github.com/opendaylight/l2switch/blob/stable/boron/addresstracker/implementation/src/main/java/org/opendaylight/l2switch/addresstracker/addressobserver/AddressObservationWriter.java#L116 > > [2] https://git.opendaylight.org/gerrit/#/c/47186/ > > On Wed, Oct 19, 2016 at 9:43 AM, Luis Gomez <ece...@gmail.com > <mailto:ece...@gmail.com>> wrote: > > Hi Sai, after looking at the test more in detail this is not exactly the > behavior (sorry for the confusion), what is > really happening is: > > - Mininet restarts > - There is no host in address-tracker after restart > - We do the classic mininet pingall test > - After the pingall test the host addresses (10.0.0.1, 10.0.0.2, > 10.0.0.3) are seen in all switches, while in normal > scenario 10.0.0.1 should be only in s1 to-host port, 10.0.0.2 should be > only in s2 to-host port and 10.0.0.3 should be > only in s3 to-host port. > > The normal scenario happens when we leave 2 secs between mininet stop + > start. > > So 2 questions: > > - What is the impact of address tracker registering remote IP addresses > in the switch-to-switch ports? As far as I can > see flows are generated correctly even in this case. > - Any idea why the address tracker would get confused and add IP > addresses to the switch-to-switch ports? maybe the > application fails to identify these ports as going switch-to-switch? > > BR/Luis > > >> On Oct 19, 2016, at 7:42 AM, Luis Gomez <ece...@gmail.com >> <mailto:ece...@gmail.com>> wrote: >> >> Timer=600 sec but in Boron when we leave just 2 secs between mininet >> stop & start, we do not see any IP address, also >> in Beryllium we do no see any IP address after restarting mininet, so >> are you sure switch disconnect/connect does not >> impact address tracker cache? >> >> BR/Luis >> >>> On Oct 19, 2016, at 3:37 AM, Sai MarapaReddy <sai.marapare...@gmail.com >>> <mailto:sai.marapare...@gmail.com>> wrote: >>> >>> Hi Luis, >>> >>> Addresses will be removed if this condition [1] passes. >>> >>> Default value of time 'timeStampUpdateInterval' is mentioned here [2]. >>> >>> Please try reducing the default value of time to make sure to addresses >>> are flushed. >>> >>> Since every time the mininet connects with same IP / Mac values, >>> addresses will only be removed if condition at [1] is >>> met. >>> >>> >>> >>> [1] >>> https://github.com/opendaylight/l2switch/blob/master/addresstracker/implementation/src/main/java/org/opendaylight/l2switch/addresstracker/addressobserver/AddressObservationWriter.java#L144 >>> >>> <https://github.com/opendaylight/l2switch/blob/master/addresstracker/implementation/src/main/java/org/opendaylight/l2switch/addresstracker/addressobserver/AddressObservationWriter.java#L144> >>> >>> [2] >>> https://github.com/opendaylight/l2switch/blob/master/addresstracker/implementation/src/main/yang/address-tracker-config.yang#L17 >>> >>> <https://github.com/opendaylight/l2switch/blob/master/addresstracker/implementation/src/main/yang/address-tracker-config.yang#L17> >>> >>> >>> >>> On Tue, Oct 18, 2016 at 7:30 PM, Luis Gomez <ece...@gmail.com >>> <mailto:ece...@gmail.com>> wrote: >>> >>> Thanks Sai, yes I have a question regrading the first issue below: >>> any idea why address tracker would keep >>> discovered IP address information after restarting mininet? like >>> what is today in the code to flush the address >>> cache and why this is not working when mininet restarts? >>> >>> BR/Luis >>> >>>> On Oct 18, 2016, at 6:56 PM, Sai MarapaReddy >>>> <sai.marapare...@gmail.com <mailto:sai.marapare...@gmail.com>> wrote: >>>> >>>> Thanks Luis. >>>> >>>> Yes that is drop-all flow. Please let us know if you need any >>>> further help from l2switch. >>>> >>>> On Tue, Oct 18, 2016 at 1:03 PM, Luis Gomez <ece...@gmail.com >>>> <mailto:ece...@gmail.com>> wrote: >>>> >>>> Actually this is the drop flow so it is expected, anyway going >>>> deeper into l2switch test issues I see: >>>> >>>> - The address tracker issue is because hosts addresses are not >>>> getting cleared when we restart mininet with >>>> no sleep in-between. >>>> - The loop remover issue could be related to some test issue, >>>> I am trying to clean up/repair the test here: >>>> https://git.opendaylight.org/gerrit/#/c/47095/ >>>> <https://git.opendaylight.org/gerrit/#/c/47095/> >>>> >>>> BR/Luis >>>> >>>> > On Oct 18, 2016, at 10:02 AM, Luis Gomez <ece...@gmail.com >>>> <mailto:ece...@gmail.com>> wrote: >>>> > >>>> > There are definitely weird things going on in l2switch, the >>>> first ERROR you mention produces some weird >>>> flow (no match and no action) in the switch: >>>> > >>>> > Flow 55: >>>> > "cookie": 3098476543630901303, >>>> > "flags": "", >>>> > "hard-timeout": 0, >>>> > "id": "55", >>>> > "idle-timeout": 0, >>>> > "match": {}, >>>> > >>>> "opendaylight-flow-statistics:flow-statistics": { >>>> > "byte-count": 0, >>>> > "duration": { >>>> > "nanosecond": 239000000, >>>> > "second": 36 >>>> > }, >>>> > "packet-count": 0 >>>> > }, >>>> > "priority": 0, >>>> > "table_id": 0 >>>> > >>>> > >>>> > The second issue, I am not sure what is the impact but >>>> definitely something to look at. >>>> > >>>> > I think in general we will need l2switch support to >>>> troubleshoot and progress in current issues. >>>> > >>>> > BR/Luis >>>> > >>>> > >>>> >> On Oct 17, 2016, at 8:09 AM, Miroslav Macko >>>> <miroslav.ma...@pantheon.tech >>>> <mailto:miroslav.ma...@pantheon.tech>> wrote: >>>> >> >>>> >> Hello Luis and dev guys, >>>> >> >>>> >> There is this info from l2-switch in the karaf log: >>>> >> >>>> >> 2016-10-14 22:43:24,941 | INFO | pool-16-thread-1 | >>>> FlowWriterServiceImpl | 229 - >>>> org.opendaylight.l2switch.main.impl - 0.5.0.SNAPSHOT | In >>>> addMacToMacFlowsUsingShortestPath: No flows added. >>>> Source and Destination ports are same. >>>> >> >>>> >> Is it ok? >>>> >> >>>> >> And this debug message(it has incorrect severity): >>>> >> >>>> >> 2016-10-14 22:43:55,282 | ERROR | entLoopGroup-5-3 | >>>> DeviceFlowRegistryImpl | 210 - >>>> org.opendaylight.openflowplugin.impl - 0.4.0.SNAPSHOT | Flow >>>> with flowId 85 already exists in table 0 >>>> >> >>>> >> So it looks like that flows with the same ID are added. >>>> >> >>>> >> What in short these failing tests are doing and what it >>>> should test please? >>>> >> >>>> >> Thanks, >>>> >> Miro >>>> >> >>>> >> ________________________________________ >>>> >> Od: Luis Gomez <ece...@gmail.com <mailto:ece...@gmail.com>> >>>> >> Odoslané: 15. októbra 2016 1:01 >>>> >> Komu: Tomáš Slušný >>>> >> Kópia: An Ho; openflowplugin-dev; OpenDayLight-L2switch-Dev >>>> >> Predmet: Re: [openflowplugin-dev] [L2switch-dev] Blocker >>>> bug analysis >>>> >> >>>> >> No luck :), still fails on "address tracker" and "loop >>>> remover": >>>> >> >>>> >> >>>> https://jenkins.opendaylight.org/releng/view/l2switch/job/l2switch-csit-1node-switch-only-carbon/154/ >>>> >>>> <https://jenkins.opendaylight.org/releng/view/l2switch/job/l2switch-csit-1node-switch-only-carbon/154/> >>>> >> >>>> >>>> https://logs.opendaylight.org/releng/jenkins092/l2switch-csit-1node-switch-only-carbon/154/archives/log.html.gz >>>> >>>> <https://logs.opendaylight.org/releng/jenkins092/l2switch-csit-1node-switch-only-carbon/154/archives/log.html.gz> >>>> >> >>>> >> BR/Luis >>>> >> >>>> >> >>>> >>> On Oct 14, 2016, at 2:26 PM, Luis Gomez <ece...@gmail.com >>>> <mailto:ece...@gmail.com>> wrote: >>>> >>> >>>> >>> OK Tomas, I will try your patch >>>> https://git.opendaylight.org/gerrit/#/c/46390/ >>>> <https://git.opendaylight.org/gerrit/#/c/46390/> on >>>> l2switch-csit-1node-switch-only-carbon and let you know. >>>> >>> >>>> >>> >>>> >>>> On Oct 14, 2016, at 1:23 PM, Tomáš Slušný >>>> <tomas.slu...@pantheon.tech >>>> <mailto:tomas.slu...@pantheon.tech>> wrote: >>>> >>>> >>>> >>>> Also, 6710 was already merged only in master branch, but >>>> not yet in boron. It have cherry-pick ready >>>> here: https://git.opendaylight.org/gerrit/#/c/46761/ >>>> <https://git.opendaylight.org/gerrit/#/c/46761/>, but >>>> until it is merged, I think it would be better to check for >>>> improvement in Jenkins test for Carbon here: >>>> >>>> https://jenkins.opendaylight.org/releng/view/l2switch/job/l2switch-csit-1node-switch-only-carbon/ >>>> >>>> <https://jenkins.opendaylight.org/releng/view/l2switch/job/l2switch-csit-1node-switch-only-carbon/> >>>> >>>> Od: Tomáš Slušný <tomas.slu...@pantheon.tech >>>> <mailto:tomas.slu...@pantheon.tech>> >>>> >>>> Odoslané: 14. októbra 2016 22:13 >>>> >>>> Komu: Luis Gomez; Abhijit Kumbhare >>>> >>>> Kópia: An Ho; openflowplugin-dev; >>>> OpenDayLight-L2switch-Dev >>>> >>>> Predmet: Re: [openflowplugin-dev] [L2switch-dev] Blocker >>>> bug analysis >>>> >>>> >>>> >>>> 6710 was related to both cluster and single node, >>>> because even in single node scenario, we are >>>> registering to ClusterSingletonService, what had problems when >>>> we tried to re-register, when we do not fully >>>> closed previous registration (so, when we disconnected and >>>> reconnected too fast). Most part of this fix was >>>> done in bug 6710, but part of it had to be done also in >>>> openflowplugin side in this gerrit patch: >>>> https://git.opendaylight.org/gerrit/#/c/46390/ >>>> <https://git.opendaylight.org/gerrit/#/c/46390/>. So, Luis, >>>> is it possible to run failing l2switch test on that patch and >>>> see, if it solves anything, or not? >>>> >>>> >>>> >>>> Od: Luis Gomez <ece...@gmail.com >>>> <mailto:ece...@gmail.com>> >>>> >>>> Odoslané: 14. októbra 2016 21:49 >>>> >>>> Komu: Abhijit Kumbhare >>>> >>>> Kópia: An Ho; openflowplugin-dev; >>>> OpenDayLight-L2switch-Dev >>>> >>>> Predmet: Re: [openflowplugin-dev] [L2switch-dev] Blocker >>>> bug analysis >>>> >>>> >>>> >>>> Well I think if openflowplugin people cannot figure out >>>> where the issue is, the next would be to get >>>> some debug and further analysis from l2switch people >>>> explaining the application miss-behavior. >>>> >>>> >>>> >>>> If there are no people available in l2switch to do this, >>>> we can lower the priority to major or critical. >>>> >>>> >>>> >>>> BR/Luis >>>> >>>> >>>> >>>> >>>> >>>>> On Oct 14, 2016, at 12:35 PM, Luis Gomez >>>> <ece...@gmail.com <mailto:ece...@gmail.com>> wrote: >>>> >>>>> >>>> >>>>> As far as I can tell nothing we have done so far has >>>> helped l2switch test, it is still failing. >>>> >>>>> >>>> >>>>> BR/Luis >>>> >>>>> >>>> >>>>>> On Oct 14, 2016, at 9:53 AM, Abhijit Kumbhare >>>> <abhijitk...@gmail.com <mailto:abhijitk...@gmail.com>> >>>> wrote: >>>> >>>>>> >>>> >>>>>> About the first one - >>>> https://bugs.opendaylight.org/show_bug.cgi?id=6575 >>>> <https://bugs.opendaylight.org/show_bug.cgi?id=6575>: >>>> >>>>>> >>>> >>>>>> Tomas has said in yesterday's meeting that this was >>>> dependent on >>>> https://bugs.opendaylight.org/show_bug.cgi?id=6710 >>>> <https://bugs.opendaylight.org/show_bug.cgi?id=6710>. And >>>> that it was fixed. Can we retest? Also a question for Tomas: >>>> 6710 is cluster related - while 6575 is not >>>> clear if it is cluster or single node. Are we sure 6575 is >>>> fixed? >>>> >>>>>> >>>> >>>>>> The second one: >>>> https://bugs.opendaylight.org/show_bug.cgi?id=6917 >>>> <https://bugs.opendaylight.org/show_bug.cgi?id=6917> - Shuva >>>> will respond about that some time. >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> On Thu, Oct 13, 2016 at 9:50 PM, Luis Gomez >>>> <ece...@gmail.com <mailto:ece...@gmail.com>> wrote: >>>> >>>>>> Hi all, >>>> >>>>>> >>>> >>>>>> We are getting close to Boron SR1 so I think it makes >>>> sense to review the 2 blocking issues we have: >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> 1) https://bugs.opendaylight.org/show_bug.cgi?id=6575 >>>> <https://bugs.opendaylight.org/show_bug.cgi?id=6575> >>>> >>>>>> >>>> >>>>>> Summary: >>>> >>>>>> >>>> >>>>>> l2switch does not work well when mininet is >>>> disconnected and connected with no time in-between. >>>> >>>>>> >>>> >>>>>> Description: >>>> >>>>>> >>>> >>>>>> This is kind of old issue, since the He->Li migration >>>> the l2switch has experienced random issues in >>>> the system test: >>>> >>>>>> >>>> >>>>>> >>>> https://jenkins.opendaylight.org/releng/view/l2switch/job/l2switch-csit-1node-switch-only-boron/ >>>> >>>> <https://jenkins.opendaylight.org/releng/view/l2switch/job/l2switch-csit-1node-switch-only-boron/> >>>> >>>>>> >>>> >>>>>> Same test passes fine in Beryllium as you can see below: >>>> >>>>>> >>>> >>>>>> >>>> https://jenkins.opendaylight.org/releng/view/l2switch/job/l2switch-csit-1node-switch-only-beryllium/ >>>> >>>> <https://jenkins.opendaylight.org/releng/view/l2switch/job/l2switch-csit-1node-switch-only-beryllium/> >>>> >>>>>> >>>> >>>>>> The last discovery (just before Boron release) was that >>>> giving more time between stop mininet and >>>> start mininet made the suite pass. >>>> >>>>>> >>>> >>>>>> Criticality: >>>> >>>>>> >>>> >>>>>> Although this was a clear regression in l2switch test >>>> (Be->B), this bug was not initially marked as >>>> blocker because it was not trivial to reproduce (e.g. switch >>>> connection flap). >>>> >>>>>> >>>> >>>>>> Risk of not fixing: >>>> >>>>>> >>>> >>>>>> l2switch and other similar applications relying on >>>> ofplugin may not work well when switch connection >>>> flaps. >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> 2) https://bugs.opendaylight.org/show_bug.cgi?id=6917 >>>> <https://bugs.opendaylight.org/show_bug.cgi?id=6917> >>>> >>>>>> >>>> >>>>>> Summary: >>>> >>>>>> >>>> >>>>>> Flow matching function (operational flow >>>> reconciliation) is not stable. >>>> >>>>>> >>>> >>>>>> Description: >>>> >>>>>> >>>> >>>>>> I discovered this issue doing some random flow push >>>> test in my laptop using POSTMAN: adding and >>>> deleting the same flow few times produced an alien ID in the >>>> operational flow. >>>> >>>>>> After that I have created a test that does exactly >>>> that: add flow, verify operational ID, delete flow, >>>> sleep 5s, repeat. With these simple steps the issue shows >>>> consistently for Boron (new test): >>>> >>>>>> >>>> >>>>>> >>>> >>>> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-boron/758/archives/log.html.gz >>>> >>>> <https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-boron/758/archives/log.html.gz> >>>> >>>>>> >>>> >>>>>> But not in Beryllium: >>>> >>>>>> >>>> >>>>>> >>>> >>>> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-beryllium/1854/archives/log.html.gz >>>> >>>> <https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-beryllium/1854/archives/log.html.gz> >>>> >>>>>> >>>> >>>>>> Criticality: >>>> >>>>>> >>>> >>>>>> Besides the test regression, I think there are >>>> applications in ODL relying on operational flow ID that >>>> would be negatively impacted by this bug. >>>> >>>>>> >>>> >>>>>> Risk of not fixing: >>>> >>>>>> >>>> >>>>>> OF applications relying on operational flow ID (e.g. to >>>> confirm flows) can sporadically fail. >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> _______________________________________________ >>>> >>>>>> L2switch-dev mailing list >>>> >>>>>> L2switch-dev@lists.opendaylight.org >>>> <mailto:L2switch-dev@lists.opendaylight.org> >>>> >>>>>> >>>> https://lists.opendaylight.org/mailman/listinfo/l2switch-dev >>>> <https://lists.opendaylight.org/mailman/listinfo/l2switch-dev> >>>> >>>>>> >>>> >>>>> >>>> >>>> >>>> >>>> TomášSlušný >>>> >>>> Software Developer >>>> >>>> >>>> >>>> Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia >>>> >>>> R&D centrum / Janka Kráľa 9 / 974 01 Banská Bystrica / >>>> Slovakia >>>> >>>> +421 911 083 902 <tel:%2B421%20911%20083%20902> / >>>> tomas.slu...@pantheon.tech >>>> <mailto:tomas.slu...@pantheon.tech> >>>> >>>> reception: +421 2 206 65 114 >>>> <tel:%2B421%202%20206%2065%20114> / www.pantheon.sk >>>> <http://www.pantheon.sk/> >>>> >>>> >>>> >>>> >>>> >>>> TomášSlušný >>>> >>>> Software Developer >>>> >>>> >>>> >>>> Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia >>>> >>>> R&D centrum / Janka Kráľa 9 / 974 01 Banská Bystrica / >>>> Slovakia >>>> >>>> +421 911 083 902 <tel:%2B421%20911%20083%20902> / >>>> tomas.slu...@pantheon.tech >>>> <mailto:tomas.slu...@pantheon.tech> >>>> >>>> reception: +421 2 206 65 114 >>>> <tel:%2B421%202%20206%2065%20114> / www.pantheon.sk >>>> <http://www.pantheon.sk/> >>>> >>>> >>>> >>>> >>>> >>> >>>> >> >>>> >> _______________________________________________ >>>> >> openflowplugin-dev mailing list >>>> >> openflowplugin-...@lists.opendaylight.org >>>> <mailto:openflowplugin-...@lists.opendaylight.org> >>>> >> >>>> https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev >>>> >>>> <https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev> >>>> >> MiroslavMacko >>>> >> Software Developer >>>> >> >>>> >> Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia >>>> >> R&D centrum / Janka Kráľa 9 / 974 01 Banská Bystrica / >>>> Slovakia >>>> >> / miroslav.ma...@pantheon.tech >>>> <mailto:miroslav.ma...@pantheon.tech> >>>> >> reception: +421 2 206 65 114 >>>> <tel:%2B421%202%20206%2065%20114> / www.pantheon.sk >>>> <http://www.pantheon.sk/> >>>> >> >>>> >> [logo] >>>> >> >>>> >> >>>> > >>>> >>>> _______________________________________________ >>>> L2switch-dev mailing list >>>> L2switch-dev@lists.opendaylight.org >>>> <mailto:L2switch-dev@lists.opendaylight.org> >>>> https://lists.opendaylight.org/mailman/listinfo/l2switch-dev >>>> <https://lists.opendaylight.org/mailman/listinfo/l2switch-dev> >>>> >>>> >>> >>> >> > > > > > _______________________________________________ > L2switch-dev mailing list > L2switch-dev@lists.opendaylight.org > https://lists.opendaylight.org/mailman/listinfo/l2switch-dev > _______________________________________________ L2switch-dev mailing list L2switch-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/l2switch-dev
