On Thu, Jul 12, 2018 at 8:26 PM, Sam Hague <sha...@redhat.com> wrote:
> > > On Thu, Jul 12, 2018 at 10:04 AM Aswin Suryanarayanan <asury...@redhat.com> > wrote: > >> >> >> On Thu, Jul 12, 2018 at 8:20 AM, D Arunprakash < >> d.arunprak...@ericsson.com> wrote: >> >>> Aswin, >>> >>> Could you please refer the attached packet-in dump, where there are new >>> ct fields (120, 121, 124, 125) in the match? >>> >>> >>> >>> So, if we use ct_clear in the flow, is it expected to clear the new ct >>> fields as well. I’m not much familiar with CT fields, could you please >>> check and let me know what we should do now? >>> >> >> ct_clear is expected to clear these , but it is more like workaround. Was >> this added by aclservice, does disabling port security solve the issue? if >> it helps ,I doubt if we have bug in ovs and that is the reason we are >> seeing these fields as aclservice is not using it. >> >>> >>> >>> Is the ct_clear bug present in ovs2.8 as well? >>> >> >> ct_clear works only in ovs2.9+. We are planning to move to ovs2.9 right? >> > We need to come up with a plan since we don't have anything beyond 2.7.2 > that works: 2.8.2 has issues and 2.9.0 has a group bug. So we need at least > 2.9.2 which we have been waiting for to get an rpm with patch [10]. So we > were just trying to find some way to test with something so we could get > some visibility. If we think 2.9.2 will fix everything then we should push > to get that in, but we also need the ofp nsh patches right? > if these headers were introduced by aclservice(I think yes, as we don't use anything related to NSH here), we do not require the nsh patches for fixing this. ct_clear should be removing them. > > [10] https://review.rdoproject.org/r/#/c/13839/ > >> >>> >>> >>> >>> Regards, >>> >>> Arun >>> >>> *From:* Aswin Suryanarayanan [mailto:asury...@redhat.com] >>> *Sent:* Wednesday, July 11, 2018 11:24 AM >>> *To:* D Arunprakash <d.arunprak...@ericsson.com> >>> *Cc:* Anil Vishnoi <vishnoia...@gmail.com>; Jamo Luhrsen < >>> jluhr...@redhat.com>; openflowplugin-dev <openflowplugin-dev@lists. >>> opendaylight.org>; Luis Gomez <ece...@gmail.com>; odl netvirt dev < >>> netvirt-...@lists.opendaylight.org> >>> >>> *Subject:* Re: [netvirt-dev] ovs 2.8.2 does not work well in csit >>> >>> >>> >>> >>> >>> >>> >>> On Wed, Jul 11, 2018 at 10:17 AM, D Arunprakash < >>> d.arunprak...@ericsson.com> wrote: >>> >>> Aswin, >>> >>> >>> >>> From http://www.openvswitch.org/support/dist-docs/ovs-ofctl.8.html, >>> ct_clear clears the following ct fields, >>> >>> >>> >>> ct_clear >>> >>> Clears connection tracking state from the flow, >>> zeroing >>> >>> ct_state, ct_zone, ct_mark, and ct_label. >>> >>> >>> >>> This action was introduced in Open vSwitch 2.6.90. >>> >>> >>> >>> But from the ovs2.8+, there are new CT fields introduced, will ct_clear >>> those as well? >>> >>> >>> >>> The ct_clear is intended to clear the states, I don't think (I could be >>> wrong) the other fields will be present in a packet submitted back from to >>> the pipeline from conntrack, they are used when we send the packet to >>> conntrack. >>> >>> In this case I think we need to identify who added these extra fields, >>> if it was due to acl service it should have been cleared by ct_clear. But >>> the ct_clear functionality was broken in the kernel datapath in the initial >>> ovs2.9 release and was addressed by [*]. We should have this fix for this >>> to work as expected. >>> >>> >>> [*]https://patchwork.ozlabs.org/patch/864353/ >>> >>> >>> >>> CONNECTION TRACKING FIELDS >>> >>> Summary: >>> >>> Name Bytes Mask RW? Prereqs NXM/OXM Support >>> >>> ──────────── ────── ───── ──── ──────── ──────────────── >>> >>> ct_state 4 yes no none OVS 2.5+ >>> >>> ct_zone 2 no no none OVS 2.5+ >>> >>> ct_mark 4 yes yes none OVS 2.5+ >>> >>> ct_label 16 yes yes none OVS 2.5+ >>> >>> ct_nw_src 4 yes no CT OVS 2.8+ >>> >>> ct_nw_dst 4 yes no CT OVS 2.8+ >>> >>> >>> >>> ct_ipv6_src 16 yes no CT OVS 2.8+ >>> >>> ct_ipv6_dst 16 yes no CT OVS 2.8+ >>> >>> ct_nw_proto 1 no no CT OVS 2.8+ >>> >>> ct_tp_src 2 yes no CT OVS 2.8+ >>> >>> ct_tp_dst 2 yes no CT OVS 2.8+ >>> >>> >>> >>> >>> >>> Regards, >>> >>> Arun >>> >>> >>> >>> *From:* netvirt-dev-boun...@lists.opendaylight.org [mailto: >>> netvirt-dev-boun...@lists.opendaylight.org] *On Behalf Of *Aswin >>> Suryanarayanan >>> *Sent:* Tuesday, July 10, 2018 11:06 PM >>> *To:* Anil Vishnoi <vishnoia...@gmail.com> >>> *Cc:* Jamo Luhrsen <jluhr...@redhat.com>; openflowplugin-dev < >>> openflowplugin-dev@lists.opendaylight.org>; Luis Gomez <ece...@gmail.com>; >>> odl netvirt dev <netvirt-...@lists.opendaylight.org> >>> >>> >>> *Subject:* Re: [netvirt-dev] ovs 2.8.2 does not work well in csit >>> >>> >>> >>> >>> >>> >>> >>> On Mon, Jul 9, 2018 at 11:25 PM, Anil Vishnoi <vishnoia...@gmail.com> >>> wrote: >>> >>> Sam, >>> >>> >>> >>> We are currently working on getting all the NSH related patches from >>> jamie in, once we get those patches in, we can see if we see these other >>> serialization issue. >>> >>> >>> >>> On Mon, Jul 9, 2018 at 8:24 AM Sam Hague <sha...@redhat.com> wrote: >>> >>> Arun, Anil, >>> >>> >>> >>> thanks for merging [1]. There are other deserialization problems failing >>> the tests. I filed [3] to track the issues. Can you take a look and see if >>> more similar changes are needed? The conntrack fields overlapping with the >>> nsh fields definitely makes sense as the tests are conntrack based. >>> >>> >>> >>> Arun, Jaime, >>> >>> >>> >>> did you also try with ovs 2.9.x? I am guessing the same problems are >>> there. >>> >>> >>> >>> Thanks, Sam >>> >>> >>> >>> [1] https://git.opendaylight.org/gerrit/#/c/73735/ >>> >>> [2] https://git.opendaylight.org/gerrit/#/c/72320/. >>> >>> [3] https://jira.opendaylight.org/browse/OPNFLWPLUG-1023 >>> >>> [4] https://logs.opendaylight.org/releng/vex-yul-odl- >>> jenkins-1/builder-copy-sandbox-logs/196/shague- >>> queens-netvirt-csit-1node-openstack-queens-upstream- >>> stateful-fluorine/7/odl_1/odl1_karaf.log.gz >>> >>> >>> >>> On Mon, Jul 9, 2018 at 4:09 AM Jaime Caamaño Ruiz <jcaam...@suse.de> >>> wrote: >>> >>> Hello Sam >>> >>> This is most likely happenning because since OVS 2.8, openflow OXM/NXM >>> ids that were unofficially used for the NSH fields on Yi Yang patch are >>> now being officially used for other different fields. >>> >>> On this specific case, it is OXM field 120 in the Nicira NXM1 class, >>> which Yi Yang used for NshNp, but is now officially used for ct_nw_src. >>> So openflowplugin is wrongly interpreting ct_nw_src as NshNp. >>> >>> BR >>> Jaime. >>> >>> >>> >>> -----Original Message----- >>> From: Sam Hague <sha...@redhat.com> >>> To: Anil Vishnoi <vishnoia...@gmail.com> >>> Cc: Dayavanti Gopal Kamath <dayavanti.gopal.kam...@ericsson.com>, Luis >>> Gomez <ece...@gmail.com>, HCL Tech Venkatrangan G - ERS <venkatrangang@ >>> hcl.com>, odl netvirt dev <netvirt-...@lists.opendaylight.org>, >>> openflowplugin-dev <openflowplugin-dev@lists.opendaylight.org>, Jamo >>> Luhrsen <jluhr...@redhat.com>, Manuel Buil <mb...@suse.com>, jcaamano@s >>> use.de >>> Subject: Re: [netvirt-dev] ovs 2.8.2 does not work well in csit >>> Date: Sun, 8 Jul 2018 22:44:30 -0400 >>> >>> >>> >>> On Sun, Jul 8, 2018 at 5:05 PM Anil Vishnoi <vishnoia...@gmail.com> >>> wrote: >>> > I believe upstream openflowplugin CSIT uses 2.8.0/1 version and that >>> > did not contain the new NSH support and that's working fine. Seems >>> > like the new NSH support is causing this issue. Looking like in your >>> > CSIT someone is trying to install the flow with the NSH match or your >>> > switch has NSH based flow. MatchConvertor comes into play when either >>> > you are pushing flow down to the switch or reading and deserializing >>> > the stats/packet_in coming from switch. >>> > >>> > >>> >>> Anil, the exception is coming from packet_in messages and not a flow >>> being installed. The features for sfc/nsh are not installed so pretty >>> sure nothing is trying to use nsh. Can you look again at the two >>> exceptions to see if that is correct? OVS 2.8.0 added userspace NSH >>> support, with more support added in 2.8.1 and 2.8.2. I see the ofp csit >>> is using 2.8.1. it has deserialization errors but not the same as the >>> netvirt csit. ovs 2.8.2 says it moved the NSH support to the standard >>> spec support so maybe that is why it is causing more problems that >>> 2.8.1? >>> >>> >>> >>> There was a similar issue [1] fixed a while back in netvirt which was >>> observed with Ovs2.9. When the packet was send to netfilter by acl >>> service , the return packet which was submitted back to the pipeline. had >>> many extra headers. When this was send to the controller for resolving a >>> PNF , the OF plugin failed to parse these including the NSH headers. This >>> was addressed by calling ct_clear which removed all the extra headers >>> added. Since we are not using NSH/SFC these might have got added due to a >>> similar reason. If that is the case I think we should be checking if ovs is >>> doing this intentionally. >>> >>> [1]https://git.opendaylight.org/gerrit/#/c/69686/ >>> >>> >>> > We already have patches pushed by Jamie to implement these new NSH >>> > fields and remove the old one, hopefully that should fix it. >>> > >>> >>> How far along are these patches? >>> > On Sun, Jul 8, 2018 at 5:26 AM Sam Hague <sha...@redhat.com> wrote: >>> > > Adding Daya to see if they have been doing any testing with ovs >>> > > 2.8.2 or later. Also forgot netvirt-dev and openflowplugin-dev >>> > > lists. >>> > > >>> > > On Sat, Jul 7, 2018 at 5:05 PM Sam Hague <sha...@redhat.com> wrote: >>> > > > Jamo, >>> > > > >>> > > > did we ever have any runs using ovs 2.8.2 - or really anything >>> > > > later than 2.7.2? [1] is a job using ovs 2.8.2 and it fails a few >>> > > > tests because of an openflow deserialization error with some NSH >>> > > > packets - but this is just our normal csit and not anything >>> > > > related to sfc. My 2.9 jobs also had some failures but I didn't >>> > > > inspect them before the sandbox cleared up this weekend. >>> > > > >>> > > > NSH support was added in 2.8 so that makes sense this could cause >>> > > > a problem. The exception below is coming out when the test fails >>> > > > - a ping fails, so I guess an arp or other punt packet is being >>> > > > mapped to nsh and not being decoded by openflowplugin. >>> > > > >>> > > > Jaime, Venkat, >>> > > > >>> > > > have you seen issues like this? Is it because openflowplugin has >>> > > > not been updated to use the new nsh and maybe some older nsh >>> > > > support is causing problems? >>> > > > >>> > > > Thanks, Sam >>> > > > >>> > > > [1] https://jenkins.opendaylight.org/sandbox/job/shague-queens-ne >>> > > > tvirt-csit-1node-openstack-queens-upstream-stateful-fluorine/3/ >>> > > > >>> > > > 2018-07-07T15:28:56,461 | WARN | epollEventLoopGroup-9-1 | >>> > > > MatchExtensionHelper | 364 - >>> > > > org.opendaylight.openflowplugin - 0.7.0.SNAPSHOT | Convertor for >>> > > > MatchEntry{_matchEntryValue=NshNpCaseValue{_nshNpValues=NshNpValu >>> > > > es{_value=41, augmentation=[]}, augmentation=[]}, >>> > > > _oxmClass=interface >>> > > > org.opendaylight.yang.gen.v1.urn.opendaylight.openflow.oxm.rev150 >>> > > > 225.Nxm1Class, _oxmMatchField=interface >>> > > > org.opendaylight.yang.gen.v1.urn.opendaylight.openflowjava.nx.mat >>> > > > ch.rev140421.NxmNxNshNp, _hasMask=false, augmentation=[]} for >>> > > > version 4 with match path PACKET_IN_MESSAGE_MATCH not found. >>> > > > 2018-07-07T15:28:56,462 | WARN | epollEventLoopGroup-9-1 | >>> > > > OFDecoder | 386 - >>> > > > org.opendaylight.openflowplugin.openflowjava.openflow-protocol- >>> > > > impl - 0.7.0.SNAPSHOT | Message deserialization >>> > > > failedjava.lang.NullPointerException: null at >>> > > > org.opendaylight.openflowplugin.openflow.md.core.extension.MatchE >>> > > > xtensionHelper.injectExtension(MatchExtensionHelper.java:83) >>> > > > [364:org.opendaylight.openflowplugin:0.7.0.SNAPSHOT] at >>> > > > org.opendaylight.openflowplugin.impl.protocol.deserialization.mat >>> > > > ch.MatchDeserializer.deserializeEntry(MatchDeserializer.java:104) >>> > > > >>> > > > >>> > > >>> > > _______________________________________________ >>> > > netvirt-dev mailing list >>> > > netvirt-...@lists.opendaylight.org >>> > > https://lists.opendaylight.org/mailman/listinfo/netvirt-dev >>> > >>> > >>> >>> >>> >>> >>> -- >>> >>> Thanks >>> >>> Anil >>> >>> >>> _______________________________________________ >>> netvirt-dev mailing list >>> netvirt-...@lists.opendaylight.org >>> https://lists.opendaylight.org/mailman/listinfo/netvirt-dev >>> >>> >>> >>> >>> >> >> _______________________________________________ >> openflowplugin-dev mailing list >> openflowplugin-dev@lists.opendaylight.org >> https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev >> >
_______________________________________________ openflowplugin-dev mailing list openflowplugin-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
