Hello Ilya, Thank you for your response and informing other developers.
Using "Clear Actions" instruction to reset the stack of actions, or trying to somehow reset the packet did not solve the issue. I think, it would be cumbersome and not very performant, to route the packet out and then back in into the OvS to do append the GTP header and send it to the destination. We could as an alternative send the packet to another datapath after the encapsulation, to somehow force a clean packet, but even then, we need to append the GTP Decapsulation with ETH header pushing. Also, we need some virtual interfaces, and in case of DPDK, they also need to be evaluated for performance. At the end, we use the Userspace for performance. I would really appreciate, if I could get a relatively fast response here, since I really need to fix this very urgently. What would be the safest solution here to quickly implement? If you guys could tell, for example where the best place is, for the exclusion for the additional ETH encapsulation in case of the GTP, that would save my life, until there is a proper solution. Regards, Dincer Mit freundlichen Grüßen Dincer Beken E-Mail: [email protected]<mailto:[email protected]> Tel.: +49 8331 9959-852 Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und löschen Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet. This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the contents in this e-mail is strictly forbidden. Dincer Beken Software Developer t: +49 8331 99 59-852<tel:+49%208331%2099%2059-852> e: [email protected]<mailto:[email protected]> w: https://blackned.de [cid:blackned_d1650eb0-ca34-4593-90da-c73d0e84f2cd.png] [cid:linkedin_d8fe2332-efbd-461a-923a-3153732a489e.png]<https://www.linkedin.com/company/8960382> blackned GmbH · zugspitzstraße 1 · 87751 heimertingen geschäftsführer: timo haas · hrb memmingen 17319 Diese eMail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese eMail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und löschen Sie diese eMail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser eMail ist nicht gestattet. This eMail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this eMail in error) please notify the sender immediately and delete this eMail. Any unauthorized copying, disclosure or distribution of the contents in this eMail is strictly forbidden. ________________________________ Von: Ilya Maximets <[email protected]> Gesendet: Dienstag, 29. März 2022 11:59 An: Dincer Beken <[email protected]>; [email protected] <[email protected]>; Jan Scheurich <[email protected]> Cc: [email protected] <[email protected]>; William Tu <[email protected]> Betreff: Re: [ovs-dev] [EXTERNAL] Proper way to recirculate packet // OvS Crash On 3/29/22 11:39, Dincer Beken wrote: > Hello again, > > > > If the packet is recirculated back, and if it is a stripped-off L3 packet > (GTP decapsulation), OvS does not recognize it as a former GTP packet and > appends add the pending encapsulation flag on top of it. But the must be > encapsulated with GTP first, and the ETH header will be appended on top. > > Setting the following in "xlate_actions" > // ctx.pending_encap = true; > > fixes the issue (of course just for this case, this cannot be a general > solution). > > But can it be, that the case of GTP recirculation is just not considered, and > this condition of adding the L2 encapsulation has to be extended to check for > GTP like tunnels? > > The crash occurs during commit_encap_decap_action -> when the packet type is > not recognized (after the GTPU decap) > > If i did something wrong, I would kindly also ask for your advice. The crash is definitely is not an expected outcome of any scenario, so that should not happen and has to be fixed. I'm not very familiar with the L3 support in OVS, so I can't say what is going on from the top of my head, unfortunately. Jan, you worked on implementation of L3 tunnels, do you have some thoughs? On a quick glance, I'd say that OVS may not handle pop and subsequent push in the same list of actions correctly for L3 tunnels. And we defenitly don't have a relevant unit test for that, so I'd not exclude the possibility that such scenario might be just broken, since it was never tested before... P.S. The likely reason why you don't have a coredump is that the process terminates with ABRT in OVS_NOT_REACHED() inside the commit_encap_decap_action(). It doesn't really crash. Just a guess though. Best regards, Ilya Maximets. > > Really interested in your feedback. > > Best Regard, > Dincer > > Dincer Beken > Software Developer > > t: +49 8331 99 59-852<tel:+49%208331%2099%2059-852> > > e: [email protected]<mailto:[email protected]> > w: https://blackned.de > > > > [cid:blackned_d1650eb0-ca34-4593-90da-c73d0e84f2cd.png] > > [cid:linkedin_d8fe2332-efbd-461a-923a-3153732a489e.png]<https://www.linkedin.com/company/8960382> > > > blackned GmbH · zugspitzstraße 1 · 87751 heimertingen > geschäftsführer: timo haas · hrb memmingen 17319 > > Diese eMail enthält vertrauliche und/oder rechtlich geschützte Informationen. > Wenn Sie nicht der richtige Adressat sind oder diese eMail irrtümlich > erhalten haben, informieren Sie bitte sofort den Absender und löschen Sie > diese eMail. > Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser eMail ist nicht > gestattet. > > This eMail may contain confidential and/or privileged information. > If you are not the intended recipient (or have received this eMail in error) > please notify the sender immediately and delete this eMail. > Any unauthorized copying, disclosure or distribution of the contents in this > eMail is strictly forbidden. > > ________________________________ > Von: dev <[email protected]> im Auftrag von Dincer Beken > <[email protected]> > Gesendet: Dienstag, 29. März 2022 04:09 > An: [email protected] <[email protected]> > Betreff: Re: [ovs-dev] [EXTERNAL] Proper way to recirculate packet // OvS > Crash > > Checking further into the code, I see that even if I put the ETH header in > front, the packet is not matched against a gtp-vport, since the packet will > be a layer2 packet. Even if I hardcodedly fetch a gtp vport, in tnl_find or > tnl_find_exact, the rule of pushing back the GTP header will not be applied. > > Could you please confirm this? > > Regards, > Dincer > > Dincer Beken > Software Developer > > t: +49 8331 99 59-852<tel:+49%208331%2099%2059-852> > > e: [email protected]<mailto:[email protected]> > w: https://blackned.de > > > > [cid:blackned_d1650eb0-ca34-4593-90da-c73d0e84f2cd.png] > > [cid:linkedin_d8fe2332-efbd-461a-923a-3153732a489e.png]<https://www.linkedin.com/company/8960382> > > > blackned GmbH · zugspitzstraße 1 · 87751 heimertingen > geschäftsführer: timo haas · hrb memmingen 17319 > > Diese eMail enthält vertrauliche und/oder rechtlich geschützte Informationen. > Wenn Sie nicht der richtige Adressat sind oder diese eMail irrtümlich > erhalten haben, informieren Sie bitte sofort den Absender und löschen Sie > diese eMail. > Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser eMail ist nicht > gestattet. > > This eMail may contain confidential and/or privileged information. > If you are not the intended recipient (or have received this eMail in error) > please notify the sender immediately and delete this eMail. > Any unauthorized copying, disclosure or distribution of the contents in this > eMail is strictly forbidden. > > ________________________________ > Von: dev <[email protected]> im Auftrag von Dincer Beken > <[email protected]> > Gesendet: Dienstag, 29. März 2022 03:12 > An: [email protected] <[email protected]> > Betreff: Re: [ovs-dev] [EXTERNAL] Proper way to recirculate packet // OvS > Crash > > https://mail.openvswitch.org/pipermail/ovs-dev/2017-May/332897.html > > https://www.youtube.com/watch?v=rOp1tUGIbYs > > So essentially, this is what I am doing during the decapsulation, such that > the OvS does not crash. > > An additional question would be, if I need to specify an Input VPORt, as well? > > > > Dincer Beken > Software Developer > > t: +49 8331 99 59-852<tel:+49%208331%2099%2059-852> > > e: [email protected]<mailto:[email protected]> > w: https://blackned.de > > > > [cid:blackned_d1650eb0-ca34-4593-90da-c73d0e84f2cd.png] > > [cid:linkedin_d8fe2332-efbd-461a-923a-3153732a489e.png]<https://www.linkedin.com/company/8960382> > > > blackned GmbH · zugspitzstraße 1 · 87751 heimertingen > geschäftsführer: timo haas · hrb memmingen 17319 > > Diese eMail enthält vertrauliche und/oder rechtlich geschützte Informationen. > Wenn Sie nicht der richtige Adressat sind oder diese eMail irrtümlich > erhalten haben, informieren Sie bitte sofort den Absender und löschen Sie > diese eMail. > Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser eMail ist nicht > gestattet. > > This eMail may contain confidential and/or privileged information. > If you are not the intended recipient (or have received this eMail in error) > please notify the sender immediately and delete this eMail. > Any unauthorized copying, disclosure or distribution of the contents in this > eMail is strictly forbidden. > > ________________________________ > Von: dev <[email protected]> im Auftrag von Dincer Beken > <[email protected]> > Gesendet: Dienstag, 29. März 2022 02:45 > An: [email protected] <[email protected]> > Betreff: Re: [ovs-dev] [EXTERNAL] Proper way to recirculate packet // OvS > Crash > > Hello, > > After looking further into the issue, after the decapsulation, the packet is > a L3 packet without a L2 header anymore. If I push the ETH header manually, > the received port cannot be matched anymore. > > Do we have to push the ETH header (some random dummy), onto the decapsulated > packet, before we encapsulate it again? > > Thank you, > Dincer > > Dincer Beken > Software Developer > > t: +49 8331 99 59-852<tel:+49%208331%2099%2059-852> > > e: [email protected]<mailto:[email protected]> > w: https://blackned.de > > > > [cid:blackned_d1650eb0-ca34-4593-90da-c73d0e84f2cd.png] > > [cid:linkedin_d8fe2332-efbd-461a-923a-3153732a489e.png]<https://www.linkedin.com/company/8960382> > > > blackned GmbH · zugspitzstraße 1 · 87751 heimertingen > geschäftsführer: timo haas · hrb memmingen 17319 > > Diese eMail enthält vertrauliche und/oder rechtlich geschützte Informationen. > Wenn Sie nicht der richtige Adressat sind oder diese eMail irrtümlich > erhalten haben, informieren Sie bitte sofort den Absender und löschen Sie > diese eMail. > Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser eMail ist nicht > gestattet. > > This eMail may contain confidential and/or privileged information. > If you are not the intended recipient (or have received this eMail in error) > please notify the sender immediately and delete this eMail. > Any unauthorized copying, disclosure or distribution of the contents in this > eMail is strictly forbidden. > > ________________________________ > Von: dev <[email protected]> im Auftrag von Dincer Beken > <[email protected]> > Gesendet: Dienstag, 29. März 2022 00:57 > An: [email protected] <[email protected]> > Betreff: [EXTERNAL] [ovs-dev] Proper way to recirculate packet // OvS Crash > > Hello, > > I am using the userspace OvS (2.14.) and the userspace GTP tunnel. > I am decapsulating a GTP packet in table0 -> sending the decapsulated packet > over to Table1, where I again want to encapsulate it again (with a different > tunnel identifier) and submit to (another?) gtp port. > > root@RiduxV4:~/Documents# ovs-vsctl show > 3271f3b8-4aa6-40a2-af9b-37353238b104 > Bridge dp2 > Controller "tcp:127.0.0.1" > is_connected: true > datapath_type: netdev > Port gtpu8 > Interface gtpu8 > type: gtpu > options: {dst_port="2152", key=flow, remote_ip=flow} > Port dp2 > Interface dp2 > type: internal > Port gtpu9 > Interface gtpu9 > type: gtpu > options: {dst_port="2152", key="0x00", > remote_ip="192.168.177.101"} > Port ethvlan1000 > Interface ethvlan1000 > type: internal > > root@RiduxV4:~/Documents# ovs-ofctl dump-flows dp2 > cookie=0x1234, duration=277.786s, table=0, n_packets=30, n_bytes=2314, > priority=1250,ip,in_port=eth1 actions=output:ethvlan1000 > > cookie=0x1234, duration=277.736s, table=0, n_packets=1, n_bytes=34, > priority=1400,ip,in_port=gtpu8 > actions=mod_dl_dst:99:99:99:99:99:99,resubmit(,1) > > cookie=0x1234, duration=274.699s, table=1, n_packets=1, n_bytes=34, > priority=1100,ip,in_port=gtpu8,nw_dst=192.168.10.3 > actions=load:0x55667701->NXM_NX_TUN_ID[],load:0xc0a8b164->NXM_NX_TUN_IPV4_DST[],output:gtpu8 > > > So to repeat, the packet got decapsulated in table 0 and then moved over to > table1 where it should be encapsulated again. > > Encapsulation and Decapsulation work, only encapsulating a decapsulated > packet, without leaving OvS does not work in Userspace. > > If I select the output interface as the intput interface, OvS recognize this > and does not allow it, so far so good. > > But if I select the IN_PORt or another GTP tunnel (gtpu9), OvS crashes. I > could not generate a core dump, yet. > > compose_output_action__ -> xlate_commit_actions(ctx); > > Is there a possiblity to implement this in the userspace? Is there a way, to > clear the actions stack, and send a fresh decapsulated packet in the > recirculation, such that OvS does not recognize that it was encapsulated > before? > > Looking forward for your support, > > Dincer > > Dincer Beken > Software Developer > > t: +49 8331 99 59-852<tel:+49%208331%2099%2059-852> > > e: [email protected]<mailto:[email protected]> > w: https://blackned.de > > > > [cid:blackned_d1650eb0-ca34-4593-90da-c73d0e84f2cd.png] > > [cid:linkedin_d8fe2332-efbd-461a-923a-3153732a489e.png]<https://www.linkedin.com/company/8960382> > > > blackned GmbH · zugspitzstraße 1 · 87751 heimertingen > geschäftsführer: timo haas · hrb memmingen 17319 > > Diese eMail enthält vertrauliche und/oder rechtlich geschützte Informationen. > Wenn Sie nicht der richtige Adressat sind oder diese eMail irrtümlich > erhalten haben, informieren Sie bitte sofort den Absender und löschen Sie > diese eMail. > Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser eMail ist nicht > gestattet. > > This eMail may contain confidential and/or privileged information. > If you are not the intended recipient (or have received this eMail in error) > please notify the sender immediately and delete this eMail. > Any unauthorized copying, disclosure or distribution of the contents in this > eMail is strictly forbidden. > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
