[ovs-discuss] Need help with adding custom IPv6 extension header match and tag support
Hello, I need to add a custom feature to OVS for my research where it can match on a custom IPv6 extension header, and be able to perform the action of appending a datagram with a custom IPv6 extension header. I cloned the latest version and followed https://github.com/ openvswitch/ovs/blob/7169d4fdc8e564331c7a5ad910bc1d 99cb413712/FAQ.rst#development however couldn't locate the struct flow in lib/flow.h. The file lib/meta-flow.h does not exist either. I'd appreciate if you could point me in the right direction. And btw, is there some logging function which I can call anywhere to print to a log file? For instance, calling log(packet->md) in miniflow_extract so that the metadata of each passed packet is printed in some log file live. That would help big time understanding the code. Regards, Alan ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Need help with adding custom IPv6 extension header match and tag support
I did not try to look for the file elsewhere. If it is located somewhere other than wherever it is stated to be, I automatically assume a possible change in its functionality/purpose and ask for confirmation, especially when I am new to it. Here are some "useful" answers I came up with after spending some time with the code, hopefully would help others. 1- The documentation in the code is not as up to date as it is in openvswitch.org. Refer to FAQ->Development in openvswitch.org instead. 2- VLOG function can be used anywhere in the userspace code to print to ovs-vswitch.log file. If you did not explicitly set a path for it during compliation, default is /var/log/openvswitch or /usr/var/log/openvswitch. Remember to see the current logging settings using the "ovs-appctl vlog/list" command and change to desired settings using the "ovs-appctl vlog/set" command. Use printk() in kernel space code, output can be found in syslog/dmesg. Alan 2017-12-29 18:16 GMT+01:00 Gregory Rose <gvrose8...@gmail.com>: > On 12/19/2017 6:16 AM, Alan Kayahan wrote: > > Hello, > > I need to add a custom feature to OVS for my research where it can match > on a custom IPv6 extension header, and be able to perform the action of > appending a datagram with a custom IPv6 extension header. > > I cloned the latest version and followed https://github.com/op > envswitch/ovs/blob/7169d4fdc8e564331c7a5ad910bc1d99cb413712/ > FAQ.rst#development however couldn't locate the struct flow in > lib/flow.h. The file lib/meta-flow.h does not exist either. > > I'd appreciate if you could point me in the right direction. And btw, is > there some logging function which I can call anywhere to print to a log > file? For instance, calling log(packet->md) in miniflow_extract so that the > metadata of each passed packet is printed in some log file live. That would > help big time understanding the code. > > Regards, > Alan > > > The Linux "find" utility works great for finding files. > > $ find . -name flow.h > ./datapath/flow.h > ./include/openvswitch/flow.h > ./lib/flow.h > > $find . -name meta-flow.h > ./include/openvswitch/meta-flow.h > > - Greg > > > > ___ > discuss mailing > listdiscuss@openvswitch.orghttps://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] To which functions is the first ever incoming packet passed to?
Hello I am in the process of understanding the OVS code following the information given by David in this thread https://mail.openvswitch.org/pipermail/ovs- discuss/2016-March/040236.html So I have the following simple setup where namespaces C1 and C2 are connected to SW1. ip netns add C1 ip netns add C2 ip link add C1-eth0 type veth peer name SW1-C1 ip link add C2-eth0 type veth peer name SW1-C2 ip link set SW1-C1 up ip link set SW1-C2 up ip link set C1-eth0 netns C1 ip link set C2-eth0 netns C2 ip netns exec C1 ifconfig C1-eth0 10.50.10.1/24 up ip netns exec C2 ifconfig C2-eth0 10.50.10.2/24 up ovs-ctl start ovs-vsctl add-br SW1 ovs-vsctl add-port SW1 SW1-C1 ovs-vsctl add-port SW1 SW1-C2 There are no controllers, no flow rules added, and they ping each other just fine. I inserted a log function call in the ovs_dp_process_packet() in datapath.c so every time it is called I should get a message printed to a file. However nothing is printed even though the pings work. My expectation is that the first echo request should enter the if(unlikely(!flow)) clause in ovs_dp_process_packet() and trigger an upcall. What am I doing wrong? Thanks, Alan ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Lost in translation! OFP to DPDK datapath OVS actions?
Hello, Per my understanding from /ovs-discuss/2015-May/037313.html, ofproto-dpif-xlate translates OFP actions into Netlink format, which is useful for datapath actions implemented in the kernel module. Does that hold for OFP to DPDK datapath actions too? I couldn't find a connection from ofproto-dpif-xlate to odp_execute_actions. Where does the translation of ofpact structs into OVS actions implemented in DPDK datapath take place? Alan ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Lost in translation! OFP to DPDK datapath OVS actions?
Indeed. My custom action works when I put nl_msg_put_flag(ctx->odp_actions, OVS_ACTION_ATTR_MY_ACTION); however none of the other cases follow this pattern. They all set flow->xxx from the struct casted ofpact instead. Since both setting ctx->odp_actions and flow->xxx works, I have the following questions 1) Where do the actions defined in ctx->odp_actions get actuated? 2) Where do the actions for the struct flow get actuated, and how does the actuator know what OVS actions to actuate? 3) Can we say that do_xlate_actions() is currently being used outside the purpose that its name implies, because all it does is to set the fields of the struct flow to those set in ofpact struct? Thanks 2018-01-12 18:36 GMT+01:00 Ben Pfaff <b...@ovn.org>: > On Fri, Jan 12, 2018 at 11:39:44AM +0100, Alan Kayahan wrote: > > Hello, > > > > Per my understanding from /ovs-discuss/2015-May/037313.html, > > ofproto-dpif-xlate translates OFP actions into Netlink format, which is > > useful for datapath actions implemented in the kernel module. Does that > > hold for OFP to DPDK datapath actions too? > > Yes. > > > I couldn't find a connection from ofproto-dpif-xlate to > > odp_execute_actions. Where does the translation of ofpact structs into > > OVS actions implemented in DPDK datapath take place? > > ofproto-dpif-xlate does the translation for both datapaths, in the same > way. > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Is a new field in flow.h necessary for matching on a custom v6 extension header?
Hello, I have a custom v6 extension header, in which I would like to perform an LPM match on only one 32bit field. Since an extension header is technically not a new field, do I still need to introduce anything in the struct flow? In miniflow_extract(), what I did was to add a case in parse_ipv6_ext_hdrs__ as (*nw_proto == IPPROTO_TRH) where IPPROTO_TRH is my custom proto number, and parse the *datap to my struct trh_hdr, then data_try_pull. Is this the appropriate approach? If yes, what would the appropriate way be for accessing the 32bit field in trh_hdr to perform the match? Thanks, Alan ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Is a new field in flow.h necessary for matching on a custom v6 extension header?
I understand that there must be a construct in the struct flow, and introducing a new be32 will save the day for me. However looking at this from a broader perspective, say matching on the fields of any v6 extension header made into the OF specification, neither adding individual fields nor adding every v6 extension header into the struct flow looks elegant to me. What do you think? 2018-02-21 17:10 GMT+01:00 Ben Pfaff <b...@ovn.org>: > On Mon, Feb 19, 2018 at 06:15:30PM +0100, Alan Kayahan wrote: > > I have a custom v6 extension header, in which I would like to perform an > > LPM match on only one 32bit field. > > > > Since an extension header is technically not a new field, do I still need > > to introduce anything in the struct flow? > > Without anything in struct flow, how do you propose to match on it? > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] To which functions is the first ever incoming packet passed to?
Turned out I wasn't properly installing the kernel modules I modified. 2017-12-21 23:39 GMT+01:00 Alan Kayahan <hsy...@gmail.com>: > Hello > > I am in the process of understanding the OVS code following the > information given by David in this thread https://mail.openvswitc > h.org/pipermail/ovs-discuss/2016-March/040236.html > > So I have the following simple setup where namespaces C1 and C2 are > connected to SW1. > > ip netns add C1 > ip netns add C2 > > ip link add C1-eth0 type veth peer name SW1-C1 > ip link add C2-eth0 type veth peer name SW1-C2 > ip link set SW1-C1 up > ip link set SW1-C2 up > > ip link set C1-eth0 netns C1 > ip link set C2-eth0 netns C2 > ip netns exec C1 ifconfig C1-eth0 10.50.10.1/24 up > ip netns exec C2 ifconfig C2-eth0 10.50.10.2/24 up > > ovs-ctl start > ovs-vsctl add-br SW1 > ovs-vsctl add-port SW1 SW1-C1 > ovs-vsctl add-port SW1 SW1-C2 > > There are no controllers, no flow rules added, and they ping each > other just fine. I inserted a log function call in the > ovs_dp_process_packet() in datapath.c so every time it is called I should > get a message printed to a file. However nothing is printed even though the > pings work. My expectation is that the first echo request should enter the > if(unlikely(!flow)) clause in ovs_dp_process_packet() and trigger an > upcall. What am I doing wrong? > > Thanks, > Alan > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Any timeline on PISCES in the mainline OVS?
Hello, I read about PISCES and watched its presentation. The presenter mentioned that the PISCES is expected to be in the mainline OVS. Is there any news regarding that, or regarding any kind of P4 integration? Would it be appropriate to use this mailing list to ask questions about PISCES as its git looks rather dormant? Thanks, Alan ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Can't set IPv6 next header field in packet buffer
That something simple was my port connection. Traffic was never passing through the switch. 2018-01-17 12:06 GMT+01:00 Alan Kayahan <hsy...@gmail.com>: > Hello, > > In my custom action, setting the ethertype as following works > > struct eth_header *eh = dp_packet_eth(b); > eh->eth_type = htons(ETH_TYPE_IPV6) > > However setting the next header field in IPv6 header as following doesn't. > > struct ip6_hdr *nh = dp_packet_l3(b) > nh->ip6_nxt = htons(IPPROTO_AH); > > I guess I am missing something simple and would appreciate some guidance. > > Alan > > > > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Can't set IPv6 next header field in packet buffer
Hello, In my custom action, setting the ethertype as following works struct eth_header *eh = dp_packet_eth(b); eh->eth_type = htons(ETH_TYPE_IPV6) However setting the next header field in IPv6 header as following doesn't. struct ip6_hdr *nh = dp_packet_l3(b) nh->ip6_nxt = htons(IPPROTO_AH); I guess I am missing something simple and would appreciate some guidance. Alan ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Is there an option to disable metadata propagation on patch ports?
Thanks Ben. So then the flow struct, along with its members excluding metadata, tunnel and regs propagate, correct? 2018-04-05 21:30 GMT+02:00 Ben Pfaff <b...@ovn.org>: > compose_output_action__(), in ofproto/ofproto-dpif-xlate.c, clears > metadata for patch port traversals in the block that begins "if > (xport->peer)", with this code: > > flow->metadata = htonll(0); > memset(>tunnel, 0, sizeof flow->tunnel); > memset(flow->regs, 0, sizeof flow->regs); > > > On Thu, Apr 05, 2018 at 08:52:31PM +0200, Alan Kayahan wrote: > > Thatis my question and theory :) How can I see that? Can you point me > > to the code where such decision is taken? > > > > 2018-04-05 20:39 GMT+02:00 Ben Pfaff <b...@ovn.org>: > > > > > What metadata is propagating? > > > > > > On Thu, Apr 05, 2018 at 08:14:51PM +0200, Alan Kayahan wrote: > > > > I introduced a new 32bit field in flow.h to match on. A push_header > > > action > > > > appends a header, which contains TLVs, to the packet and sets this > field > > > > for the first time. An increment action sets this field to a value > that > > > > resides in the first TLV, if increment action is called again, the > field > > > is > > > > set to the value in the second TLV and so on. > > > > > > > > I have a chain of br0-br1-br2 connected via patch ports. In br0 flow > rule > > > > calls push_header and out to br1. In br1, the flow rule matches on > the > > > port > > > > connected to br0, plus match on the 32bit field.The action for this > rule > > > is > > > > increment, then out to br2. Just like br1, br2 matches on the port > from > > > br1 > > > > plus the match on the 32bit field for the value in the TLV. > > > > > > > > Everything works well until br2. But the match on the field for the > value > > > > in the TLV doesn't work. If I remove the match on the field at br2 > and > > > just > > > > use in_port and redirect the traffic to packet hex printer, I do see > that > > > > the value in the TLV has been set correctly at the offset of this > 32bit > > > > field. > > > > > > > > The increment action depends on the iteration of TLVs in the header > at > > > each > > > > packet, therefore is at datapath and can not update the flow > context. So > > > if > > > > the flow metadata is propagating (which seems to be the only > > > explanation), > > > > the new field is set to the old value, which is the culprit. Perhaps > a > > > > better approach is to create an own context just like nat, iterate > the > > > TLVs > > > > and populate the context with all available values when push_header > is > > > > first called, then redesign the increment action so that it just > pops the > > > > next value from the context and emits the data-plane action. > Eliminates > > > > TLVs iteration per packet at the dataplane. > > > > > > > > But just to save the day, do you have a trick to stop this > propagation? > > > Or > > > > do you think the problem might be something else? > > > > > > > > Thanks! > > > > > > > > 2018-04-05 19:41 GMT+02:00 Ben Pfaff <b...@ovn.org>: > > > > > > > > > On Thu, Apr 05, 2018 at 07:31:35PM +0200, Alan Kayahan wrote: > > > > > > OVS patch ports allow the propagation of the metadata (e.g. flow > > > context) > > > > > > across the connected switches. > > > > > > > > > > > > Is there an option to disable the metadata propagation feature? I > > > need > > > > > this > > > > > > for my research to benchmark certain behavior. Or patch ports > become > > > > > > nothing but veth pairs when this feature is disabled? > > > > > > > > > > Patch ports zero out most metadata. What additional metadata do > you > > > > > want them to clear? > > > > > > > > > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Is there an option to disable metadata propagation on patch ports?
Hello, OVS patch ports allow the propagation of the metadata (e.g. flow context) across the connected switches. Is there an option to disable the metadata propagation feature? I need this for my research to benchmark certain behavior. Or patch ports become nothing but veth pairs when this feature is disabled? Thanks, Alan ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] There's no available (non-isolated) pmd thread on numa node 0, Expect reduced performance.
Hi Ian, > As you are setting all lcore and pmd core to node 1 why are you giving 1024 memory to node 0? > When processing packets for this port it means cpu is accessing data across the numa nodes which causes a performance penalty I am benchmarking performance in different settings and trying to understand the roles of OVS and DPDK in mediating core affinity of pmds and hugepage utilization. Your answer helps a lot! > try using ‘other_config:dpdk-socket-mem=0,4096’ and see if you still see the issue. But this warning should appear regardless the socket-mem allocation right? If my understanding is correct, when OVS pmd's are pinned to 10-19 and the VM tespmd app is pinned to core 2; the OVSpmd thread running on node 1 is having to access a huge-page on node 0 which VMtestpmd happens to access as well. Thanks, Alan 2018-04-12 18:28 GMT+02:00 Stokes, Ian <ian.sto...@intel.com>: > Hi, > > > > I was able to reproduce the issue on my system. > > > > As you are setting all lcore and pmd core to node 1 why are you giving > 1024 memory to node 0? > > > > I saw the same issue on my system but the warning did not appear once > memory was allocated to node 1 only. > > > > I would think the VM being launched is using memory for the vhost port > from node 0, however the queue for the vhost port is assigned to core 14 > which is on node1. When processing packets for this port it means cpu is > accessing data across the numa nodes which causes a performance penalty, > hence the warning. > > > > To avoid you should ensure all memory and cores operate on the same node > where possible, try using ‘other_config:dpdk-socket-mem=0,4096’ and see > if you still see the issue. > > > > Thanks > > Ian > > > > *From:* ovs-discuss-boun...@openvswitch.org [mailto:ovs-discuss-bounces@ > openvswitch.org] *On Behalf Of *Alan Kayahan > *Sent:* Thursday, April 12, 2018 2:27 AM > *To:* ovs-discuss@openvswitch.org > *Subject:* [ovs-discuss] There's no available (non-isolated) pmd thread > on numa node 0, Expect reduced performance. > > > > Hello, > > > > On the following setup, where all cores but 0 are isolated, > > > > available: 2 nodes (0-1) > > node 0 cpus: 0 1 2 3 4 5 6 7 8 9 > > node 1 cpus: 10 11 12 13 14 15 16 17 18 19 > > > > I am trying to start OVS entirely on numa node 1 as following > > > > ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true > other_config:dpdk-lcore-mask=0x00400 other_config:pmd-cpu-mask=0xffc00 > other_config:dpdk-socket-mem=1024,4096 > > > > However when I create a vhost port SRC to attach a VNF(via virtio) on node > 0, I get the following > > > > dpif_netdev|WARN|There's no available (non-isolated) pmd thread on numa > node 0. Queue 0 on port 'SRC' will be assigned to the pmd on core 14 (numa > node 1). Expect reduced performance. > > > > Any ideas? > > > > Thanks > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Is there an option to disable metadata propagation on patch ports?
Thatis my question and theory :) How can I see that? Can you point me to the code where such decision is taken? 2018-04-05 20:39 GMT+02:00 Ben Pfaff <b...@ovn.org>: > What metadata is propagating? > > On Thu, Apr 05, 2018 at 08:14:51PM +0200, Alan Kayahan wrote: > > I introduced a new 32bit field in flow.h to match on. A push_header > action > > appends a header, which contains TLVs, to the packet and sets this field > > for the first time. An increment action sets this field to a value that > > resides in the first TLV, if increment action is called again, the field > is > > set to the value in the second TLV and so on. > > > > I have a chain of br0-br1-br2 connected via patch ports. In br0 flow rule > > calls push_header and out to br1. In br1, the flow rule matches on the > port > > connected to br0, plus match on the 32bit field.The action for this rule > is > > increment, then out to br2. Just like br1, br2 matches on the port from > br1 > > plus the match on the 32bit field for the value in the TLV. > > > > Everything works well until br2. But the match on the field for the value > > in the TLV doesn't work. If I remove the match on the field at br2 and > just > > use in_port and redirect the traffic to packet hex printer, I do see that > > the value in the TLV has been set correctly at the offset of this 32bit > > field. > > > > The increment action depends on the iteration of TLVs in the header at > each > > packet, therefore is at datapath and can not update the flow context. So > if > > the flow metadata is propagating (which seems to be the only > explanation), > > the new field is set to the old value, which is the culprit. Perhaps a > > better approach is to create an own context just like nat, iterate the > TLVs > > and populate the context with all available values when push_header is > > first called, then redesign the increment action so that it just pops the > > next value from the context and emits the data-plane action. Eliminates > > TLVs iteration per packet at the dataplane. > > > > But just to save the day, do you have a trick to stop this propagation? > Or > > do you think the problem might be something else? > > > > Thanks! > > > > 2018-04-05 19:41 GMT+02:00 Ben Pfaff <b...@ovn.org>: > > > > > On Thu, Apr 05, 2018 at 07:31:35PM +0200, Alan Kayahan wrote: > > > > OVS patch ports allow the propagation of the metadata (e.g. flow > context) > > > > across the connected switches. > > > > > > > > Is there an option to disable the metadata propagation feature? I > need > > > this > > > > for my research to benchmark certain behavior. Or patch ports become > > > > nothing but veth pairs when this feature is disabled? > > > > > > Patch ports zero out most metadata. What additional metadata do you > > > want them to clear? > > > > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Is there an option to disable metadata propagation on patch ports?
I introduced a new 32bit field in flow.h to match on. A push_header action appends a header, which contains TLVs, to the packet and sets this field for the first time. An increment action sets this field to a value that resides in the first TLV, if increment action is called again, the field is set to the value in the second TLV and so on. I have a chain of br0-br1-br2 connected via patch ports. In br0 flow rule calls push_header and out to br1. In br1, the flow rule matches on the port connected to br0, plus match on the 32bit field.The action for this rule is increment, then out to br2. Just like br1, br2 matches on the port from br1 plus the match on the 32bit field for the value in the TLV. Everything works well until br2. But the match on the field for the value in the TLV doesn't work. If I remove the match on the field at br2 and just use in_port and redirect the traffic to packet hex printer, I do see that the value in the TLV has been set correctly at the offset of this 32bit field. The increment action depends on the iteration of TLVs in the header at each packet, therefore is at datapath and can not update the flow context. So if the flow metadata is propagating (which seems to be the only explanation), the new field is set to the old value, which is the culprit. Perhaps a better approach is to create an own context just like nat, iterate the TLVs and populate the context with all available values when push_header is first called, then redesign the increment action so that it just pops the next value from the context and emits the data-plane action. Eliminates TLVs iteration per packet at the dataplane. But just to save the day, do you have a trick to stop this propagation? Or do you think the problem might be something else? Thanks! 2018-04-05 19:41 GMT+02:00 Ben Pfaff <b...@ovn.org>: > On Thu, Apr 05, 2018 at 07:31:35PM +0200, Alan Kayahan wrote: > > OVS patch ports allow the propagation of the metadata (e.g. flow context) > > across the connected switches. > > > > Is there an option to disable the metadata propagation feature? I need > this > > for my research to benchmark certain behavior. Or patch ports become > > nothing but veth pairs when this feature is disabled? > > Patch ports zero out most metadata. What additional metadata do you > want them to clear? > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] There's no available (non-isolated) pmd thread on numa node 0, Expect reduced performance.
Hello, On the following setup, where all cores but 0 are isolated, available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 4 5 6 7 8 9 node 1 cpus: 10 11 12 13 14 15 16 17 18 19 I am trying to start OVS entirely on numa node 1 as following ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true other_config:dpdk-lcore-mask=0x00400 other_config:pmd-cpu-mask=0xffc00 other_config:dpdk-socket-mem=1024,4096 However when I create a vhost port SRC to attach a VNF(via virtio) on node 0, I get the following dpif_netdev|WARN|There's no available (non-isolated) pmd thread on numa node 0. Queue 0 on port 'SRC' will be assigned to the pmd on core 14 (numa node 1). Expect reduced performance. Any ideas? Thanks ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] There's no available (non-isolated) pmd thread on numa node 0, Expect reduced performance.
Hi Ian, >How are you starting the VM? QEMU or Libvirt? It is a docker container. I pass the following EAL parameters to the app that is running within echo $EAL_PARAMS -l 1 --master-lcore 1 -n 1 -m 1024 --file-prefix=nf1 --no-pci --vdev=virtio_user2,path=/var/run/openvswitch/NF1-v0,mac=00:00:92:00:00:03 --vdev=virtio_user3,path=/var/run/openvswitch/NF1-v1,mac=00:00:92:00:00:04 >to ensure that the vhost port used memory from the same socket as the core it’s PMD is running on I had to compile DPDK with CONFIG_RTE_LIBRTE_VHOST_NUMA=y. Wow thanks. This could even be the answer for the troubles I am having in my dpdk app. Much appreciated! Regards, Alan 2018-04-16 19:38 GMT+02:00 Stokes, Ian <ian.sto...@intel.com>: > Hi Alan, > > > > How are you starting the VM? QEMU or Libvirt? > > > > The dpdk vhost ports are associated with the numa node the virtqueue memory > has been allocated on initially. So if running the VM you may want to use > taskset –c with QEMU to allocate cores associated with numa node 1 to run the > VM. If using libvirt try to ensure vcupin corresponds to numa 1 cores also in > the xml. > > > > In my testing to ensure that the vhost port used memory from the same socket > as the core it’s PMD is running on I had to compile DPDK with > CONFIG_RTE_LIBRTE_VHOST_NUMA=y. This will avoid the warning altogether > regardless if memory is allocated to both sockets. > > > > If you’re interested in how to test this there is a blog for using vhost numa > aware that could be of use: > > > > https://software.intel.com/en-us/articles/vhost-user-numa-awareness-in-open-vswitch-with-dpdk > > > > Hope this helps. > > Ian > > > > *From:* Alan Kayahan [mailto:hsy...@gmail.com] > *Sent:* Friday, April 13, 2018 8:05 AM > *To:* Stokes, Ian <ian.sto...@intel.com> > *Cc:* ovs-discuss@openvswitch.org > *Subject:* Re: [ovs-discuss] There's no available (non-isolated) pmd > thread on numa node 0, Expect reduced performance. > > > > Hi Ian, > > > > > As you are setting all lcore and pmd core to node 1 why are you giving > 1024 memory to node 0? > > > When processing packets for this port it means cpu is accessing data > across the numa nodes which causes a performance penalty > > I am benchmarking performance in different settings and trying to > understand the roles of OVS and DPDK in mediating core affinity of pmds and > hugepage utilization. Your answer helps a lot! > > > > > try using ‘other_config:dpdk-socket-mem=0,4096’ and see if you still > see the issue. > > But this warning should appear regardless the socket-mem allocation > right? If my understanding is correct, when OVS pmd's are pinned to 10-19 > and the VM tespmd app is pinned to core 2; the OVSpmd thread running on > node 1 is having to access a huge-page on node 0 which VMtestpmd happens to > access as well. > > > > Thanks, > > Alan > > > > > > 2018-04-12 18:28 GMT+02:00 Stokes, Ian <ian.sto...@intel.com>: > > Hi, > > > > I was able to reproduce the issue on my system. > > > > As you are setting all lcore and pmd core to node 1 why are you giving > 1024 memory to node 0? > > > > I saw the same issue on my system but the warning did not appear once > memory was allocated to node 1 only. > > > > I would think the VM being launched is using memory for the vhost port > from node 0, however the queue for the vhost port is assigned to core 14 > which is on node1. When processing packets for this port it means cpu is > accessing data across the numa nodes which causes a performance penalty, > hence the warning. > > > > To avoid you should ensure all memory and cores operate on the same node > where possible, try using ‘other_config:dpdk-socket-mem=0,4096’ and see > if you still see the issue. > > > > Thanks > > Ian > > > > *From:* ovs-discuss-boun...@openvswitch.org [mailto:ovs-discuss-bounces@ > openvswitch.org] *On Behalf Of *Alan Kayahan > *Sent:* Thursday, April 12, 2018 2:27 AM > *To:* ovs-discuss@openvswitch.org > *Subject:* [ovs-discuss] There's no available (non-isolated) pmd thread > on numa node 0, Expect reduced performance. > > > > Hello, > > > > On the following setup, where all cores but 0 are isolated, > > > > available: 2 nodes (0-1) > > node 0 cpus: 0 1 2 3 4 5 6 7 8 9 > > node 1 cpus: 10 11 12 13 14 15 16 17 18 19 > > > > I am trying to start OVS entirely on numa node 1 as following > > > > ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true > other_config:dpdk-lcore-mask=0x00400 other_config:pmd-cpu-mask=0xffc00 > other_con
Re: [ovs-discuss] Is a new field in flow.h necessary for matching on a custom v6 extension header?
Makes total sense. I started with PISCES and made some progress, however it feels like going into uncharted waters. I feared I might hit something above my pay grade, so I am back to modifying ovs instead. Best of luck on your P4 journey :) 2018-02-23 19:22 GMT+01:00 Ben Pfaff <b...@ovn.org>: > Ultimately, I expect a P4-based solution, in which the controller > supplies a P4 program that extracts the fields that it is interested in. > This is my big project for OVS 2.10. (I don't know how successful I'll > be yet.) > > On Fri, Feb 23, 2018 at 12:37:40AM +0100, Alan Kayahan wrote: > > I understand that there must be a construct in the struct flow, and > > introducing a new be32 will save the day for me. However looking at this > > from a broader perspective, say matching on the fields of any v6 > extension > > header made into the OF specification, neither adding individual fields > nor > > adding every v6 extension header into the struct flow looks elegant to > me. > > What do you think? > > > > 2018-02-21 17:10 GMT+01:00 Ben Pfaff <b...@ovn.org>: > > > > > On Mon, Feb 19, 2018 at 06:15:30PM +0100, Alan Kayahan wrote: > > > > I have a custom v6 extension header, in which I would like to > perform an > > > > LPM match on only one 32bit field. > > > > > > > > Since an extension header is technically not a new field, do I still > need > > > > to introduce anything in the struct flow? > > > > > > Without anything in struct flow, how do you propose to match on it? > > > > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Native tunnel routing failed error for VXLAN between ovs bridges
Hello,
I have ovs bridges SW-00 and SW-01 operating in their respective
docker containers as following
-Container A--
2: ovs-netdev: mtu 1500 qdisc noop state
DOWN group default qlen 1000
link/ether 96:56:12:93:cc:59 brd ff:ff:ff:ff:ff:ff
3: SW-00: mtu 1500 qdisc noop state DOWN
group default qlen 1000
link/ether e6:1e:5e:86:4a:42 brd ff:ff:ff:ff:ff:ff
394: eth0@if395: mtu 1500 qdisc
noqueue state UP group default
link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0
valid_lft forever preferred_lft forever
401: VETH-00_01@if400: mtu 1500
qdisc noqueue state UP group default qlen 1000
link/ether c6:90:ee:5d:bb:8a brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet 192.168.0.1/24 scope global VETH-00_01
valid_lft forever preferred_lft forever
e67f7443-fbb1-488e-9884-4c755a61f6ab
Bridge "SW-00"
Controller "tcp:172.17.0.1:6653"
is_connected: true
Port "VXLAN-00_01"
Interface "VXLAN-00_01"
type: vxlan
options: {key="1", local_ip="192.168.0.1",
remote_ip="192.168.0.2"}
Port "SW-00"
Interface "SW-00"
type: internal
ovs_version: "2.10.90"
cookie=0x0, duration=1853.837s, table=0, n_packets=7, n_bytes=124,
priority=10,in_port=10 actions=output:"VXLAN-00_01"
-Container B--
2: ovs-netdev: mtu 1500 qdisc noop state
DOWN group default qlen 1000
link/ether c2:aa:82:0b:9e:f6 brd ff:ff:ff:ff:ff:ff
3: SW-01: mtu 1500 qdisc noop state DOWN
group default qlen 1000
link/ether 2a:f7:3d:b8:06:48 brd ff:ff:ff:ff:ff:ff
396: eth0@if397: mtu 1500 qdisc
noqueue state UP group default
link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.17.0.3/16 brd 172.17.255.255 scope global eth0
valid_lft forever preferred_lft forever
400: VETH-01_00@if401: mtu 1500
qdisc noqueue state UP group default qlen 1000
link/ether 76:6b:e4:02:18:7f brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet 192.168.0.2/24 scope global VETH-01_00
valid_lft forever preferred_lft forever
8fceec3b-e926-40e5-92ee-0d6ffe7e458a
Bridge "SW-01"
Controller "tcp:172.17.0.1:6653"
is_connected: true
Port "VXLAN-01_00"
Interface "VXLAN-01_00"
type: vxlan
options: {key="1", local_ip="192.168.0.2",
remote_ip="192.168.0.1"}
Port "SW-01"
Interface "SW-01"
type: internal
ovs_version: "2.10.90"
cookie=0x0, duration=1882.835s, table=0, n_packets=0, n_bytes=0,
priority=10,in_port="VXLAN-01_00" actions=output:"5"
/-
So the two containers are connected via VETH pair and can ping.
However the VXLAN traffic is not passing any traffic; the packets are
hitting the rule on SW-00, however they are not making it to SW-01.
ovs-appctl ofproto/trace SW-00 in_port=10
Flow:
in_port=10,vlan_tci=0x,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x
bridge("SW-00")
---
0. in_port=10, priority 10
output:2
-> output to native tunnel
>> native tunnel routing failed
Final flow: unchanged
Megaflow:
recirc_id=0,eth,in_port=10,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x
Datapath actions: drop
ovs-appctl ovs/route/show
Route Table:
Cached: 127.0.0.1/32 dev lo SRC 127.0.0.1 local
Cached: 172.17.0.2/32 dev eth0 SRC 172.17.0.2 local
Cached: 192.168.0.1/32 dev VETH-00_01 SRC 192.168.0.1 local
Cached: 127.0.0.0/8 dev lo SRC 127.0.0.1 local
Cached: 192.168.0.0/24 dev VETH-00_01 SRC 192.168.0.1
Cached: 172.17.0.0/16 dev eth0 SRC 172.17.0.2
Cached: 0.0.0.0/0 dev eth0 GW 172.17.0.1 SRC 172.17.0.2
The ofproto trace reports "native tunnel routing failed", despite
there exists an appropriate entry in ovs/route/show. I'd appreciate
any input on why this could be happening.
Regards,
Alan
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Native tunnel routing failed error for VXLAN between ovs bridges
After reading some example deployments, I removed the IP address
assignments from VETH ends. Instead, I assigned the IP addresses to
bridges themselves (all 3 in the same subnet) and set the link status
for the bridges to UP. Now I see the following.
bridge("SW-00")
---
0. in_port=5, priority 10
output:9
-> output to native tunnel
-> tunneling to 192.168.0.2 via SW-00
-> neighbor cache miss for 192.168.0.2 on bridge SW-00, sending ARP request
Final flow: unchanged
Megaflow:
recirc_id=0,eth,in_port=5,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x
Datapath actions: drop
However I don't see any packet_in from the bridge at the controller
for such ARP request. Any ideas?
On Mon, Oct 29, 2018 at 6:54 AM Alan Kayahan wrote:
>
> Hello,
>
> I have ovs bridges SW-00 and SW-01 operating in their respective
> docker containers as following
>
> -Container A--
> 2: ovs-netdev: mtu 1500 qdisc noop state
> DOWN group default qlen 1000
> link/ether 96:56:12:93:cc:59 brd ff:ff:ff:ff:ff:ff
> 3: SW-00: mtu 1500 qdisc noop state DOWN
> group default qlen 1000
> link/ether e6:1e:5e:86:4a:42 brd ff:ff:ff:ff:ff:ff
> 394: eth0@if395: mtu 1500 qdisc
> noqueue state UP group default
> link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
> inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0
>valid_lft forever preferred_lft forever
> 401: VETH-00_01@if400: mtu 1500
> qdisc noqueue state UP group default qlen 1000
> link/ether c6:90:ee:5d:bb:8a brd ff:ff:ff:ff:ff:ff link-netnsid 1
> inet 192.168.0.1/24 scope global VETH-00_01
>valid_lft forever preferred_lft forever
>
>
> e67f7443-fbb1-488e-9884-4c755a61f6ab
> Bridge "SW-00"
> Controller "tcp:172.17.0.1:6653"
> is_connected: true
> Port "VXLAN-00_01"
> Interface "VXLAN-00_01"
> type: vxlan
> options: {key="1", local_ip="192.168.0.1",
> remote_ip="192.168.0.2"}
> Port "SW-00"
> Interface "SW-00"
> type: internal
> ovs_version: "2.10.90"
>
> cookie=0x0, duration=1853.837s, table=0, n_packets=7, n_bytes=124,
> priority=10,in_port=10 actions=output:"VXLAN-00_01"
>
>
> -Container B--
> 2: ovs-netdev: mtu 1500 qdisc noop state
> DOWN group default qlen 1000
> link/ether c2:aa:82:0b:9e:f6 brd ff:ff:ff:ff:ff:ff
> 3: SW-01: mtu 1500 qdisc noop state DOWN
> group default qlen 1000
> link/ether 2a:f7:3d:b8:06:48 brd ff:ff:ff:ff:ff:ff
> 396: eth0@if397: mtu 1500 qdisc
> noqueue state UP group default
> link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
> inet 172.17.0.3/16 brd 172.17.255.255 scope global eth0
>valid_lft forever preferred_lft forever
> 400: VETH-01_00@if401: mtu 1500
> qdisc noqueue state UP group default qlen 1000
> link/ether 76:6b:e4:02:18:7f brd ff:ff:ff:ff:ff:ff link-netnsid 1
> inet 192.168.0.2/24 scope global VETH-01_00
>valid_lft forever preferred_lft forever
>
>
> 8fceec3b-e926-40e5-92ee-0d6ffe7e458a
> Bridge "SW-01"
> Controller "tcp:172.17.0.1:6653"
> is_connected: true
> Port "VXLAN-01_00"
> Interface "VXLAN-01_00"
> type: vxlan
> options: {key="1", local_ip="192.168.0.2",
> remote_ip="192.168.0.1"}
> Port "SW-01"
> Interface "SW-01"
> type: internal
> ovs_version: "2.10.90"
>
> cookie=0x0, duration=1882.835s, table=0, n_packets=0, n_bytes=0,
> priority=10,in_port="VXLAN-01_00" actions=output:"5"
>
> /-
>
> So the two containers are connected via VETH pair and can ping.
> However the VXLAN traffic is not passing any traffic; the packets are
> hitting the rule on SW-00, however they are not making it to SW-01.
>
> ovs-appctl ofproto/trace SW-00 in_port=10
> Flow:
> in_port=10,vlan_tci=0x,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x
>
> bridge("SW-00")
> ---
> 0. in_port=10, priority 10
> output:2
> -> output to native tunnel
> >> native tunnel routing failed
>
> Final flow: unchanged
> Megaflow:
> recirc_id=0,eth,in_port=10,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x
> Datapath actions: drop
>
> ovs-appctl ovs/route/show
> Route Ta
[ovs-discuss] OVS bridges in docker containers segfault when dpdkvhostuser port is added.
Hello, I have 3 OVS bridges on the same host, connected to each other as br1<->br2<->br3. br1 and br3 are connected to the docker container cA via dpdkvhostuser port type (I know it is deprecated, the app works this way only). The DPDK app running in cA generate packets, which traverse bridges br1->br2->br3, then ends up back at the DPDK app. This setup works fine. Now I am trying to put each OVS bridge into its respective docker container. I connect the containers with veth pairs, then add the veth ports to the bridges. Next, I add a dpdkvhostuser port named SRC to br1, so far so good. The moment I add a dpdkvhostuser port named SNK to br3, ovs-vswitchd services in br1's and br3's containers segfault. Following are the backtraces from each, --br1's container--- [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock -vconsole:emer -vsyslo'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x5608fa0f321b in netdev_rxq_recv (rx=0x7ff13c34ee80, batch=batch@entry=0x7ff1bbb4d890) at lib/netdev.c:702 702retval = rx->netdev->netdev_class->rxq_recv(rx, batch); [Current thread is 1 (Thread 0x7ff1bbb4e700 (LWP 376))] (gdb) bt #0 0x5608fa0f321b in netdev_rxq_recv (rx=0x7ff13c34ee80, batch=batch@entry=0x7ff1bbb4d890) at lib/netdev.c:702 #1 0x5608fa0cce65 in dp_netdev_process_rxq_port ( pmd=pmd@entry=0x7ff1bbb4f010, rxq=0x5608fb651be0, port_no=1) at lib/dpif-netdev.c:3279 #2 0x5608fa0cd296 in pmd_thread_main (f_=) at lib/dpif-netdev.c:4145 #3 0x5608fa14a836 in ovsthread_wrapper (aux_=) at lib/ovs-thread.c:348 #4 0x7ff1c52517fc in start_thread (arg=0x7ff1bbb4e700) at pthread_create.c:465 #5 0x7ff1c4815b5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 --br3's container--- [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock -vconsole:emer -vsyslo'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x55c517e3abcb in rte_mempool_free_memchunks () [Current thread is 1 (Thread 0x7f202351f300 (LWP 647))] (gdb) bt #0 0x55c517e3abcb in rte_mempool_free_memchunks () #1 0x55c517e3ad46 in rte_mempool_free.part () #2 0x55c518218b78 in dpdk_mp_free (mp=0x7f603fe66a00) at lib/netdev-dpdk.c:599 #3 0x55c518218ff0 in dpdk_mp_free (mp=) at lib/netdev-dpdk.c:593 #4 netdev_dpdk_mempool_configure (dev=0x7f1f7ffeac00) at lib/netdev-dpdk.c:629 #5 0x55c51821a98d in dpdk_vhost_reconfigure_helper (dev=0x7f1f7ffeac00) at lib/netdev-dpdk.c:3599 #6 0x55c51821ac8b in netdev_dpdk_vhost_reconfigure (netdev=0x7f1f7ffebcc0) at lib/netdev-dpdk.c:3624 #7 0x55c51813fe6b in port_reconfigure (port=0x55c51a4522a0) at lib/dpif-netdev.c:3341 #8 reconfigure_datapath (dp=dp@entry=0x55c51a46efc0) at lib/dpif-netdev.c:3822 #9 0x55c5181403e8 in do_add_port (dp=dp@entry=0x55c51a46efc0, devname=devname@entry=0x55c51a456520 "SNK", type=0x55c51834f7bd "dpdkvhostuser", port_no=port_no@entry=1) at lib/dpif-netdev.c:1584 #10 0x55c51814059b in dpif_netdev_port_add (dpif=, netdev=0x7f1f7ffebcc0, port_nop=0x7fffb4eef68c) at lib/dpif-netdev.c:1610 #11 0x55c5181469be in dpif_port_add (dpif=0x55c51a469350, netdev=netdev@entry=0x7f1f7ffebcc0, port_nop=port_nop@entry=0x7fffb4eef6ec) at lib/dpif.c:579 ---Type to continue, or q to quit--- #12 0x55c5180f9f28 in port_add (ofproto_=0x55c51a464ee0, netdev=0x7f1f7ffebcc0) at ofproto/ofproto-dpif.c:3645 #13 0x55c5180ecafe in ofproto_port_add (ofproto=0x55c51a464ee0, netdev=0x7f1f7ffebcc0, ofp_portp=ofp_portp@entry=0x7fffb4eef7e8) at ofproto/ofproto.c:1999 #14 0x55c5180d97e6 in iface_do_create (errp=0x7fffb4eef7f8, netdevp=0x7fffb4eef7f0, ofp_portp=0x7fffb4eef7e8, iface_cfg=0x55c51a46d590, br=0x55c51a4415b0) at vswitchd/bridge.c:1799 #15 iface_create (port_cfg=0x55c51a46e210, iface_cfg=0x55c51a46d590, br=0x55c51a4415b0) at vswitchd/bridge.c:1837 #16 bridge_add_ports__ (br=br@entry=0x55c51a4415b0, wanted_ports=wanted_ports@entry=0x55c51a441690, with_requested_port=with_requested_port@entry=true) at vswitchd/bridge.c:931 #17 0x55c5180db87a in bridge_add_ports (wanted_ports=0x55c51a441690, br=0x55c51a4415b0) at vswitchd/bridge.c:942 #18 bridge_reconfigure (ovs_cfg=ovs_cfg@entry=0x55c51a46ea80) at vswitchd/bridge.c:661 #19 0x55c5180df989 in bridge_run () at vswitchd/bridge.c:3016 #20 0x55c517dbc535 in main (argc=, argv=) at vswitchd/ovs-vswitchd.c:120 Note that /dev/hugepages of the host is shared with all containers. I have a feeling that br3 is overwriting the hugepage file of br1. Any ideas? Regards, Alan ___
[ovs-discuss] How to match the VLAN VID of frames tunneled to a VXLAN port?
Hello, I am trying to build a simplified version of the "sample L2 transparent network service chaining implementation" described in section 6 of this document https://www.opennetworking.org/wp-content/uploads/2014/10/L4-L7_Service_Function_Chaining_Solution_Architecture.pdf. My simplifications: 1) 3 OVS bridges on 3 nodes 2) Just a single flow and a single chain to steer without bypassing any service. 3) No 5-tuple matching, just in_port 4) VXLAN remoteIPs are hardcoded, key=flow, and attached as a port to the bridge Problem: Matching the VLAN field inbound to a VXLAN port doesn't work. cookie=0x0, duration=15.549s, table=0, n_packets=0, n_bytes=0, priority=10,tun_id=0x32,in_port="VXLAN-02_01",dl_vlan=100,dl_type=0x8100 actions=pop_vlan,output:"Host-eth0" If I remove the VLAN match as following, the host receives the ethernet frames with the VLAN tag that was pushed at the ingress on the first bridge. cookie=0x0, duration=15.549s, table=0, n_packets=0, n_bytes=0, priority=10,tun_id=0x32,in_port="VXLAN-02_01", actions=output:"Host-eth0" So the VXLAN tunneling works fine, however matching on the VLAN does not work as suggested in the document. For instance for the flow1 at SWA, the action is to push VLAN tag 100, set tunnel ID to 50 and tunnel it to SWB. Then the match field for flow1 at SWB is the in_port and VLAN100, which doesn't seem to work for me. Should there be a decapsulate_vxlan action there? What am I missing? Regards, Alan ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Native tunnel routing failed error for VXLAN between ovs bridges
Solved this one, and here is what I have done in case it may be
helpful to someone.
My setup is a derivative of
http://docs.openvswitch.org/en/latest/howto/userspace-tunneling/,
where I am using a single ovs bridge per node and connect the ovs
bridges of each node via VETH pairs instead. The guide states that
"This additional bridge is required when running Open vSwitch in
userspace rather than kernel-based Open vSwitch. The purpose of this
bridge is to allow use of the kernel network stack for routing and ARP
resolution.". Getting rid of this bridge means that I had to take care
of the tunnel related ARP by myself.
First https://mail.openvswitch.org/pipermail/ovs-discuss/2016-April/040618.html
fixed the packet_in ARP request issue. I had to avoid .NORMAL or
.FLOOD actions in ARP handling because ARP requests looped between the
VXLAN ports. So my workaround first install 2 rules;
arp_tpa="bridgeIP" actions=LOCAL and nw_dst="bridgeIP" actions=LOCAL.
Then use ovs-appctl tnl/arp/set for "bridgeIP" to "bridgeMAC".
On Tue, Oct 30, 2018 at 4:10 PM Alan Kayahan wrote:
>
> After reading some example deployments, I removed the IP address
> assignments from VETH ends. Instead, I assigned the IP addresses to
> bridges themselves (all 3 in the same subnet) and set the link status
> for the bridges to UP. Now I see the following.
>
> bridge("SW-00")
> ---
> 0. in_port=5, priority 10
> output:9
> -> output to native tunnel
> -> tunneling to 192.168.0.2 via SW-00
> -> neighbor cache miss for 192.168.0.2 on bridge SW-00, sending ARP
> request
>
> Final flow: unchanged
> Megaflow:
> recirc_id=0,eth,in_port=5,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x
> Datapath actions: drop
>
> However I don't see any packet_in from the bridge at the controller
> for such ARP request. Any ideas?
> On Mon, Oct 29, 2018 at 6:54 AM Alan Kayahan wrote:
> >
> > Hello,
> >
> > I have ovs bridges SW-00 and SW-01 operating in their respective
> > docker containers as following
> >
> > -Container A--
> > 2: ovs-netdev: mtu 1500 qdisc noop state
> > DOWN group default qlen 1000
> > link/ether 96:56:12:93:cc:59 brd ff:ff:ff:ff:ff:ff
> > 3: SW-00: mtu 1500 qdisc noop state DOWN
> > group default qlen 1000
> > link/ether e6:1e:5e:86:4a:42 brd ff:ff:ff:ff:ff:ff
> > 394: eth0@if395: mtu 1500 qdisc
> > noqueue state UP group default
> > link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
> > inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0
> >valid_lft forever preferred_lft forever
> > 401: VETH-00_01@if400: mtu 1500
> > qdisc noqueue state UP group default qlen 1000
> > link/ether c6:90:ee:5d:bb:8a brd ff:ff:ff:ff:ff:ff link-netnsid 1
> > inet 192.168.0.1/24 scope global VETH-00_01
> >valid_lft forever preferred_lft forever
> >
> >
> > e67f7443-fbb1-488e-9884-4c755a61f6ab
> > Bridge "SW-00"
> > Controller "tcp:172.17.0.1:6653"
> > is_connected: true
> > Port "VXLAN-00_01"
> > Interface "VXLAN-00_01"
> > type: vxlan
> > options: {key="1", local_ip="192.168.0.1",
> > remote_ip="192.168.0.2"}
> > Port "SW-00"
> > Interface "SW-00"
> > type: internal
> > ovs_version: "2.10.90"
> >
> > cookie=0x0, duration=1853.837s, table=0, n_packets=7, n_bytes=124,
> > priority=10,in_port=10 actions=output:"VXLAN-00_01"
> >
> >
> > -Container B--
> > 2: ovs-netdev: mtu 1500 qdisc noop state
> > DOWN group default qlen 1000
> > link/ether c2:aa:82:0b:9e:f6 brd ff:ff:ff:ff:ff:ff
> > 3: SW-01: mtu 1500 qdisc noop state DOWN
> > group default qlen 1000
> > link/ether 2a:f7:3d:b8:06:48 brd ff:ff:ff:ff:ff:ff
> > 396: eth0@if397: mtu 1500 qdisc
> > noqueue state UP group default
> > link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
> > inet 172.17.0.3/16 brd 172.17.255.255 scope global eth0
> >valid_lft forever preferred_lft forever
> > 400: VETH-01_00@if401: mtu 1500
> > qdisc noqueue state UP group default qlen 1000
> > link/ether 76:6b:e4:02:18:7f brd ff:ff:ff:ff:ff:ff link-netnsid 1
> > inet 192.168.0.2/24 scope global VETH-01_00
> >valid_lft forever preferred_lft f
Re: [ovs-discuss] How to match the VLAN VID of frames tunneled to a VXLAN port?
Your response made me doubt the simple things so I tried some combinations. Removed the match dl_type=0x8100 from the rule and it worked. Following is how the rule looks like now. cookie=0x0, duration=20.297s, table=0, n_packets=7, n_bytes=614, priority=10,tun_id=0x32,in_port="VXLAN-02_01",dl_vlan=100 actions=pop_vlan,output:"Host-eth0" dl_type had always been a habitual OF precondition for me for VLAN operations. Enjoy your weekend Ben! Alan On Sun, Nov 4, 2018 at 6:06 AM Ben Pfaff wrote: > > Hmm, VLAN match should work fine for that case. I'm surprised that > there are problems. > > On Sun, Nov 04, 2018 at 05:37:05AM +0100, Alan Kayahan wrote: > > Thanks for the response Ben. > > If I am understanding the document correctly, the packets coming in > > from the host (classified as Flow1 on page 29) in Switch A are tagged > > with VLAN100, then tunneled to O3 (SwitchB's IP address) with a VNI ID > > 50. So there is only 1 VLAN field and that is the inner (the VLAN > > field of the frame encapsulated in the VXLAN tunnel), which I need to > > match on. > > > > Alan > > On Sat, Nov 3, 2018 at 4:25 PM Ben Pfaff wrote: > > > > > > On Sat, Nov 03, 2018 at 06:45:08AM +0100, Alan Kayahan wrote: > > > > Hello, > > > > > > > > I am trying to build a simplified version of the "sample L2 > > > > transparent network service chaining implementation" described in > > > > section 6 of this document > > > > https://www.opennetworking.org/wp-content/uploads/2014/10/L4-L7_Service_Function_Chaining_Solution_Architecture.pdf. > > > > > > > > My simplifications: > > > > 1) 3 OVS bridges on 3 nodes > > > > 2) Just a single flow and a single chain to steer without bypassing any > > > > service. > > > > 3) No 5-tuple matching, just in_port > > > > 4) VXLAN remoteIPs are hardcoded, key=flow, and attached as a port to > > > > the bridge > > > > > > > > Problem: Matching the VLAN field inbound to a VXLAN port doesn't work. > > > > > > Do you expect the VLAN field to match the inner or outer VLAN? It will > > > actually match the inner VLAN? ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] How to match the VLAN VID of frames tunneled to a VXLAN port?
Thanks for the response Ben. If I am understanding the document correctly, the packets coming in from the host (classified as Flow1 on page 29) in Switch A are tagged with VLAN100, then tunneled to O3 (SwitchB's IP address) with a VNI ID 50. So there is only 1 VLAN field and that is the inner (the VLAN field of the frame encapsulated in the VXLAN tunnel), which I need to match on. Alan On Sat, Nov 3, 2018 at 4:25 PM Ben Pfaff wrote: > > On Sat, Nov 03, 2018 at 06:45:08AM +0100, Alan Kayahan wrote: > > Hello, > > > > I am trying to build a simplified version of the "sample L2 > > transparent network service chaining implementation" described in > > section 6 of this document > > https://www.opennetworking.org/wp-content/uploads/2014/10/L4-L7_Service_Function_Chaining_Solution_Architecture.pdf. > > > > My simplifications: > > 1) 3 OVS bridges on 3 nodes > > 2) Just a single flow and a single chain to steer without bypassing any > > service. > > 3) No 5-tuple matching, just in_port > > 4) VXLAN remoteIPs are hardcoded, key=flow, and attached as a port to the > > bridge > > > > Problem: Matching the VLAN field inbound to a VXLAN port doesn't work. > > Do you expect the VLAN field to match the inner or outer VLAN? It will > actually match the inner VLAN? ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] OVS bridges in docker containers segfault when dpdkvhostuser port is added.
OVS Version 2.9.90 DPDK Version 17.11 Ubuntu 17.10 Kernel 4.13.0-21-generic Docker 18.06.1-ce I figured out that no hugepage was being consumed by any of the switches. What is happening is that when ovs-vswitchd is starting, the number of free hugepages drop all the way to 0 (in /dev/hugepages, I see files rtemap_0 to rtemap_7 created), then all of them get deleted and the free hugepages count is back to what it was before (I was expecting it to be decremented by one). So I was running all containers with a one-to-one mapping (docker argument -v /dev/hugepages:/dev/hugepages) when this problem was occurring. I went ahead and created a subfolder in /dev/hugepages in the host for each OVS container and mapped as "-v /dev/hugepages/br-n:/dev/hugepages" where n is the bridge number. This resolved the segfault problem with dpdkvhost port attachment. However I still observe the behavior where the free hugepages count hits 0 (files rtemap_0 to rtemap_7 being created in /dev/hugepages/br-n for each bridge) while ovs-vswitchd is starting in a container. Then all rtemap files but one are deleted, and the free hugepages count is decremented by one from the number before starting ovs-vswitchd, which is an indication that one hugepage is allocated successfully. Packets seem to be forwarded just fine and everything looks ok to me, but I am curious whether this is the normal behavior for allocating hugepages to OVS. On Fri, Nov 2, 2018 at 7:01 PM Alan Kayahan wrote: > > Thanks for the response all. > > @Ian > 1GB pagesize, 8 total pages. OVS is launched without the > dpdk-socket-mem options so it should take 1GB. When one switch is > started, free hugepage count drops to 7. When I launch another, I'd > expect it to drop to 6 but it crashes instead. > > My DPDK apps create the hugepage file in /dev/hugepages with the name > I specify. I am assuming ovs-vswitchd is responsible for the naming of > ovs hugepage files in /dev/hugepages. I believe it wouldn't be a > problem (I will test this and respond) if both dpdk bridges were > managed by the same ovs-vswitchd service. But in the containerized > scenario, two ovs-vswitchd services are accessing the same > /dev/hugepaes path. Dont you think this would be a problem? Or is it > the openvswitch kernel module that is in charge of hugepage > coordination? > > @Ben > Will retrieve the info you requested as soon as I eliminate couple of > other possible causes. > > Alan > On Wed, Oct 31, 2018 at 11:00 AM Stokes, Ian wrote: > > > > > On Thu, Oct 25, 2018 at 09:51:38PM +0200, Alan Kayahan wrote: > > > > Hello, > > > > > > > > I have 3 OVS bridges on the same host, connected to each other as > > > > br1<->br2<->br3. br1 and br3 are connected to the docker container cA > > > > via dpdkvhostuser port type (I know it is deprecated, the app works > > > > this way only). The DPDK app running in cA generate packets, which > > > > traverse bridges br1->br2->br3, then ends up back at the DPDK app. > > > > This setup works fine. > > > > > > > > Now I am trying to put each OVS bridge into its respective docker > > > > container. I connect the containers with veth pairs, then add the veth > > > > ports to the bridges. Next, I add a dpdkvhostuser port named SRC to > > > > br1, so far so good. The moment I add a dpdkvhostuser port named SNK > > > > to br3, ovs-vswitchd services in br1's and br3's containers segfault. > > > > Following are the backtraces from each, > > > > What version of OVS and DPDK are you using? > > > > > > > > > > --br1's container--- > > > > > > > > [Thread debugging using libthread_db enabled] Using host libthread_db > > > > library "/lib/x86_64-linux-gnu/libthread_db.so.1". > > > > Core was generated by `ovs-vswitchd > > > > unix:/usr/local/var/run/openvswitch/db.sock -vconsole:emer -vsyslo'. > > > > Program terminated with signal SIGSEGV, Segmentation fault. > > > > #0 0x5608fa0f321b in netdev_rxq_recv (rx=0x7ff13c34ee80, > > > > batch=batch@entry=0x7ff1bbb4d890) at lib/netdev.c:702 > > > > 702retval = rx->netdev->netdev_class->rxq_recv(rx, batch); > > > > [Current thread is 1 (Thread 0x7ff1bbb4e700 (LWP 376))] > > > > (gdb) bt > > > > #0 0x5608fa0f321b in netdev_rxq_recv (rx=0x7ff13c34ee80, > > > > batch=batch@entry=0x7ff1bbb4d890) at lib/netdev.c:702 > > > > #1 0x5608fa0cce65 in dp_netdev_process_rxq_port ( > > > > pmd=pmd@entry=0x7ff1bbb4f010, rxq=0x5608fb651be0, port_no=1)
Re: [ovs-discuss] OVS bridges in docker containers segfault when dpdkvhostuser port is added.
Thanks for the response all. @Ian 1GB pagesize, 8 total pages. OVS is launched without the dpdk-socket-mem options so it should take 1GB. When one switch is started, free hugepage count drops to 7. When I launch another, I'd expect it to drop to 6 but it crashes instead. My DPDK apps create the hugepage file in /dev/hugepages with the name I specify. I am assuming ovs-vswitchd is responsible for the naming of ovs hugepage files in /dev/hugepages. I believe it wouldn't be a problem (I will test this and respond) if both dpdk bridges were managed by the same ovs-vswitchd service. But in the containerized scenario, two ovs-vswitchd services are accessing the same /dev/hugepaes path. Dont you think this would be a problem? Or is it the openvswitch kernel module that is in charge of hugepage coordination? @Ben Will retrieve the info you requested as soon as I eliminate couple of other possible causes. Alan On Wed, Oct 31, 2018 at 11:00 AM Stokes, Ian wrote: > > > On Thu, Oct 25, 2018 at 09:51:38PM +0200, Alan Kayahan wrote: > > > Hello, > > > > > > I have 3 OVS bridges on the same host, connected to each other as > > > br1<->br2<->br3. br1 and br3 are connected to the docker container cA > > > via dpdkvhostuser port type (I know it is deprecated, the app works > > > this way only). The DPDK app running in cA generate packets, which > > > traverse bridges br1->br2->br3, then ends up back at the DPDK app. > > > This setup works fine. > > > > > > Now I am trying to put each OVS bridge into its respective docker > > > container. I connect the containers with veth pairs, then add the veth > > > ports to the bridges. Next, I add a dpdkvhostuser port named SRC to > > > br1, so far so good. The moment I add a dpdkvhostuser port named SNK > > > to br3, ovs-vswitchd services in br1's and br3's containers segfault. > > > Following are the backtraces from each, > > What version of OVS and DPDK are you using? > > > > > > > --br1's container--- > > > > > > [Thread debugging using libthread_db enabled] Using host libthread_db > > > library "/lib/x86_64-linux-gnu/libthread_db.so.1". > > > Core was generated by `ovs-vswitchd > > > unix:/usr/local/var/run/openvswitch/db.sock -vconsole:emer -vsyslo'. > > > Program terminated with signal SIGSEGV, Segmentation fault. > > > #0 0x5608fa0f321b in netdev_rxq_recv (rx=0x7ff13c34ee80, > > > batch=batch@entry=0x7ff1bbb4d890) at lib/netdev.c:702 > > > 702retval = rx->netdev->netdev_class->rxq_recv(rx, batch); > > > [Current thread is 1 (Thread 0x7ff1bbb4e700 (LWP 376))] > > > (gdb) bt > > > #0 0x5608fa0f321b in netdev_rxq_recv (rx=0x7ff13c34ee80, > > > batch=batch@entry=0x7ff1bbb4d890) at lib/netdev.c:702 > > > #1 0x5608fa0cce65 in dp_netdev_process_rxq_port ( > > > pmd=pmd@entry=0x7ff1bbb4f010, rxq=0x5608fb651be0, port_no=1) > > > at lib/dpif-netdev.c:3279 > > > #2 0x5608fa0cd296 in pmd_thread_main (f_=) > > > at lib/dpif-netdev.c:4145 > > > #3 0x5608fa14a836 in ovsthread_wrapper (aux_=) > > > at lib/ovs-thread.c:348 > > > #4 0x7ff1c52517fc in start_thread (arg=0x7ff1bbb4e700) > > > at pthread_create.c:465 > > > #5 0x7ff1c4815b5f in clone () > > > at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 > > > > > > --br3's container--- > > > > > > [Thread debugging using libthread_db enabled] Using host libthread_db > > > library "/lib/x86_64-linux-gnu/libthread_db.so.1". > > > Core was generated by `ovs-vswitchd > > > unix:/usr/local/var/run/openvswitch/db.sock -vconsole:emer -vsyslo'. > > > Program terminated with signal SIGSEGV, Segmentation fault. > > > #0 0x55c517e3abcb in rte_mempool_free_memchunks () [Current > > > thread is 1 (Thread 0x7f202351f300 (LWP 647))] > > > (gdb) bt > > > #0 0x55c517e3abcb in rte_mempool_free_memchunks () > > > #1 0x55c517e3ad46 in rte_mempool_free.part () > > > #2 0x55c518218b78 in dpdk_mp_free (mp=0x7f603fe66a00) > > > at lib/netdev-dpdk.c:599 > > > #3 0x55c518218ff0 in dpdk_mp_free (mp=) > > > at lib/netdev-dpdk.c:593 > > > #4 netdev_dpdk_mempool_configure (dev=0x7f1f7ffeac00) at > > > lib/netdev-dpdk.c:629 > > > #5 0x55c51821a98d in dpdk_vhost_reconfigure_helper > > (dev=0x7f1f7ffeac00) > > > at lib/netdev-dpdk.c:3599 > > > #6 0x55c51821ac8b in netdev_
