Re: [ovs-discuss] Re：Re： [HELP] Question about icmp pkt marked Invalid by userspace conntrack
Hi Timo
On Wed, Nov 13, 2019 at 1:00 AM txfh2007 wrote:
> Hi Darrell:
> I have set burst size 2 times of rate（e.g. if rate is 1Gbps, then
> burst is 2Gbps）. and have tried different rate value. the result is as
> below:
> meter rate/iperf test result: 1G / 800M
> meter rate/iperf test result: 500M / 420M
> meter rate/iperf test result: 200M / 172M
> meter rate/iperf test result: 100M / 95M
> meter rate/iperf test result:50M / 45M
> meter rate/iperf test result:10M / 9M
> It seems if rate is lower, the test result is more accurate, but for
> rate above 200M , the actual rate limit result is not as expected.
>
Can you try this patch to see the effect on precision ?
diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
index 4720ba1..01c0280 100644
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -5633,6 +5633,7 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct
dp_packet_batch *packets_,
}
meter_lock(dp, meter_id);
+now = time_usec();
meter = dp->meters[meter_id];
if (!meter) {
goto out;
Thanks Darrell
>
> Thanks
> Timo
>
>
> ：Re: [ovs-discuss] Re：Re： [HELP] Question about icmp pkt marked Invalid by
> userspace conntrack
>
>
> Hi Timo
>
>
>
> On Sun, Nov 10, 2019 at 11:58 PM txfh2007 wrote:
>
> Hi Darrell:
>
> I have tried to manual set flow table and meter action, to arrange
> meter action at the end of the flow pipeline(just before the output
> action), and delete conntrack related actions. But the iperf result is also
> around 800Mbps(meter rate is 1Gbps as below).
> Should I print any message to verify that userspace meter works as
> expected ?
>
> ovs-ofctl dump-meters br-int -O openflow13
> OFPST_METER_CONFIG reply (OF1.3) (xid=0x2):
> meter=1 kbps burst stats bands=
> type=drop rate=100 burst_size=120
>
> ovs-appctl dpif/dump-flows br-int
> recirc_id(0),in_port(5),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no),
> packets:570308, bytes:817648729, used:0.191s, flags:SP., actions:meter(0),3
> recirc_id(0),in_port(3),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no),
> packets:291956, bytes:19551792, used:0.191s, flags:SP., actions:5
>
>
> Just to recap here:
>
>
> The test is sending b/w 2 VMs attached to the same host.
>
>
>
> Now you are rechecking the base case of the test by removing the conntrack
> rules and are applying a meter rule in one direction only.
> However, the same problem is observed where 0.8 Gbps is seen vs 1 Gbps
> meter setting
> Without meter, you can get 5 Gbps.
>
>
> 1/ To investigate, try explicitly setting the burst size high to take
> burstiness out as a factor.
> 2/ Also try other meter rates which might help see where the issue is.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
>
> ：txfh2007
> ：Ben Pfaff ; ovs-discuss
> ：Re: [ovs-discuss] Re：Re： [HELP] Question about icmp pkt marked Invalid by
> userspace conntrack
>
>
> Hi Timo
>
> On Wed, Nov 6, 2019 at 9:37 PM txfh2007 wrote:
>
> Hi Darrell:
> Sorry, I forgot to tell you the attached flow is based on VM tx
> direction rate limit. So the datapath action order is conntrack -> meter ->
> forward decision -> output, For the VM rx direction rate limit, the
> datapath flow is as below, please help to check, thank you!
>
>
> For both directions, I think you want to apply the flow meter at the end
> of the pipeline.
> Can you do that and then check the numbers again.
>
>Also, for the same flow table and meter configuration, the kernel
> datapath rate limit is more accurate than userspace datapath.
> For VM rx direction rate limit:
>
>
> ct_state(-new+est-rel-rpl-inv+trk),ct_label(0/0x1),recirc_id(0x29),in_port(5),packet_type(ns=0,id=0),eth(src=fa:16:3e:33:02:d8,dst=fa:16:3e:12:d7:77),eth_type(0x0800),ipv4(dst=
> 192.168.1.10/255.255.255.248,proto=6,frag=no),tcp_flags(ack),
> packets:1031455, bytes:1481163900, used:0.149s, flags:.,
> actions:ct(zone=4),recirc(0x2a)
>
> ct_state(-new+est-rel-rpl-inv+trk),ct_label(0/0x1),recirc_id(0x2a),in_port(5),packet_type(ns=0,id=0),eth(src=fa:16:3e:33:02:d8,dst=fa:16:3e:12:d7:77),eth_type(0x0800),ipv4(src=192.168.1.8,dst=192.168.1.10,proto=6,frag=no),
> packets:1685180, bytes:2415638857, used:0.118s, flags:P., actions:meter(1),6
>
>
>
>
>
>
>
>
>
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] problem with multiple pop_mpls
> On Nov 14, 2019, at 1:46 AM, Adel Belkhiri wrote: > > Hi All, > > I have an issue with popping multiple mpls headers with ovs. The pushing via > push_mpls works fine but the popping via pop_mpls triggers a warning in the > log and the operation is not accomplished. > > |WARN|system@ovs-system: execute pop_mpls(eth_type=0x8847),pop_mpls > (eth_type=0x800),2 failed (Invalid argument) on packet mpls,vlan_tci=0x, > dl_src=ba:bd:0d:9d:c5:ee,dl_dst=ea:f5:19:d4:e2:4f,mpls_label=15,mpls_tc=0,mpls_ttl=64,mpls_bos=0,mpls_lse1=45376 > with metadata skb_priority(0),skb_mark(0), in_port(1) mtu 0 > > I'm using the last version of OVS : 2.12.90. My OpenFlow rules are the > following : > > sudo ovs-ofctl add-flow $S1 > "table=0,in_port=$S1_ETH2,eth_type=0x8847,mpls_bos=1,actions=pop_mpls:0x8847,resubmit(,0)" > sudo ovs-ofctl add-flow $S1 > "table=0,in_port=$S1_ETH2,eth_type=0x8847,mpls_bos=0,actions=pop_mpls:0x800,output:$S1_ETH1" > > > Any hint how to solve this? Why pop_mpls:0x800 ? > Many thanks. > Adele. > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] problem with multiple pop_mpls
Hi All, I have an issue with popping multiple mpls headers with ovs. The pushing via push_mpls works fine but the popping via pop_mpls triggers a warning in the log and the operation is not accomplished. |WARN|system@ovs-system: execute pop_mpls(eth_type=0x8847),pop_mpls (eth_type=0x800),2 failed (Invalid argument) on packet mpls,vlan_tci=0x, dl_src=ba:bd:0d:9d:c5:ee,dl_dst=ea:f5:19:d4:e2:4f,mpls_label=15,mpls_tc=0,mpls_ttl=64,mpls_bos=0,mpls_lse1=45376 with metadata skb_priority(0),skb_mark(0), in_port(1) mtu 0 I'm using the last version of OVS : 2.12.90. My OpenFlow rules are the following : sudo ovs-ofctl add-flow $S1 "table=0,in_port=$S1_ETH2,eth_type=0x8847,mpls_bos=1,actions=pop_mpls:0x8847,resubmit(,0)" sudo ovs-ofctl add-flow $S1 "table=0,in_port=$S1_ETH2,eth_type=0x8847,mpls_bos=0,actions=pop_mpls:0x800,output:$S1_ETH1" Any hint how to solve this? Many thanks. Adele. ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] OVS/OVN docker image for each stable release
On Tue, Nov 12, 2019 at 10:57 PM Numan Siddique wrote: > On Wed, Nov 13, 2019 at 12:02 AM Shivaram Mysore > wrote: > > > > No need to indicate "built on Ubuntu" for docker image tags. > > Alpine tag is specifically used as it used different libraries and image > size is small. Ideally, for Docker images, we should use Alpine Linux. If > OVS for Alpine is latest, then image size will be further reduced. > > > > Note thAt at the end of the day, container is just a delivery or > packaging vehicle. > > > > /Shivaram > > ::Sent from my mobile device:: > > > > On Nov 12, 2019, at 9:49 AM, aginwala wrote: > > > > > > Thanks Shivaram: > > > > On Tue, Nov 12, 2019 at 9:28 AM Shivaram Mysore < > shivaram.mys...@gmail.com> wrote: > >> > >> I am not sure why "*_debian" is used. The image should work across > OS. I have not seen use of "*_linux" as most docker images use some form > of shell scripts. > >> > > Because the container image published is ubuntu and hence we tagged it > with _debian. It doesn't indicate it will not work on rhel. If we all agree > we can remove the tags and update the readme.md on docker.io that each > container image is using ubuntu as base image. I am fine with any approach. > >> > >> Also, in my opinion, the docker image should not build OVS. If it can > add appropriate OVS packages like > https://github.com/servicefractal/ovs/blob/master/Dockerfile is better as > they are already tested. Building OVS as a part of this will cause more > testing impacts and is unnecessary. The objective is to run OVS in a > container image. I would keep it simple. > > I think the idea was to have an OVS container image with the latest > master code right Aliasgar ? Yes. E.g ovs docker image for ovs release 2.12.0 with debian/rhel will checkout v2.12.0 code from git and build it. That way source code will exist in docker image from which ovs2.12.0 will be installed on that container. > > > Getting OVS packages is good, but then the debian/ubuntu/fedora > packages should be updated as soon as OVS does a release. You mean to say e.g install with dpkg -i for latest version pushed using *2.12.0.deb for debian and skip building from source Code? So I think the open question now is; do we want to have source code in version specific container image with ovs installed from source code or we just need version specific ovs installed without source in the container? Thanks > Numan > > >> > > I think the objective is to have an image per upstream stable ovs > release and hence building it in container. Hope everyone is ok here. > >> > >> On Tue, Nov 12, 2019 at 12:51 AM aginwala wrote: > >>> > >>> Thanks Guru. > >>> > >>> On Mon, Nov 11, 2019 at 1:03 PM Guru Shetty wrote: > > > > On Mon, 11 Nov 2019 at 10:08, aginwala wrote: > > > > > > > > On Mon, Nov 11, 2019 at 9:00 AM Guru Shetty wrote: > >> > >> > >> > >> On Fri, 8 Nov 2019 at 14:41, aginwala wrote: > >>> > >>> openvswitch.ko ships default with newer kernel but if we want to > use say stt, we need to build it with respective kernel for host on which > we will run. Hence, to skip host level installation , we pack the modules > in container. > >> > >> > >> It is not clear to me. Is DKMS enabled here? Or is it that > openvswitch/ovs:2.12.0_debian_4.15.0-66-generic will only work on kernel > 4.15.0-66-generic? > >> > > > > No. Dkms is not enabled because idea is to release a new docker > image for every new kernel upgrade on compute (Not sure if dkms will help > much in container case as we are not installing on host). Do you have any > specific use case which? Yes on host with 4.15.0-66-generic. > > > It will probably be very hard to release each OVS version to so many > available kernels. How do you decide which kernel that you want to release > a image for? What is the plan here? I think it makes sense to release one > image without a kernel module packed with it. > > >>> Agree, we can't publish too many images based on different kernel > versions. Hence, I am ok with the approach you proposed by publishing > single image for each stable release leveraging host kernel modules. I have > pushed 2 debian images for each stable releases 2.11.2_debian and > 2.12.0_debian under openvswitch/ovs accordingly. I also sent the > corresponding patch https://patchwork.ozlabs.org/patch/1193372/ to > refactor the docker builds to support an option to skip kernel modules for > ovs repo so that user can choose to build/run with/without kernel modules. > Let me know further. > >>> > > > >>> > >>> > >>> On Fri, Nov 8, 2019 at 2:37 PM Guru Shetty wrote: > > > > On Fri, 8 Nov 2019 at 14:18, aginwala wrote: > > > > Hi all: > > > > > > I have pushed two images to public openvswitch org on docker.io > for ovs and ovn; > > OVS for ubuntu with 4.15 kern
Re: [ovs-discuss] OVS-bridges Troubleshooting
On Wed, 13 Nov 2019 14:02:33 +0200 aeris wrote: > Hello, > > I'm new here, so sorry if i missed something or writing to wrong > maillist > > A bit of foreword - > > We had an installation of ovs in our cloud environment and faced > networking issue with vms at random time, but mostly 3-7 minutes > after reboot(it just stops answering on every request(typical ICMP) > during time from a few seconds to a few minutes) > > Our internal network for such kind of requests looks like this - > > Physical-switch-->linux > bond-->ovs-bridge1-->ovs-bridge2-->ovs-bridge3-->VM > > With ovs-tcpdump i found that there is two-sided communication on > link to VM and between ovs-bridge1 and linux-bond, but only one-sided > communication(only icmp-replies/requests depending on from which side > ping was issued) on link between ovs-bridge2 and ovs-bridge3 and no > packet on any other interface. > > So, actually, my main question is - what instruments do u use when > troubleshooting such issues, which I in turn can use in my > implementation. And did you faced similar problems in your > environments? I see you're using ovs-tcpdump, but you haven't told us if you're using DPDK or not. Also, how are the bridges connected to each other? veth? patch ports? and last, what do you have in the flow table? One thing that you might find useful is ofproto/trace: https://developers.redhat.com/blog/2016/10/12/tracing-packets-inside-open-vswitch/ fbl ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] OVS DPDK: Failed to create memory pool for netdev
On Mon, 11 Nov 2019 14:45:13 + "Tobias Hofmann \(tohofman\) via discuss" wrote: > Hi Flavio, > > to follow up on this: I have just upgraded DPDK to 18.11 and OVS to > 2.11 and I don't see this issue anymore. Also, I don't observe any > "ring error" messages although the MTU is still at 9216 and OvS only > has 1Gb of memory. Do you have an idea which change in DPDK/OvS might > have resolved it? There are lots and lots of changes in OVS and DPDK between those versions and 2.11 is considered stable, so I am glad that you could update and that it works for you. fbl ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] OVS-bridges Troubleshooting
Hello, I'm new here, so sorry if i missed something or writing to wrong maillist A bit of foreword - We had an installation of ovs in our cloud environment and faced networking issue with vms at random time, but mostly 3-7 minutes after reboot(it just stops answering on every request(typical ICMP) during time from a few seconds to a few minutes) Our internal network for such kind of requests looks like this - Physical-switch-->linux bond-->ovs-bridge1-->ovs-bridge2-->ovs-bridge3-->VM With ovs-tcpdump i found that there is two-sided communication on link to VM and between ovs-bridge1 and linux-bond, but only one-sided communication(only icmp-replies/requests depending on from which side ping was issued) on link between ovs-bridge2 and ovs-bridge3 and no packet on any other interface. So, actually, my main question is - what instruments do u use when troubleshooting such issues, which I in turn can use in my implementation. And did you faced similar problems in your environments? ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Re： Re：Re： [HELP] Question about icmp pkt marked Invalid by userspace conntrack
Hi Darrell: I have set burst size 2 times of rate（e.g. if rate is 1Gbps, then burst is 2Gbps）. and have tried different rate value. the result is as below: meter rate/iperf test result: 1G / 800M meter rate/iperf test result: 500M / 420M meter rate/iperf test result: 200M / 172M meter rate/iperf test result: 100M / 95M meter rate/iperf test result:50M / 45M meter rate/iperf test result:10M / 9M It seems if rate is lower, the test result is more accurate, but for rate above 200M , the actual rate limit result is not as expected. Thanks Timo ：Re: [ovs-discuss] Re：Re： [HELP] Question about icmp pkt marked Invalid by userspace conntrack Hi Timo On Sun, Nov 10, 2019 at 11:58 PM txfh2007 wrote: Hi Darrell: I have tried to manual set flow table and meter action, to arrange meter action at the end of the flow pipeline(just before the output action), and delete conntrack related actions. But the iperf result is also around 800Mbps(meter rate is 1Gbps as below). Should I print any message to verify that userspace meter works as expected ? ovs-ofctl dump-meters br-int -O openflow13 OFPST_METER_CONFIG reply (OF1.3) (xid=0x2): meter=1 kbps burst stats bands= type=drop rate=100 burst_size=120 ovs-appctl dpif/dump-flows br-int recirc_id(0),in_port(5),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no), packets:570308, bytes:817648729, used:0.191s, flags:SP., actions:meter(0),3 recirc_id(0),in_port(3),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no), packets:291956, bytes:19551792, used:0.191s, flags:SP., actions:5 Just to recap here: The test is sending b/w 2 VMs attached to the same host. Now you are rechecking the base case of the test by removing the conntrack rules and are applying a meter rule in one direction only. However, the same problem is observed where 0.8 Gbps is seen vs 1 Gbps meter setting Without meter, you can get 5 Gbps. 1/ To investigate, try explicitly setting the burst size high to take burstiness out as a factor. 2/ Also try other meter rates which might help see where the issue is. -- ：txfh2007 ：Ben Pfaff ; ovs-discuss ：Re: [ovs-discuss] Re：Re： [HELP] Question about icmp pkt marked Invalid by userspace conntrack Hi Timo On Wed, Nov 6, 2019 at 9:37 PM txfh2007 wrote: Hi Darrell: Sorry, I forgot to tell you the attached flow is based on VM tx direction rate limit. So the datapath action order is conntrack -> meter -> forward decision -> output, For the VM rx direction rate limit, the datapath flow is as below, please help to check, thank you! For both directions, I think you want to apply the flow meter at the end of the pipeline. Can you do that and then check the numbers again. Also, for the same flow table and meter configuration, the kernel datapath rate limit is more accurate than userspace datapath. For VM rx direction rate limit: ct_state(-new+est-rel-rpl-inv+trk),ct_label(0/0x1),recirc_id(0x29),in_port(5),packet_type(ns=0,id=0),eth(src=fa:16:3e:33:02:d8,dst=fa:16:3e:12:d7:77),eth_type(0x0800),ipv4(dst=192.168.1.10/255.255.255.248,proto=6,frag=no),tcp_flags(ack), packets:1031455, bytes:1481163900, used:0.149s, flags:., actions:ct(zone=4),recirc(0x2a) ct_state(-new+est-rel-rpl-inv+trk),ct_label(0/0x1),recirc_id(0x2a),in_port(5),packet_type(ns=0,id=0),eth(src=fa:16:3e:33:02:d8,dst=fa:16:3e:12:d7:77),eth_type(0x0800),ipv4(src=192.168.1.8,dst=192.168.1.10,proto=6,frag=no), packets:1685180, bytes:2415638857, used:0.118s, flags:P., actions:meter(1),6 ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
