Thanks for the test scenarios. It seems to be a problem related to dp_hash when in_port=CONTROLLER. I guess it would work if you specify selection_method=hash in the group. Could you try that? By default it is using dp_hash, which requires recirculation as the trace output shows. It is possible that the dp_hash didn't get any value (0) from kernel datapath so that it would keep recirculating, something similar as [0]. However, it needs to be verified by ovs-dpctl dump-flows (instead of ovs-ofctl dump-flows) and dmesg | tail. I didn't find it included in your logs. This command needs to be executed immediately after the packet-out before the cache is cleared.
Although I suspect it could be the recirculation problem, I don't think it is the exact same root cause of [0] because in your case the packet doesn't need to be slowpathed, so I guess the patch would not help for your case, and I can't explain yet why in_port=LOCAL works. I will stop speculating before getting more information. [0] - https://patchwork.ozlabs.org/project/openvswitch/patch/[email protected]/ On Thu, May 21, 2020 at 9:58 PM 陳奕任 <[email protected]> wrote: > Hello, > > Thanks for your quick response. The following contains 4 scenarios (0 ~ > 3): > > scenario 0: packet-out to "select" group and set in_port=CONTROLLER > scenario 1: packet-out to "fast-failover" group and set > in_port=CONTROLLER > scenario 2: packet-out to "SELECT" group and set in_port=LOCAL > scenario 3: packet-out to "TABLE" port and set in_port=CONTROLLER > > In scenario 1 and 2, my composed packets are sent, but they cannot be sent > in scenario 0 and 3. > > ==== > root@host1# ovs-ofctl -O OpenFlow15 show adapter > OFPT_FEATURES_REPLY (OF1.5) (xid=0x2): dpid:0000bad0babe0051 > n_tables:254, n_buffers:0 > capabilities: FLOW_STATS TABLE_STATS PORT_STATS GROUP_STATS QUEUE_STATS > BUNDLES > OFPST_PORT_DESC reply (OF1.5) (xid=0x3): > 1(eth2): addr:06:ef:e9:1a:0c:96 > config: 0 > state: LIVE > current: 10GB-FD COPPER > speed: 10000 Mbps now, 0 Mbps max > LOCAL(adapter): addr:a2:2b:ed:d2:1b:4b > config: 0 > state: LIVE > speed: 0 Mbps now, 0 Mbps max > OFPT_GET_CONFIG_REPLY (OF1.5) (xid=0x9): frags=normal miss_send_len=0 > > > *====== scenario 0: packet-out to "select" group and set > in_port=CONTROLLER ====== * > ======== part of the code (OpenFlow 1.5)======== > buffer_id=ofproto.OFP_NO_BUFFER > in_port = ofproto.OFPP_CONTROLLER > actions = [ parser.OFPActionGroup(1) ] > msg = parser.OFPPacketOut(datapath=datapath, buffer_id=buffer_id, > match=parser.OFPMatch(in_port=in_port), actions=actions, data=packet.data) > datapath.send_msg(msg) > > ======== after RyuApp runs, before packet-out ======== > root@host1# ovs-ofctl -O OpenFlow15 dump-flows adapter > cookie=0x0, duration=11.384s, table=0, n_packets=0, n_bytes=0, > idle_age=11, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080 > actions=group:1 > cookie=0x0, duration=11.384s, table=0, n_packets=0, n_bytes=0, > idle_age=11, priority=4096,tcp,in_port=eth2,tp_dst=8080 > actions=CONTROLLER:65509 > > root@host1# ovs-ofctl -O OpenFlow15 dump-groups adapter > OFPST_GROUP_DESC reply (OF1.5) (xid=0x2): > group_id=1,type=select,bucket=bucket_id:0,actions=output:eth2 > > root@host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter > OFPST_GROUP reply (OF1.5) (xid=0x6): > > > group_id=1,duration=94.104s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0 > > root@host1# ovs-appctl ofproto/trace-packet-out adapter > 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1' > Flow: > tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0 > > bridge("adapter") > ----------------- > group:1 > -> no live bucket > > Final flow: unchanged > Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no > Datapath actions: hash(l4(0)),recirc(0x1) > > ======== after packet-out (telnet to port 8080 from other host) ======== > root@host1# ovs-ofctl -O OpenFlow15 dump-flows adapter > cookie=0x0, duration=148.805s, table=0, n_packets=0, n_bytes=0, > idle_age=148, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080 > actions=group:1 > cookie=0x0, duration=148.806s, table=0, n_packets=4, n_bytes=296, > idle_age=6, priority=4096,tcp,in_port=eth2,tp_dst=8080 > actions=CONTROLLER:65509 > > root@host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter > OFPST_GROUP reply (OF1.5) (xid=0x6): > > > group_id=1,duration=208.300s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0 > > root@host1# ovs-appctl ofproto/trace-packet-out adapter > 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1' > Flow: > tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0 > > bridge("adapter") > ----------------- > group:1 > -> no live bucket > > Final flow: unchanged > Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no > Datapath actions: hash(l4(0)),recirc(0x18) > > > > *====== scenario 1: packet-out to "fast-failover" group and set > in_port=CONTROLLER ====== * > ======== part of the code (OpenFlow 1.5)======== > buffer_id=ofproto.OFP_NO_BUFFER > in_port = ofproto.OFPP_CONTROLLER > actions = [ parser.OFPActionGroup(1) ] > msg = parser.OFPPacketOut(datapath=datapath, buffer_id=buffer_id, > match=parser.OFPMatch(in_port=in_port), actions=actions, data=packet.data) > datapath.send_msg(msg) > > ======== after RyuApp runs, before packet-out ======== > root@host1# ovs-ofctl -O OpenFlow15 dump-flows adapter > cookie=0x0, duration=20.749s, table=0, n_packets=0, n_bytes=0, > idle_age=20, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080 > actions=group:1 > cookie=0x0, duration=20.748s, table=0, n_packets=0, n_bytes=0, > idle_age=20, priority=4096,tcp,in_port=eth2,tp_dst=8080 > actions=CONTROLLER:65509 > > root@host1# ovs-ofctl -O OpenFlow15 dump-groups adapter > OFPST_GROUP_DESC reply (OF1.5) (xid=0x2): > group_id=1,type=ff,bucket=bucket_id:0,watch_port:eth2,actions=output:eth2 > > root@host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter > OFPST_GROUP reply (OF1.5) (xid=0x6): > > > group_id=1,duration=173.081s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0 > > root@host1# ovs-appctl ofproto/trace-packet-out adapter > 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1' > Flow: > tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0 > > bridge("adapter") > ----------------- > group:1 > -> using bucket 0 > bucket 0 > output:1 > > Final flow: unchanged > Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no > Datapath actions: 2 > > ======== after packet-out (telnet to port 8080 from other host) ======== > root@host1# ovs-ofctl -O OpenFlow15 dump-flows adapter > cookie=0x0, duration=333.717s, table=0, n_packets=0, n_bytes=0, > idle_age=333, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080 > actions=group:1 > cookie=0x0, duration=333.716s, table=0, n_packets=5, n_bytes=346, > idle_age=32, priority=4096,tcp,in_port=eth2,tp_dst=8080 > actions=CONTROLLER:65509 > > root@host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter > OFPST_GROUP reply (OF1.5) (xid=0x6): > > > group_id=1,duration=365.835s,ref_count=1,packet_count=3,byte_count=194,bucket0:packet_count=3,byte_count=194 > > root@host1# ovs-appctl ofproto/trace-packet-out adapter > 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1' > Flow: > tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0 > > bridge("adapter") > ----------------- > group:1 > -> using bucket 0 > bucket 0 > output:1 > > Final flow: unchanged > Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no > Datapath actions: 2 > > > > *====== scenario 2: packet-out to "SELECT" group and set in_port=LOCAL > ====== * > ======== part of the code (OpenFlow 1.5)======== > buffer_id=ofproto.OFP_NO_BUFFER > in_port = ofproto.OFPP_LOCAL > actions = [ parser.OFPActionGroup(1) ] > msg = parser.OFPPacketOut(datapath=datapath, buffer_id=buffer_id, > match=parser.OFPMatch(in_port=in_port), actions=actions, data=packet.data) > datapath.send_msg(msg) > > ======== after RyuApp runs, before packet-out ======== > root@host1# ovs-ofctl -O OpenFlow15 dump-flows adapter > cookie=0x0, duration=65.134s, table=0, n_packets=0, n_bytes=0, > idle_age=65, priority=8192,tcp,in_port=LOCAL,tp_src=8080 actions=group:1 > cookie=0x0, duration=65.134s, table=0, n_packets=0, n_bytes=0, > idle_age=65, priority=4096,tcp,in_port=eth2,tp_dst=8080 > actions=CONTROLLER:65509 > > root@host1# ovs-ofctl -O OpenFlow15 dump-groups adapter > OFPST_GROUP_DESC reply (OF1.5) (xid=0x2): > group_id=1,type=select,bucket=bucket_id:0,actions=output:eth2 > > root@host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter > OFPST_GROUP reply (OF1.5) (xid=0x6): > > > group_id=1,duration=172.172s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0 > > root@host1# ovs-appctl ofproto/trace-packet-out adapter > 'tcp,in_port=LOCAL,tp_src=8080' 'group:1' > Flow: > tcp,in_port=LOCAL,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0 > > bridge("adapter") > ----------------- > group:1 > -> no live bucket > > Final flow: unchanged > Megaflow: recirc_id=0,eth,ip,in_port=LOCAL,nw_frag=no > Datapath actions: hash(l4(0)),recirc(0x1a) > > ======== after packet-out (telnet to port 8080 from other host) ======== > root@host1# ovs-ofctl -O OpenFlow15 dump-flows adapter > cookie=0x0, duration=296.942s, table=0, n_packets=0, n_bytes=0, > idle_age=296, priority=8192,tcp,in_port=LOCAL,tp_src=8080 actions=group:1 > cookie=0x0, duration=296.942s, table=0, n_packets=5, n_bytes=346, > idle_age=17, priority=4096,tcp,in_port=eth2,tp_dst=8080 > actions=CONTROLLER:65509 > > root@host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter > OFPST_GROUP reply (OF1.5) (xid=0x6): > > > group_id=1,duration=334.464s,ref_count=1,packet_count=3,byte_count=194,bucket0:packet_count=3,byte_count=194 > > root@host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter > OFPST_GROUP reply (OF1.5) (xid=0x6): > > > group_id=1,duration=334.464s,ref_count=1,packet_count=3,byte_count=194,bucket0:packet_count=3,byte_count=194 > > root@host1# ovs-appctl ofproto/trace-packet-out adapter > 'tcp,in_port=LOCAL,tp_src=8080' 'group:1' > Flow: > tcp,in_port=LOCAL,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0 > > bridge("adapter") > ----------------- > group:1 > -> no live bucket > > Final flow: unchanged > Megaflow: recirc_id=0,eth,ip,in_port=LOCAL,nw_frag=no > Datapath actions: hash(l4(0)),recirc(0x1f) > > > > *====== scenario 3: packet-out to "TABLE" port and set in_port=CONTROLLER > ====== * > ======== part of the code (OpenFlow 1.5)======== > buffer_id=ofproto.OFP_NO_BUFFER > in_port = ofproto.OFPP_CONTROLLER > actions = [ parser.OFPActionOutput(ofproto.OFPP_TABLE) ] > msg = parser.OFPPacketOut(datapath=datapath, buffer_id=buffer_id, > match=parser.OFPMatch(in_port=in_port), actions=actions, data=packet.data) > datapath.send_msg(msg) > > ======== after RyuApp runs, before packet-out ======== > root@host1# ovs-ofctl -O OpenFlow15 dump-flows adapter > cookie=0x0, duration=21.992s, table=0, n_packets=0, n_bytes=0, > idle_age=21, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080 > actions=group:1 > cookie=0x0, duration=21.992s, table=0, n_packets=0, n_bytes=0, > idle_age=21, priority=4096,tcp,in_port=eth2,tp_dst=8080 > actions=CONTROLLER:65509 > > root@host1# ovs-ofctl -O OpenFlow15 dump-groups adapter > OFPST_GROUP_DESC reply (OF1.5) (xid=0x2): > group_id=1,type=select,bucket=bucket_id:0,actions=output:eth2 > > root@host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter > OFPST_GROUP reply (OF1.5) (xid=0x6): > > > group_id=1,duration=93.852s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0 > > root@host1# ovs-appctl ofproto/trace-packet-out adapter > 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1' > Flow: > tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0 > > bridge("adapter") > ----------------- > group:1 > -> no live bucket > > Final flow: unchanged > Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no > Datapath actions: hash(l4(0)),recirc(0x21) > > ======== after packet-out (telnet to port 8080 from other host) ======== > root@host1# ovs-ofctl -O OpenFlow15 dump-flows adapter > cookie=0x0, duration=242.223s, table=0, n_packets=5, n_bytes=370, > idle_age=2, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080 actions=group:1 > cookie=0x0, duration=242.223s, table=0, n_packets=5, n_bytes=370, > idle_age=2, priority=4096,tcp,in_port=eth2,tp_dst=8080 > actions=CONTROLLER:65509 > > root@host1# ovs-ofctl -O OpenFlow15 dump-groups adapter > OFPST_GROUP_DESC reply (OF1.5) (xid=0x2): > group_id=1,type=select,bucket=bucket_id:0,actions=output:eth2 > > root@host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter > OFPST_GROUP reply (OF1.5) (xid=0x6): > > > group_id=1,duration=279.572s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0 > > root@host1# ovs-appctl ofproto/trace-packet-out adapter > 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1' > Flow: > tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0 > > bridge("adapter") > ----------------- > group:1 > -> no live bucket > > Final flow: unchanged > Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no > Datapath actions: hash(l4(0)),recirc(0x2c) > > > > > > Han Zhou <[email protected]> 於 2020年5月22日 週五 上午8:56寫道： > >> >> >> >> On Thu, May 21, 2020 at 12:20 PM Ben Pfaff <[email protected]> wrote: >> > >> > On Thu, May 21, 2020 at 10:50:21PM +0800, 陳奕任 wrote: >> > > Hello, >> > > >> > > >> > > ==== What you did that make the problem appear. ==== >> > > I use Ryu as the OpenFlow controller. >> > > I try to use Packet-Out with Actions=ActionGroup(1) and >> in_port=Controller, >> > > where the type of group 1 is "select". >> > > There is only one bucket and the bucket has only one ActionOutput to a >> > > physical port, because I am doing a small test for a project. >> > > >> > > I use OpenFlow13 firstly, and then I try to use OpenFlow15. both of >> them >> > > have the same problem. >> > > >> > > ==== What you expected to happen. ==== >> > > My composed packets should be sent >> > > >> > > ==== What actually happened. ==== >> > > no packet is sent. >> > > >> > > ==== other trials (other information that I think might be relevant.) >> ==== >> > > (1) If I change the group type to "FastFailover", it works. >> > > (2) if I change the in_port to "Local", it works. >> > > (3) if I change the Action=ActionOutput(Table), and add a flow entry >> > > which match is "in_port=controller" and "actions=group(1)", it doesn't >> > > work. I use " ovs-ofctl dump-group-stats" to check packet_count and >> > > byte_count, but they are not increased. >> > > However, the packet_count and byte_count are increased on the flow >> entry >> > > when I use ovs-ofctl dump-flows to check them. >> > >> > That's quite odd. Have you tried ofproto/trace-packet-out to see what's >> > going on? See ovs-vswitchd(8) for documentation. >> >> In addition, could you attach the output of "ovs-dpctl dump-flows" and >> "ovs-ofctl dump-groups" when it happens? >> >
_______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
