On 22 Mar 2023, at 17:19, Alex Zetaeffesse wrote:
> For the bridge's interfaces' list, I guess the shorter the output the
> better...
>
> root@pve:~# ovs-vsctl list-ifaces vmbr1
> enp6s0f0
> enp7s0f0
> sv_z1ad0101
> sv_z1ad0102
> sv_z1ad0103
> sv_z1ad0104
> sv_z1ad4064
Use ovs-vsctl show, as it shows the bridge, and all the ports with the
relations (and some additional config).
For example:
ovs-vsctl show
19ff182e-79a7-4f74-ae4a-5bc217d2c558
Bridge br1
Port br1
Interface br1
type: internal
Bridge ovs_pvp_br0
datapath_type: netdev
Port dpdk0p1
Interface dpdk0p1
type: dpdk
options: {dpdk-devargs="0000:17:00.1", n_rxq="2"}
Port dpdk0p0
Interface dpdk0p0
type: dpdk
options: {dpdk-devargs="0000:17:00.0", n_rxq="2"}
Port vhost0
Interface vhost0
type: dpdkvhostuserclient
options: {n_rxq="2", vhost-server-path="/tmp/vhost-sock0"}
Port ovs_pvp_br0
Interface ovs_pvp_br0
type: internal
ovs_version: "3.1.1"
> Alex
>
> On Wed, Mar 22, 2023 at 5:14 PM Alex Zetaeffesse <[email protected]> wrote:
>
>> The issue is that yet I confuse ports and interfaces.
>>
>> root@pve:~# ovs-vsctl del-port vmbr1 enp7s0f0
>> ovs-vsctl: no port named enp7s0f0
>> root@pve:~# ovs-vsctl add-port vmbr1 enp7s0f0
>> ovs-vsctl: cannot create a port named enp7s0f0 because an interface named
>> enp7s0f0 already exists on bridge vmbr1
>>
>> The trick that removed the physical interfaces was
>>
>> root@pve:~# ovs-vsctl --with-iface del-port vmbr1 enp7s0f0
>> root@pve:~# ovs-ofctl show vmbr1
>> OFPT_FEATURES_REPLY (xid=0x2): dpid:0000f6fe01a1e94d
>> n_tables:254, n_buffers:0
>> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
>> actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src
>> mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
>> 2(sv_z1ad0101): addr:1e:9a:25:9b:22:0a
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> 3(sv_z1ad0102): addr:ba:46:f5:b8:4a:63
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> 4(sv_z1ad0103): addr:56:75:a3:a8:f7:89
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> 5(sv_z1ad0104): addr:9e:a9:30:0f:a2:53
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> 6(sv_z1ad4064): addr:26:de:29:e0:8a:12
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> LOCAL(vmbr1): addr:f6:fe:01:a1:e9:4d
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
>>
>> and along with enp7s0f0, also enp6s0f0 and bond0 got removed from vmbr1
>> and bond0 was deleted :-/
>>
>> root@pve:~#
>> root@pve:~# ip link show bond0
>> Device "bond0" does not exist.
>> root@pve:~# ip link show dev bond0
>> Device "bond0" does not exist.
>> root@pve:~# ip link show dev enp7s0f0
>> 6: enp7s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
>> state UP mode DEFAULT group default qlen 1000
>> link/ether 00:1c:c4:47:63:33 brd ff:ff:ff:ff:ff:ff
>> root@pve:~# ip link show dev enp6s0f0
>> 4: enp6s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
>> state UP mode DEFAULT group default qlen 1000
>> link/ether 00:1c:c4:47:63:31 brd ff:ff:ff:ff:ff:ff
>>
>> I then added back the bond (bond0) to the bridge by issuing the following
>> command
>>
>> ovs-vsctl add-bond vmbr1 bond0 enp6s0f0 enp7s0f0
>>
>> but just the physical interfaces were added and not the intf bond0
>>
>> root@pve:~# ovs-ofctl show vmbr1
>> OFPT_FEATURES_REPLY (xid=0x2): dpid:0000001cc4476331
>> n_tables:254, n_buffers:0
>> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
>> actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src
>> mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
>> 2(sv_z1ad0101): addr:1e:9a:25:9b:22:0a
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> 3(sv_z1ad0102): addr:ba:46:f5:b8:4a:63
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> 4(sv_z1ad0103): addr:56:75:a3:a8:f7:89
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> 5(sv_z1ad0104): addr:9e:a9:30:0f:a2:53
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> 6(sv_z1ad4064): addr:26:de:29:e0:8a:12
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> 10(enp7s0f0): addr:00:1c:c4:47:63:33
>> config: 0
>> state: 0
>> current: 1GB-FD COPPER AUTO_NEG
>> advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
>> supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
>> speed: 1000 Mbps now, 1000 Mbps max
>> 11(enp6s0f0): addr:00:1c:c4:47:63:31
>> config: 0
>> state: 0
>> current: 1GB-FD COPPER AUTO_NEG
>> advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
>> supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
>> speed: 1000 Mbps now, 1000 Mbps max
>> LOCAL(vmbr1): addr:00:1c:c4:47:63:31
>> config: 0
>> state: 0
>> speed: 0 Mbps now, 0 Mbps max
>> OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
>>
>> Alex
>>
>>
>> On Wed, Mar 22, 2023 at 4:02 PM Eelco Chaudron <[email protected]>
>> wrote:
>>
>>>
>>>
>>> On 22 Mar 2023, at 15:57, Alex Zetaeffesse wrote:
>>>
>>>> Hi Eelco,
>>>>
>>>> See my answers in-line.
>>>>
>>>> On Wed, Mar 22, 2023 at 2:58 PM Eelco Chaudron <[email protected]>
>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On 22 Mar 2023, at 14:42, Alex Zetaeffesse wrote:
>>>>>
>>>>>> I will do my best in describing the goal and the issue
>>>>>> The ports connected to the bridge (I shrunk the output for
>>> readability)
>>>>> are
>>>>>> the following
>>>>>>
>>>>>> root@pve:~# ovs-ofctl show vmbr1
>>>>>> [CUT]
>>>>>> 2(sv_z1ad0101): addr:1e:9a:25:9b:22:0a
>>>>>> 3(sv_z1ad0102): addr:ba:46:f5:b8:4a:63
>>>>>> 4(sv_z1ad0103): addr:56:75:a3:a8:f7:89
>>>>>> 5(sv_z1ad0104): addr:9e:a9:30:0f:a2:53
>>>>>> 6(sv_z1ad4064): addr:26:de:29:e0:8a:12
>>>>>> 7(enp6s0f0): addr:00:1c:c4:47:63:31
>>>>>> 8(enp7s0f0): addr:00:1c:c4:47:63:33
>>>>>> 9(bond0): addr:0a:aa:d5:40:6d:9a
>>>>>> LOCAL(vmbr1): addr:00:1c:c4:47:63:31
>>>>>> OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
>>>>>>
>>>>>> First of all the ports enps6s0f0 e enps7s0f0 (the physical memebers of
>>>>>> bond0) should not be there.
>>>>>
>>>>> I would suggest creating a kernel bond with the enp6s0f0 and enp7s0f0
>>>>> members, and only adding that bond.
>>>>>
>>>>
>>>> Indeed, I'm trying to remove them from vmbr1 and I used these commands
>>> below
>>>>
>>>> ovs-vsctl --if-exists del-port vmbr1 enp6s0f0
>>>> ovs-vsctl --if-exists del-port vmbr1 enp7s0f0
>>>
>>> These commands should do the trick, try without the --if-exists. You can
>>> also check the ovs-vswitchd.log to see what happens,
>>> /var/log/openvswitch/ovs-vswitchd.log.
>>>
>>> I guess if they are removed successfully, maybe there is some script in
>>> the background re-adding them?
>>>
>>>> ovs-vsctl --id=@enp6s0f0 get Interface enp6s0f0 -- remove Port vmbr1
>>>> interfaces @enp6s0f0
>>>> ovs-vsctl --id=@enp7s0f0 get Interface enp7s0f0 -- remove Port vmbr1
>>>> interfaces @enp7s0f0
>>>>
>>>> but then "ovs-ofctl show vmbr1" shows they are still there.
>>>>
>>>> Any idea on how to get it done?
>>>>
>>>>
>>>>
>>>>>> The goal is to have just sv_xxxxx and bond0. Bond0 is supposed to
>>> receive
>>>>>> 802.1ad traffic i.e. with ethertype 0x88a8.
>>>>>> Then frames should be forwarded to each interface sv_xxxxx based on
>>> the
>>>>>> 802.1ad tag, i.e. frames tagged with the tag 101 should be forwarded
>>> to
>>>>> the
>>>>>> port q1ad0101, frames with tag 102 should be forwarded to the port
>>>>>> q1ad0102, and so on.
>>>>>
>>>>> Your sv_xx ports should be native tagged ports, for details see the
>>>>> https://docs.openvswitch.org/en/latest/faq/vlan/ documentation.
>>>>>
>>>>> Your bond port should be a trunk port.
>>>>>
>>>>> If you use the default NORMAL rule in openflow (check ovs-ofctl
>>> dump-flows
>>>>> vmbr1), this should work just fine. You can see the FDB table with the
>>>>> right VLAN tags (see ovs-appctl fdb/xxxx commands).
>>>>>
>>>>
>>>> Ok, what I thought, though I have no idea on how to write the rule, I
>>> will
>>>> practice and get back to the list.
>>>>
>>>> Thanks!
>>>>
>>>> Alex
>>>>
>>>>
>>>>>
>>>>>> I think I have to remove the physical ones from the vmbr1 and possibly
>>>>> add
>>>>>> the right flow. I don't think is the right way to do it, everything
>>>>> should
>>>>>> be accomplished through the ProxMox Interface, but at least I will
>>> have a
>>>>>> working scenario and discuss it in the ProxMox's forum.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Alex
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 22, 2023 at 10:54 AM Eelco Chaudron <[email protected]>
>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 22 Mar 2023, at 10:49, Alex Zetaeffesse wrote:
>>>>>>>
>>>>>>>> Thank you Eelco,
>>>>>>>>
>>>>>>>> the thing is that I'm troubleshooting connectivity issues and the
>>>>> problem
>>>>>>>> for the moment seems right in vmbr1.
>>>>>>>> Hence I'm trying to understand if vmbr1 works as expected.
>>>>>>>
>>>>>>> For OVS to work the bridge does not need to be up. Traffic should
>>> just
>>>>>>> work :)
>>>>>>>
>>>>>>> What are you trying to do, and does not work?
>>>>>>>
>>>>>>> You can check the following thinks:
>>>>>>>
>>>>>>> - Are the ports configured as needed (ovs-vsctl show)
>>>>>>> - Are you OpenFlow rules as needed (ovs-ofctl dump-flows <br>)
>>>>>>> - If traffic is not working, check the dp rules (when you are
>>> actively
>>>>>>> trying to send traffic as these rules time out after 10 seconds).
>>>>>>> ovs-appctl dpctl/dump-flows
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Eelco
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On Wed, Mar 22, 2023 at 10:42 AM Eelco Chaudron <
>>> [email protected]>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 21 Mar 2023, at 23:42, Alex Zetaeffesse via discuss wrote:
>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> First of all, let me be clear on the fact that I'm the first who
>>>>>>> doesn't
>>>>>>>>>> want to run a mixed environment :-)
>>>>>>>>>>
>>>>>>>>>> In my ProxMox environment I have the following:
>>>>>>>>>>
>>>>>>>>>> root@pve:~# ip link show dev vmbr1
>>>>>>>>>> 11: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>> noqueue
>>>>>>> state
>>>>>>>>>> UNKNOWN mode DEFAULT group default qlen 1000
>>>>>>>>>> link/ether 00:1c:c4:47:63:31 brd ff:ff:ff:ff:ff:ff
>>>>>>>>>
>>>>>>>>> This is the same on my Fedora and RHEL systems. The bridge show
>>> DOWN
>>>>> if
>>>>>>> I
>>>>>>>>> bring it down, and UNKNOWN state when it’s up.
>>>>>>>>>
>>>>>>>>> [vagrant@f35 ~]$ sudo ovs-vsctl add-br br1
>>>>>>>>> [vagrant@f35 ~]$ ip link show bro
>>>>>>>>> 6: br1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>>>>>>> DEFAULT
>>>>>>>>> group default qlen 1000
>>>>>>>>> link/ether c6:c4:18:57:ba:48 brd ff:ff:ff:ff:ff:ff
>>>>>>>>> [vagrant@f35 ~]$ sudo ip link set br1 up
>>>>>>>>> [vagrant@f35 ~]$ ip link show br1
>>>>>>>>> 6: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
>>> state
>>>>>>>>> UNKNOWN mode DEFAULT group default qlen 1000
>>>>>>>>> link/ether c6:c4:18:57:ba:48 brd ff:ff:ff:ff:ff:ff
>>>>>>>>>
>>>>>>>>> This might be a bug no one ever noticed :)
>>>>>>>>>
>>>>>>>>> //Eelco
>>>>>>>>>
>>>>>>>>>> root@pve:~# ovs-ofctl show vmbr1
>>>>>>>>>> OFPT_FEATURES_REPLY (xid=0x2): dpid:0000001cc4476331
>>>>>>>>>> n_tables:254, n_buffers:0
>>>>>>>>>> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS
>>>>>>> ARP_MATCH_IP
>>>>>>>>>> actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan
>>>>> mod_dl_src
>>>>>>>>>> mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
>>>>>>>>>> 2(sv_z1ad0101): addr:1e:9a:25:9b:22:0a
>>>>>>>>>> config: 0
>>>>>>>>>> state: 0
>>>>>>>>>> speed: 0 Mbps now, 0 Mbps max
>>>>>>>>>> 3(sv_z1ad0102): addr:ba:46:f5:b8:4a:63
>>>>>>>>>> config: 0
>>>>>>>>>> state: 0
>>>>>>>>>> speed: 0 Mbps now, 0 Mbps max
>>>>>>>>>> 4(sv_z1ad0103): addr:56:75:a3:a8:f7:89
>>>>>>>>>> config: 0
>>>>>>>>>> state: 0
>>>>>>>>>> speed: 0 Mbps now, 0 Mbps max
>>>>>>>>>> 5(sv_z1ad0104): addr:9e:a9:30:0f:a2:53
>>>>>>>>>> config: 0
>>>>>>>>>> state: 0
>>>>>>>>>> speed: 0 Mbps now, 0 Mbps max
>>>>>>>>>> 6(sv_z1ad4064): addr:26:de:29:e0:8a:12
>>>>>>>>>> config: 0
>>>>>>>>>> state: 0
>>>>>>>>>> speed: 0 Mbps now, 0 Mbps max
>>>>>>>>>> 7(enp6s0f0): addr:00:1c:c4:47:63:31
>>>>>>>>>> config: 0
>>>>>>>>>> state: 0
>>>>>>>>>> current: 1GB-FD COPPER AUTO_NEG
>>>>>>>>>> advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER
>>>>>>> AUTO_NEG
>>>>>>>>>> supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER
>>>>>>> AUTO_NEG
>>>>>>>>>> speed: 1000 Mbps now, 1000 Mbps max
>>>>>>>>>> 8(enp7s0f0): addr:00:1c:c4:47:63:33
>>>>>>>>>> config: 0
>>>>>>>>>> state: 0
>>>>>>>>>> current: 1GB-FD COPPER AUTO_NEG
>>>>>>>>>> advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER
>>>>>>> AUTO_NEG
>>>>>>>>>> supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER
>>>>>>> AUTO_NEG
>>>>>>>>>> speed: 1000 Mbps now, 1000 Mbps max
>>>>>>>>>> 9(bond0): addr:0a:aa:d5:40:6d:9a
>>>>>>>>>> config: 0
>>>>>>>>>> state: 0
>>>>>>>>>> speed: 0 Mbps now, 0 Mbps max
>>>>>>>>>> LOCAL(vmbr1): addr:00:1c:c4:47:63:31
>>>>>>>>>> config: 0
>>>>>>>>>> state: 0
>>>>>>>>>> speed: 0 Mbps now, 0 Mbps max
>>>>>>>>>> OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
>>>>>>>>>> root@pve:~# ovs-vsctl --version
>>>>>>>>>> ovs-vsctl (Open vSwitch) 2.15.0
>>>>>>>>>> DB Schema 8.2.0
>>>>>>>>>>
>>>>>>>>>> Is there any significant information in the ovs-dpctl output to
>>> see
>>>>>>> where
>>>>>>>>>> the problem might be?
>>>>>>>>>> Or what other command may I use to spot the issue?
>>>>>>>>>>
>>>>>>>>>> What surprises me is that two ports of the bridge vmbr1 have
>>>>> ovs-system
>>>>>>>>> as
>>>>>>>>>> master that is in DOWN state.
>>>>>>>>>>
>>>>>>>>>> root@pve:~# ip link show dev enp6s0f0
>>>>>>>>>> 4: enp6s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>>>>>> pfifo_fast
>>>>>>>>>> master ovs-system state UP mode DEFAULT group default qlen 1000
>>>>>>>>>> link/ether 00:1c:c4:47:63:31 brd ff:ff:ff:ff:ff:ff
>>>>>>>>>> root@pve:~# ip link show dev enp7s0f0
>>>>>>>>>> 6: enp7s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>>>>>> pfifo_fast
>>>>>>>>>> master ovs-system state UP mode DEFAULT group default qlen 1000
>>>>>>>>>> link/ether 00:1c:c4:47:63:33 brd ff:ff:ff:ff:ff:ff
>>>>>>>>>> root@pve:~# ip link show dev ovs-system
>>>>>>>>>> 8: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state
>>> DOWN
>>>>>>> mode
>>>>>>>>>> DEFAULT group default qlen 1000
>>>>>>>>>> link/ether f6:c6:45:e5:39:98 brd ff:ff:ff:ff:ff:ff
>>>>>>>>>>
>>>>>>>>>> Alex
>>>>>>>>>> _______________________________________________
>>>>>>>>>> discuss mailing list
>>>>>>>>>> [email protected]
>>>>>>>>>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
