On 24.09.2019 16:15, Eelco Chaudron wrote:
On 12 Sep 2019, at 12:24, Ilya Maximets wrote:
On 12.09.2019 13:19, Ilya Maximets wrote:
On 12.09.2019 13:07, Eelco Chaudron wrote:
On 12 Sep 2019, at 10:39, Ilya Maximets wrote:
On 11.09.2019 16:20, Eelco Chaudron wrote:
Currently, OVS does not register and therefore not handle the
interface reset event from the DPDK framework. This would cause a
problem in cases where a VF is used as an interface, and its
configuration changes.
As an example in the following scenario the MAC change is not
detected/acted upon until OVS is restarted without the patch applied:
$ echo 1 > /sys/bus/pci/devices/0000:05:00.1/sriov_numvfs
$ ovs-vsctl add-port ovs_pvp_br0 dpdk0 -- \
set Interface dpdk0 type=dpdk -- \
set Interface dpdk0 options:dpdk-devargs=0000:05:0a.0
$ ip link set p5p2 vf 0 mac 52:54:00:92:d3:33
Signed-off-by: Eelco Chaudron <echau...@redhat.com>
---
<snip>
@@ -4180,13 +4223,19 @@ netdev_dpdk_reconfigure(struct netdev *netdev)
&& dev->rxq_size == dev->requested_rxq_size
&& dev->txq_size == dev->requested_txq_size
&& dev->socket_id == dev->requested_socket_id
- && dev->started) {
+ && dev->started && !dev->reset_needed) {
/* Reconfiguration is unnecessary */
goto out;
}
- rte_eth_dev_stop(dev->port_id);
+ if (dev->reset_needed) {
+ rte_eth_dev_reset(dev->port_id);
Thinking more about the change and looking at flow control configuration,
it seems that on reset we'll lost configurations done in set_config().
Device reset should return it to initial state, i.e. all the default settings,
but set_config() will not be called after that.
I know, that VFs commonly doesn't support flow control, but if we'll add like
real configuration of MAC address, it will be lost too.
What do you think?
Doesn’t the full bridge run sequence take care of this?
So in callback we do netdev_request_reconfigure() which triggers the following
sequence…
bridge_run__()
...
dpif_netdev_run()
...
reconfigure_datapath()
...
netdev_dpdk_reconfigure()
But than also it will call in the next run
bridge_run()
...
bridge_delete_or_reconfigure_ports()
...
netdev_set_config()
netdev_dpdk_set_config()
Or do I miss something?
dpif_netdev_run() is called from ofproto_type_run() which is called
unconditionally from the bridge_run().
But bridge_delete_or_reconfigure_ports() only called from bridge_reconfigure(),
which is protected by the following condition:
if (ovsdb_idl_get_seqno(idl) != idl_seqno ||
if_notifier_changed(ifnotifier)) {
i.e. if there will be no DB updates or there will be no netlink notifications,
set_config() will not be called. I understand that in your scenario there
might be netlink notification for interface changes since PF is controlled
by the kernel, but it might be not the case if PF attached as a DPDK port
to OVS or some other application.
Hmm. Basically, dpdk_eth_event_callback() is an analogue of the
if_notifier, but for DPDK application.
Maybe we can add it as another trigger for above 'if' condition?
It sounds like a good idea, however, I see no clean way of doing this as the>
if_notifier_changed()/if_notifier_create() is not data path abstracted.
Any ideas for a nice clean solution here?
Guess we could add a dpdk_notifier_create() in if-notifier.c
Thoughts?
I'm not sure how the clean solution could look like, but I'd like to
rise another issue closely related to this change.
The issue is an infinite re-addition of the failed ports.
This happens if the device in userspace datapath has a linux network
interface and it's not able to be configured. For example, if the first
reconfiguration fails because of misconfiguration.
In current code victims are afxdp ports and the mellanox NIC ports
opened by the DPDK due to their bifurcated drivers (It's unlikely for
usual netdev-linux ports to fail).
The root cause: Every change in the state of the network interface
of a linux kernel device generates if-notifier event and if-notifier
event triggers the OVS code to re-apply the configuration of ports,
i.e. add broken ports back. The most obvious part is that dpif-netdev
changes the device flags before trying to configure it:
1. add_port()
2. set_flags() --> if-notifier event
3. reconfigure() --> port removal from the datapath due to
misconfiguration or any other issue in
the underlying device.
4. setting flags back --> another if-notifier event.
5. There was new if-notifier event?
yes --> re-apply all settings. --> goto step 1.
We could move applying the netdev flags after the first successful
reconfiguration and this might fix part of the problem, but many
changes that we're applying on the reconfiguration phase could
generate if-notifier events too and we can't easily avoid that.
I think, if we'll add a dpdk_notifier, we'll have same issue for
the rest of DPDK port types.
Any thoughts?
BTW, I'm not against the dpdk_notifier and I think that it's a
right direction to move, but it's the right place and time to
highlight the issue we have.
Best regards, Ilya Maximets.
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
