On 04/11/2026 11:02, David Marchand wrote:
External email: Use caution opening links or attachments
On Tue, 29 Sept 2026 at 16:31, David Marchand <[email protected]> wrote:
To avoid back and forth on this work, I went and prepared a RFC for a
preparation netdev_dpdk refactoring (based on main branch, nothing is
doca specific, so no need to go to dpdk-latest).
I think it will simplify netdev-doca.c integration.
And after the v2, more feedback on the current netdev-doca.c implementation.
Thanks David for this.
My notes:
- the main point that needs refactoring is how netdev-doca.c
reimplements DPDK device setup.
The ESW manager port is a simple netdev-dpdk Ethernet netdevice.
And the representors are entities wrapped on top of it.
I don't understand a requirement around calling direct calls to DPDK
dev_start/dev_configure/dev_stop for the representors.
Initialisation of the doca components seems easy to wrap around once
a DPDK ESW manager port is in place.
Relying on a netdev-dpdk object removes the need for knowing about
DPDK parameters, and configuration aspects.
- I also have concerns about mixing the locks between
netdev-dpdk/dpdk-common/doca.c.
Putting netdev-doca on top of netdev-dpdk makes it easier to
understand (and express with lock annotations we have).
- any physical level informations (status, speed, duplex etc...) for
the representors should be the same as the ESW manager port.
This removes the need for a specialised watchdog thread (only
monitoring the ESW manager port in netdev-dpdk.c should be enough).
Representors must also start as they can have their own link status for
example.
Though not trivial in a standard configuration, there is another
configuration in which it's more natural to see it:
This configuration is MPESW (Multi-Port ESW). In such configuration, the
uplinks of the card (e.g. p0,p1) become part of the same ESW. In such
case, p0 is still the ESW port, while p1 is a representor of it.
Then, as it is a physical device, it's easy to see it can have its own
link status.
It can also happen for "regular" representors.
I enhanced your approach with the shadow dpdk port for representors as
well. For that, I needed to allow zero-queues dpdk ports.
The doca code no longer handles the probe/close of the dpdk port. It
generates the devargs and handed over to the shadow dpdk port.
Regarding the closure of it, it cannot work as you wrote it. The issue
is that calling "netdev_close(shadow)" is not allowed inside
netdev_doca_destruct. The reason is that we are under netdev_mutex,
which is not recursive. If calling it there, we assert.
To handle this, I defer that closing to be done in the "run" call.
To report status, as we have an underlying dpdk port, I report the
status of that, taking advantage of the already running watchdog thread.
- Rx and Tx is just a matter of calling netdev-dpdk code with the ESW
manager DPDK netdev.
I tried the approach of exporting just those helpers in a new
netdev-dpdk-eth.h header, which seems enough.
- there is way too many back and forth on netdev -> netdev_doca ->
netdev conversions. This is confusing.
Only code facing the netdev API needs converting from standard to
specific type. Then all internal code work only on netdev-doca
objects.
Ack for this.
- (based on some AI investigation on this branch code) statistics are
suspicious.
I spotted a few oddities around Rx/Tx packets/bytes at the ESW
manager / representor ring level, drop counters, and most of them are
not reported through netdev API (?).
Some stats look simply copied from netdev-dpdk without any code touching
them.
The (ESW manager port) HW xstats are not reported.
Ack. I reworked the stats area. All the NIC stats (also xstats) are
relevant only for the ESW. I removed them from representors and added
"hw_" prefix for them, shown only for the ESW port.
- usage of coverage counters in the Rx/Tx code will make this hard to
debug, I would turn them into interface OVS stats.
I assume you mean netdev_doca_rx_drop_xxx ones. Those coverage counters
should never increase. It is just a way to see if that "couldn't happen"
situation indeed doesn't happen. I prefer not to report them as regular
netdev statistics and leave them as coverage ones.
- there is some leak around devargs and peer_name in the destruct code.
Nice catch for the devargs. peer_name was never allocated (or actually
used). I removed it as it belongs to a feature we have in our downstream
branch that we haven't introduced yet.
- there is still one open question for me, should we make sure
netdev-doca.c and netdev-dpdk.c don't collide in mid air?
Like a developer switching from one configuration type to the other
(especially on the ESW manager topic).
I don't understand your meaning for this. How can they "collide"?
Fwiw, I pushed a branch on top of the v2 netdev-dpdk refactor I sent yesterday.
Most (the stats need more work) of those comments are addressed.
This is only compile tested, with stubs as I don't have a DOCA setup.
https://github.com/david-marchand/ovs/commits/refactor_dpdk_doca/
That branch was very helpful.
BTW, any mlx card (>=cx6) can run doca.
To keep the same line, I pushed some patches on top of yours.
They are there to show the approach. They may not be final yet.
Suppose we can agree on that, I will squash the code into a standalone
series, rebased on your series.
It still doesn't include addressing of Eelco's comments on v6.
https://github.com/elibritstein/OVS/tree/refactor_dpdk_doca_shadow
--
David Marchand
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev