On Mon, Jun 29, 2026 at 6:19 PM Jakub Kicinski <[email protected]> wrote:
>
> On Mon, 29 Jun 2026 17:47:03 -0700 Shuhao Tan wrote:
> > > Send a netdev Netlink notification when NAPI is re-created and
> > > let the userspace re-apply the settings?
> >
> > It feels surprising that the userspace needs to reconfigure thread
> > properties when changing NIC configurations unrelated to threading.
> > Another downside is that when userspace configures NIC configurations
> > in quick succession, re-application becomes messy because a previous
> > re-application might still be in progress when the thread is gone.
>
> Can you explain more about your deployment and system configuration
> flow? We may be adding micro optimizations when the problem is that
> we recreate the NAPIs in the first place.
We have an AF_XDP application with extremely low latency and jitter
requirements running on our servers. Sami developed busypolling
threaded napi for them. Since it's an AF_XDP application, they attach
their umem to specific RX queues, and then configure threaded NAPI
busypolling to achieve low latency. That involves using the Netlink
API to set the threaded/busypolling property, grabbing the kthread
PID, and setting some properties on the kthread. Concretely, something
like:
```
local napi_id
napi_id=$(call_ynl --output-json --do queue-get \
--json "{\"ifindex\": ${ifindex}, \"id\": ${q_id}, \"type\": \"rx\"}" | \
jq -r '."napi-id"')
echo "Enabling busypolling for queue ${q_id} (NAPI ${napi_id}) on CPU ${cpu}"
call_ynl --do napi-set --json "{\"id\": \"${napi_id}\",
\"threaded\": \"busy-poll\"}" >/dev/null
local napi_kthread_pid
napi_kthread_pid=$(call_ynl --do napi-get --output-json \
--json "{\"id\": \"${napi_id}\"}" | jq -r '."pid" // empty')
taskset -pc "${cpu}" "${napi_kthread_pid}" >/dev/null
```
The bug is that the taskset configurations disappear when the user
runs an unrelated ethtool command. Yes, the root cause is that an
unrelated ethtool driver config on GVE, gve_adjust_config() will
recreate the NAPIs. My understanding is that NAPIs are recreated on
ethtool commands as WAI and standard upstream driver behavior; is that
not correct? If we held onto the same NAPIs during driver reconfigs,
this issue would indeed be fixed. Is holding onto the same NAPI during
driver reconfigurations an appropriate/feasible fix here?
Other ideas, FWIW:
1. We could add an optional netlink argument, "remember_napi_kthread,"
that customizes this behavior for users who want to remember thread
config.
2. We add an optional netlink argument: "bind_napi_kthread_to_cpu"
that not only sets threaded but also binds the created thread to a
specific cpu set, and then include that cpu set as part of the
napi_config so that's it's remembered on reconfigs.
But I think the best options are (a) not recreating napis during
driver configs or (b) this patch to park/unpark the thread, TBH.
--
Thanks,
Mina
