pci_call_probe() dispatches each probe with queue_work_on() to a housekeeping CPU of the device's node, except where it already probes inline. A VF added by one of the preceding node-placed async VF-add workers already executes on its device's node, so the dispatch buys it no locality: it funnels every VF probe through the single CPU cpumask_any_and() elects and re-serializes the parallel enable at the probe step.
Call local_pci_probe() directly when the caller is a workqueue worker (PF_WQ_WORKER) whose current CPU is on the device's node and in the same HK_TYPE_DOMAIN housekeeping mask the dispatch elects from. The gate is not VF-specific: any probe already running in a node-local housekeeping kworker takes it, while a probe from a task that is not a worker keeps the dispatch. Assisted-by: LLM Signed-off-by: Pavol Sakac <[email protected]> --- drivers/pci/pci-driver.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index e16aa59dd7ac..f5e12a5044cf 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -392,6 +392,22 @@ static bool pci_physfn_is_probed(struct pci_dev *dev) #endif } +static bool pci_probe_inline(int node) +{ + /* The CPU is a placement hint; an unstable raw read is fine. */ + int cpu = raw_smp_processor_id(); + bool ret; + + if (!(current->flags & PF_WQ_WORKER) || cpu_to_node(cpu) != node) + return false; + + rcu_read_lock(); + ret = housekeeping_cpu(cpu, HK_TYPE_DOMAIN); + rcu_read_unlock(); + + return ret; +} + static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev, const struct pci_device_id *id) { @@ -412,7 +428,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev, * device is probed from work_on_cpu() of the Physical device. */ if (node < 0 || node >= MAX_NUMNODES || !node_online(node) || - pci_physfn_is_probed(dev)) { + pci_physfn_is_probed(dev) || pci_probe_inline(node)) { error = local_pci_probe(&ddi); } else { struct pci_probe_arg arg = { .ddi = &ddi }; -- 2.47.3
