bhouse-nexthop opened a new pull request, #14136:
URL: https://github.com/apache/cloudstack/pull/14136
### Description
A system VM gets one NIC queue no matter how many CPUs it has, so every
packet interrupt lands on
CPU0. Adding CPUs to a router therefore does not move more packets, which is
the only reason to add
them.
The queue count is already a VM detail, `nic.multiqueue.number`, and the
agent already turns it into
`<driver queues='N'/>` on the interface. A user VM can set it three ways:
| how | since |
|---|---|
| `deployVirtualMachine nicmultiqueuenumber=N` | 4.18 |
| `updateVirtualMachine details[0].nic.multiqueue.number=N` | |
| the Settings tab - `listDetailOptions` offers the key for KVM | |
A system VM goes through none of them. It is not created by
`deployVirtualMachine`, it is not a
`UserVm` so `updateVirtualMachine` does not apply, and it has no details
editor. **There is no way
to set it at all**, which is why this is a fix rather than a new setting.
So a system VM now gets one queue per CPU when nothing is set:
| CPUs | emitted |
|---|---|
| 1 | nothing, identical to today |
| 4 | `queues='4'` |
| 512 | `queues='256'` |
The 256 is the tap device ceiling in the host kernel, which refuses the
interface rather than
trimming to fit. Above the CPU count it would make no difference anyway -
the guest driver uses
`min(CPUs, queues)` and pushes that at probe, so no guest side change is
needed.
A queue number already set on the VM still wins.
**The default offering is a single CPU**, so a default install emits the
same domain XML as before.
Only a system VM someone has already resized differs, and it differs in the
direction the resize
asked for.
### Worth calling out
- **There is no way to turn this off.** There is no way to turn it on today
either, so it is
symmetric, but it is worth being explicit about on a release branch.
- **The cost scales with NIC count, not just queue count.** A VPC router has
one NIC per tier, so a
4 CPU router fronting 8 tiers goes from 8 vhost threads to 32. Idle queues
are cheap, but the
thread count is not nothing on a dense host.
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] Build/CI
- [ ] Test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [x] Minor
- [ ] Trivial
### How Has This Been Tested?
`KVMGuruTest`, `Tests run: 35, Failures: 0`. Six new cases:
- a 4 CPU system VM gets `queues='4'`
- a 512 CPU one stops at the tap ceiling of 256
- a single CPU one is left alone, so no attribute is emitted
- other details on the VM are preserved
- a queue number already on the VM wins
- a user VM is never touched
The guest side was checked against the driver source the system VM runs,
Linux 6.1
`drivers/net/virtio_net.c`:
```c
/* Enable multiqueue by default */
if (num_online_cpus() >= max_queue_pairs)
vi->curr_queue_pairs = max_queue_pairs;
else
vi->curr_queue_pairs = num_online_cpus();
```
followed by `virtnet_set_queues(vi, vi->curr_queue_pairs)` at probe, so the
queues come up without
`ethtool -L`. The 256 comes from `MAX_TAP_QUEUES` in `drivers/net/tun.c`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
