Fabiano, could you sanity check this patch? Thanks!
Peter, more below:
On 7/21/2025 6:24 AM, Peter Maydell wrote:
On Wed, 16 Jul 2025 at 19:07, Steve Sistare <steven.sist...@oracle.com> wrote:
Close a race condition that causes cpr-transfer to lose VFIO
interrupts on ARM.
CPR stops VCPUs but does not disable VFIO interrupts, which may continue
to arrive throughout the transition to new QEMU.
CPR calls kvm_irqchip_remove_irqfd_notifier_gsi in old QEMU to force
future interrupts to the producer eventfd, where they are preserved.
Old QEMU then destroys the old KVM instance. However, interrupts may
already be pending in KVM state. To preserve them, call ioctl
KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES to flush them to guest RAM, where
they will be picked up when the new KVM+VCPU instance is created.
Signed-off-by: Steve Sistare <steven.sist...@oracle.com>
+static int kvm_arm_gicv3_notifier(NotifierWithReturn *notifier,
+ MigrationEvent *e, Error **errp)
+{
+ if (e->type == MIG_EVENT_PRECOPY_DONE) {
+ GICv3State *s = container_of(notifier, GICv3State, cpr_notifier);
+ return kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES,
+ NULL, true, errp);
+ }
+ return 0;
+}
static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
{
@@ -883,13 +895,17 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error
**errp)
GICD_CTLR)) {
error_setg(&s->migration_blocker, "This operating system kernel does "
"not support vGICv3 migration");
- if (migrate_add_blocker(&s->migration_blocker, errp) < 0) {
+ if (migrate_add_blocker_modes(&s->migration_blocker, MIG_MODE_NORMAL,
+ MIG_MODE_CPR_TRANSFER, errp) < 0) {
Why did you change this? It's the general "if no support, can't
migrate at all" check, which seems unrelated to cpr-transfer.
"If no support", then cpr-transfer should also be blocked.
It should have been added earlier.
- Steve
return;
}
}
if (kvm_device_check_attr(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES)) {
qemu_add_vm_change_state_handler(vm_change_state_handler, s);
+ migration_add_notifier_mode(&s->cpr_notifier,
+ kvm_arm_gicv3_notifier,
+ MIG_MODE_CPR_TRANSFER);
}
}
Otherwise the patch looks OK in general shape, but I know
nothing about cpr-transfer so a review from somebody on the
migration side would be helpful.
thanks
-- PMM