Re: [Qemu-devel] [BUG]Unassigned mem write during pci device hot-plug
> -Original Message-
> From: Michael S. Tsirkin [mailto:m...@redhat.com]
> Sent: Monday, January 07, 2019 11:06 PM
> To: xuyandong
> Cc: mar...@redhat.com; Paolo Bonzini ; qemu-
> de...@nongnu.org; Zhanghailiang ;
> wangxin (U) ; Huangweidong (C)
>
> Subject: Re: [BUG]Unassigned mem write during pci device hot-plug
>
> On Mon, Jan 07, 2019 at 02:37:17PM +, xuyandong wrote:
> > > > > > > > > > > > Hi all,
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > In our test, we configured VM with several
> > > > > > > > > > > > pci-bridges and a virtio-net nic been attached
> > > > > > > > > > > > with bus 4,
> > > > > > > > > > > >
> > > > > > > > > > > > After VM is startup, We ping this nic from host to
> > > > > > > > > > > > judge if it is working normally. Then, we hot add
> > > > > > > > > > > > pci devices to this VM with bus
> > > > > > > 0.
> > > > > > > > > > > >
> > > > > > > > > > > > We found the virtio-net NIC in bus 4 is not
> > > > > > > > > > > > working (can not
> > > > > > > > > > > > connect) occasionally, as it kick virtio backend
> > > > > > > > > > > > failure with error
> >
> > > > > > But I have another question, if we only fix this problem in
> > > > > > the kernel, the Linux version that has been released does not
> > > > > > work well on the
> > > > > virtualization platform.
> > > > > > Is there a way to fix this problem in the backend?
> >
> > Hi Michael,
> >
> > If we want to fix this problem on the backend, it is not enough to
> > consider only PCI device hot plugging, because I found that if we use
> > a command like "echo 1 > /sys/bus/pci/rescan" in guest, this problem is very
> easy to reproduce.
> >
> > From the perspective of device emulation, when guest writes 0x
> > to the BAR, guest just want to get the size of the region but not really
> updating the address space.
> > So I made the following patch to avoid update pci mapping.
> >
> > Do you think this make sense?
> >
> > [PATCH] pci: avoid update pci mapping when writing 0x to BAR
> >
> > When guest writes 0x to the BAR, guest just want to get the
> > size of the region but not really updating the address space.
> > So when guest writes 0x to BAR, we need avoid
> > pci_update_mappings or pci_bridge_update_mappings.
> >
> > Signed-off-by: xuyandong
>
> I see how that will address the common case however there are a bunch of
> issues here. First of all it's easy to trigger the update by some other
> action like
> VM migration. More importantly it's just possible that guest actually does
> want
> to set the low 32 bit of the address to all ones. For example, that is
> clearly
> listed as a way to disable all devices behind the bridge in the pci to pci
> bridge
> spec.
Ok, I see. If I only skip upate when guest writing 0x to Prefetcable
Base Upper 32 Bits
to meet the kernel double check problem.
Do you think there is still risk?
>
> Given upstream is dragging it's feet I'm open to adding a flag that will help
> keep guests going as a temporary measure.
> We will need to think about ways to restrict this as much as we can.
>
>
> > ---
> > hw/pci/pci.c| 6 --
> > hw/pci/pci_bridge.c | 8 +---
> > 2 files changed, 9 insertions(+), 5 deletions(-)
> >
> > diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 56b13b3..ef368e1 100644
> > --- a/hw/pci/pci.c
> > +++ b/hw/pci/pci.c
> > @@ -1361,6 +1361,7 @@ void pci_default_write_config(PCIDevice *d,
> > uint32_t addr, uint32_t val_in, int {
> > int i, was_irq_disabled = pci_irq_disabled(d);
> > uint32_t val = val_in;
> > +uint64_t barmask = (1 << l*8) - 1;
> >
> > for (i = 0; i < l; val >>= 8, ++i) {
> > uint8_t wmask = d->wmask[addr + i]; @@ -1369,9 +1370,10 @@
> > void pci_default_write_config(PCIDevice *d, uint32_t addr, uint32_t val_in,
> int
> > d->config[addr + i] = (d-&
Re: [Qemu-devel] [BUG]Unassigned mem write during pci device hot-plug
> -Original Message-
> From: xuyandong
> Sent: Monday, January 07, 2019 10:37 PM
> To: 'Michael S. Tsirkin'
> Cc: mar...@redhat.com; Paolo Bonzini ; qemu-
> de...@nongnu.org; Zhanghailiang ;
> wangxin (U) ; Huangweidong (C)
>
> Subject: RE: [BUG]Unassigned mem write during pci device hot-plug
>
> > > > > > > > > > > Hi all,
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > In our test, we configured VM with several
> > > > > > > > > > > pci-bridges and a virtio-net nic been attached with
> > > > > > > > > > > bus 4,
> > > > > > > > > > >
> > > > > > > > > > > After VM is startup, We ping this nic from host to
> > > > > > > > > > > judge if it is working normally. Then, we hot add
> > > > > > > > > > > pci devices to this VM with bus
> > > > > > 0.
> > > > > > > > > > >
> > > > > > > > > > > We found the virtio-net NIC in bus 4 is not working
> > > > > > > > > > > (can not
> > > > > > > > > > > connect) occasionally, as it kick virtio backend
> > > > > > > > > > > failure with error
>
> > > > > But I have another question, if we only fix this problem in the
> > > > > kernel, the Linux version that has been released does not work
> > > > > well on the
> > > > virtualization platform.
> > > > > Is there a way to fix this problem in the backend?
> > > >
> > > > There could we a way to work around this.
> > > > Does below help?
> > >
> > > I am sorry to tell you, I tested this patch and it doesn't work fine.
> > >
> > > >
> > > > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index
> > > > 236a20eaa8..7834cac4b0 100644
> > > > --- a/hw/i386/acpi-build.c
> > > > +++ b/hw/i386/acpi-build.c
> > > > @@ -551,7 +551,7 @@ static void build_append_pci_bus_devices(Aml
> > > > *parent_scope, PCIBus *bus,
> > > >
> > > > aml_append(method, aml_store(aml_int(bsel_val),
> > aml_name("BNUM")));
> > > > aml_append(method,
> > > > -aml_call2("DVNT", aml_name("PCIU"), aml_int(1) /* Device
> Check
> > */)
> > > > +aml_call2("DVNT", aml_name("PCIU"), aml_int(4) /*
> > > > + Device Check Light */)
> > > > );
> > > > aml_append(method,
> > > > aml_call2("DVNT", aml_name("PCID"), aml_int(3)/*
> > > > Eject Request */)
> >
> >
> > Oh I see, another bug:
> >
> > case ACPI_NOTIFY_DEVICE_CHECK_LIGHT:
> > acpi_handle_debug(handle,
> > "ACPI_NOTIFY_DEVICE_CHECK_LIGHT event\n");
> > /* TBD: Exactly what does 'light' mean? */
> > break;
> >
> > And then e.g. acpi_generic_hotplug_event(struct acpi_device *adev, u32
> > type) and friends all just ignore this event type.
> >
> >
> >
> > --
> > MST
>
> Hi Michael,
>
> If we want to fix this problem on the backend, it is not enough to consider
> only
> PCI device hot plugging, because I found that if we use a command like "echo
> 1 >
> /sys/bus/pci/rescan" in guest, this problem is very easy to reproduce.
>
> From the perspective of device emulation, when guest writes 0x to the
> BAR, guest just want to get the size of the region but not really updating the
> address space.
> So I made the following patch to avoid update pci mapping.
>
> Do you think this make sense?
>
> [PATCH] pci: avoid update pci mapping when writing 0x to BAR
>
> When guest writes 0x to the BAR, guest just want to get the size of
> the
> region but not really updating the address space.
> So when guest writes 0x to BAR, we need avoid pci_update_mappings or
> pci_bridge_update_mappings.
>
> Signed-off-by: xuyandong
> ---
> hw/pci/pci.c| 6 --
> hw/pci/pci_bridge.c | 8 +---
> 2 files changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 56b
Re: [Qemu-devel] [BUG]Unassigned mem write during pci device hot-plug
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > In our test, we configured VM with several pci-bridges
> > > > > > > > > > and a virtio-net nic been attached with bus 4,
> > > > > > > > > >
> > > > > > > > > > After VM is startup, We ping this nic from host to
> > > > > > > > > > judge if it is working normally. Then, we hot add pci
> > > > > > > > > > devices to this VM with bus
> > > > > 0.
> > > > > > > > > >
> > > > > > > > > > We found the virtio-net NIC in bus 4 is not working
> > > > > > > > > > (can not
> > > > > > > > > > connect) occasionally, as it kick virtio backend
> > > > > > > > > > failure with error
> > > > But I have another question, if we only fix this problem in the
> > > > kernel, the Linux version that has been released does not work
> > > > well on the
> > > virtualization platform.
> > > > Is there a way to fix this problem in the backend?
> > >
> > > There could we a way to work around this.
> > > Does below help?
> >
> > I am sorry to tell you, I tested this patch and it doesn't work fine.
> >
> > >
> > > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index
> > > 236a20eaa8..7834cac4b0 100644
> > > --- a/hw/i386/acpi-build.c
> > > +++ b/hw/i386/acpi-build.c
> > > @@ -551,7 +551,7 @@ static void build_append_pci_bus_devices(Aml
> > > *parent_scope, PCIBus *bus,
> > >
> > > aml_append(method, aml_store(aml_int(bsel_val),
> aml_name("BNUM")));
> > > aml_append(method,
> > > -aml_call2("DVNT", aml_name("PCIU"), aml_int(1) /* Device
> > > Check
> */)
> > > +aml_call2("DVNT", aml_name("PCIU"), aml_int(4) /*
> > > + Device Check Light */)
> > > );
> > > aml_append(method,
> > > aml_call2("DVNT", aml_name("PCID"), aml_int(3)/* Eject
> > > Request */)
>
>
> Oh I see, another bug:
>
> case ACPI_NOTIFY_DEVICE_CHECK_LIGHT:
> acpi_handle_debug(handle, "ACPI_NOTIFY_DEVICE_CHECK_LIGHT
> event\n");
> /* TBD: Exactly what does 'light' mean? */
> break;
>
> And then e.g. acpi_generic_hotplug_event(struct acpi_device *adev, u32 type)
> and friends all just ignore this event type.
>
>
>
> --
> MST
Hi Michael,
If we want to fix this problem on the backend, it is not enough to consider
only PCI
device hot plugging, because I found that if we use a command like
"echo 1 > /sys/bus/pci/rescan" in guest, this problem is very easy to reproduce.
From the perspective of device emulation, when guest writes 0x to the
BAR,
guest just want to get the size of the region but not really updating the
address space.
So I made the following patch to avoid update pci mapping.
Do you think this make sense?
[PATCH] pci: avoid update pci mapping when writing 0x to BAR
When guest writes 0x to the BAR, guest just want to get the size of the
region
but not really updating the address space.
So when guest writes 0x to BAR, we need avoid pci_update_mappings
or pci_bridge_update_mappings.
Signed-off-by: xuyandong
---
hw/pci/pci.c| 6 --
hw/pci/pci_bridge.c | 8 +---
2 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 56b13b3..ef368e1 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -1361,6 +1361,7 @@ void pci_default_write_config(PCIDevice *d, uint32_t
addr, uint32_t val_in, int
{
int i, was_irq_disabled = pci_irq_disabled(d);
uint32_t val = val_in;
+uint64_t barmask = (1 << l*8) - 1;
for (i = 0; i < l; val >>= 8, ++i) {
uint8_t wmask = d->wmask[addr + i];
@@ -1369,9 +1370,10 @@ void pci_default_write_config(PCIDevice *d, uint32_t
addr, uint32_t val_in, int
d->config[addr + i] = (d->config[addr + i] & ~wmask) | (val & wmask);
d->config[addr + i] &= ~(val & w1cmask); /* W1C: Write 1 to Clear */
}
-if (ranges_overlap(addr, l, PCI_BASE_ADDRESS_0, 24) ||
+if ((val_in != barmask &&
+ (ranges_overlap(addr, l, PCI_BASE_ADDRESS_0, 2
Re: [Qemu-devel] [BUG]Unassigned mem write during pci device hot-plug
> On Tue, Dec 11, 2018 at 02:55:43AM +0000, xuyandong wrote: > > On Tue, Dec 11, 2018 at 01:47:37AM +, xuyandong wrote: > > > > On Sat, Dec 08, 2018 at 11:58:59AM +, xuyandong wrote: > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In our test, we configured VM with several pci-bridges and > > > > > > > > a virtio-net nic been attached with bus 4, > > > > > > > > > > > > > > > > After VM is startup, We ping this nic from host to judge > > > > > > > > if it is working normally. Then, we hot add pci devices to > > > > > > > > this VM with bus > > > 0. > > > > > > > > > > > > > > > > We found the virtio-net NIC in bus 4 is not working (can > > > > > > > > not > > > > > > > > connect) occasionally, as it kick virtio backend failure with > > > > > > > > error > below: > > > > > > > > > > > > > > > > Unassigned mem write fc803004 = 0x1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > memory-region: pci_bridge_pci > > > > > > > > > > > > > > > > - (prio 0, RW): > > > > > > > > pci_bridge_pci > > > > > > > > > > > > > > > > fc80-fc803fff (prio 1, RW): > > > > > > > > virtio-pci > > > > > > > > > > > > > > > > fc80-fc800fff (prio 0, RW): > > > > > > > > virtio-pci-common > > > > > > > > > > > > > > > > fc801000-fc801fff (prio 0, RW): > > > > > > > > virtio-pci-isr > > > > > > > > > > > > > > > > fc802000-fc802fff (prio 0, RW): > > > > > > > > virtio-pci-device > > > > > > > > > > > > > > > > fc803000-fc803fff (prio 0, RW): > > > > > > > > virtio-pci-notify <- io mem unassigned > > > > > > > > > > > > > > > > … > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We caught an exceptional address changing while this > > > > > > > > problem happened, show as > > > > > > > > follow: > > > > > > > > > > > > > > > > Before pci_bridge_update_mappings： > > > > > > > > > > > > > > > > fc00-fc1f (prio 1, RW): > > > > > > > > alias pci_bridge_pref_mem @pci_bridge_pci > > > > > > > > fc00-fc1f > > > > > > > > > > > > > > > > fc20-fc3f (prio 1, RW): > > > > > > > > alias pci_bridge_pref_mem @pci_bridge_pci > > > > > > > > fc20-fc3f > > > > > > > > > > > > > > > > fc40-fc5f (prio 1, RW): > > > > > > > > alias pci_bridge_pref_mem @pci_bridge_pci > > > > > > > > fc40-fc5f > > > > > > > > > > > > > > > > fc60-fc7f (prio 1, RW): > > > > > > > > alias pci_bridge_pref_mem @pci_bridge_pci > > > > > > > > fc60-fc7f > > > > > > > > > > > > > > > > fc80-fc9f (prio 1, RW): > > > > > > > > alias pci_bridge_pref_mem @pci_bridge_pci > > > > > > > > fc80-fc9f > > > > > > > > <- correct Adress Spce > > > > > > > > > > > > > > > > fca0-fcbf (prio 1, RW): > > > > > > > > alias pci_bridge_pref_mem @pci_bridge_pci > > > &g
Re: [Qemu-devel] [BUG]Unassigned mem write during pci device hot-plug
On Tue, Dec 11, 2018 at 01:47:37AM +, xuyandong wrote: > > On Sat, Dec 08, 2018 at 11:58:59AM +, xuyandong wrote: > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > In our test, we configured VM with several pci-bridges and a > > > > > > virtio-net nic been attached with bus 4, > > > > > > > > > > > > After VM is startup, We ping this nic from host to judge if it > > > > > > is working normally. Then, we hot add pci devices to this VM with > > > > > > bus > 0. > > > > > > > > > > > > We found the virtio-net NIC in bus 4 is not working (can not > > > > > > connect) occasionally, as it kick virtio backend failure with error > > > > > > below: > > > > > > > > > > > > Unassigned mem write fc803004 = 0x1 > > > > > > > > > > > > > > > > > > > > > > > > memory-region: pci_bridge_pci > > > > > > > > > > > > - (prio 0, RW): > > > > > > pci_bridge_pci > > > > > > > > > > > > fc80-fc803fff (prio 1, RW): virtio-pci > > > > > > > > > > > > fc80-fc800fff (prio 0, RW): > > > > > > virtio-pci-common > > > > > > > > > > > > fc801000-fc801fff (prio 0, RW): > > > > > > virtio-pci-isr > > > > > > > > > > > > fc802000-fc802fff (prio 0, RW): > > > > > > virtio-pci-device > > > > > > > > > > > > fc803000-fc803fff (prio 0, RW): > > > > > > virtio-pci-notify <- io mem unassigned > > > > > > > > > > > > … > > > > > > > > > > > > > > > > > > > > > > > > We caught an exceptional address changing while this problem > > > > > > happened, show as > > > > > > follow: > > > > > > > > > > > > Before pci_bridge_update_mappings： > > > > > > > > > > > > fc00-fc1f (prio 1, RW): alias > > > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > > > fc00-fc1f > > > > > > > > > > > > fc20-fc3f (prio 1, RW): alias > > > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > > > fc20-fc3f > > > > > > > > > > > > fc40-fc5f (prio 1, RW): alias > > > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > > > fc40-fc5f > > > > > > > > > > > > fc60-fc7f (prio 1, RW): alias > > > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > > > fc60-fc7f > > > > > > > > > > > > fc80-fc9f (prio 1, RW): alias > > > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > > > fc80-fc9f > > > > > > <- correct Adress Spce > > > > > > > > > > > > fca0-fcbf (prio 1, RW): alias > > > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > > > fca0-fcbf > > > > > > > > > > > > fcc0-fcdf (prio 1, RW): alias > > > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > > > fcc0-fcdf > > > > > > > > > > > > fce0-fcff (prio 1, RW): alias > > > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > > > fce0-fcff > > > > > > > > > > > > > > > > > > > > > > > > After pci_bridge_update_mappings： > > > > > > > > > > > > fda0-fdbf (prio 1, RW): alias > > > > > > pci_bridge_mem @pci_bridge_pci > > > > > > 00
Re: [Qemu-devel] [BUG]Unassigned mem write during pci device hot-plug
On Sat, Dec 08, 2018 at 11:58:59AM +, xuyandong wrote: > > > > Hi all, > > > > > > > > > > > > > > > > In our test, we configured VM with several pci-bridges and a > > > > virtio-net nic been attached with bus 4, > > > > > > > > After VM is startup, We ping this nic from host to judge if it is > > > > working normally. Then, we hot add pci devices to this VM with bus 0. > > > > > > > > We found the virtio-net NIC in bus 4 is not working (can not > > > > connect) occasionally, as it kick virtio backend failure with error > > > > below: > > > > > > > > Unassigned mem write fc803004 = 0x1 > > > > > > > > > > > > > > > > memory-region: pci_bridge_pci > > > > > > > > - (prio 0, RW): pci_bridge_pci > > > > > > > > fc80-fc803fff (prio 1, RW): virtio-pci > > > > > > > > fc80-fc800fff (prio 0, RW): > > > > virtio-pci-common > > > > > > > > fc801000-fc801fff (prio 0, RW): > > > > virtio-pci-isr > > > > > > > > fc802000-fc802fff (prio 0, RW): > > > > virtio-pci-device > > > > > > > > fc803000-fc803fff (prio 0, RW): > > > > virtio-pci-notify <- io mem unassigned > > > > > > > > … > > > > > > > > > > > > > > > > We caught an exceptional address changing while this problem > > > > happened, show as > > > > follow: > > > > > > > > Before pci_bridge_update_mappings： > > > > > > > > fc00-fc1f (prio 1, RW): alias > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > fc00-fc1f > > > > > > > > fc20-fc3f (prio 1, RW): alias > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > fc20-fc3f > > > > > > > > fc40-fc5f (prio 1, RW): alias > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > fc40-fc5f > > > > > > > > fc60-fc7f (prio 1, RW): alias > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > fc60-fc7f > > > > > > > > fc80-fc9f (prio 1, RW): alias > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > fc80-fc9f > > > > <- correct Adress Spce > > > > > > > > fca0-fcbf (prio 1, RW): alias > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > fca0-fcbf > > > > > > > > fcc0-fcdf (prio 1, RW): alias > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > fcc0-fcdf > > > > > > > > fce0-fcff (prio 1, RW): alias > > > > pci_bridge_pref_mem @pci_bridge_pci > > > > fce0-fcff > > > > > > > > > > > > > > > > After pci_bridge_update_mappings： > > > > > > > > fda0-fdbf (prio 1, RW): alias > > > > pci_bridge_mem @pci_bridge_pci fda0-fdbf > > > > > > > > fdc0-fddf (prio 1, RW): alias > > > > pci_bridge_mem @pci_bridge_pci fdc0-fddf > > > > > > > > fde0-fdff (prio 1, RW): alias > > > > pci_bridge_mem @pci_bridge_pci fde0-fdff > > > > > > > > fe00-fe1f (prio 1, RW): alias > > > > pci_bridge_mem @pci_bridge_pci fe00-fe1f > > > > > > > > fe20-fe3f (prio 1, RW): alias > > > > pci_bridge_mem @pci_bridge_pci fe20-fe3f > > > > > > > > fe40-fe5f (prio 1, RW): alias > > > > pci_bridge_mem @pci_bridge_pci fe40-fe5f > > > > > > > > f
Re: [Qemu-devel] [BUG]Unassigned mem write during pci device hot-plug
On Sat, Dec 08, 2018 at 11:58:59AM +, xuyandong wrote: > > Hi all, > > > > > > > > In our test, we configured VM with several pci-bridges and a > > virtio-net nic been attached with bus 4, > > > > After VM is startup, We ping this nic from host to judge if it is > > working normally. Then, we hot add pci devices to this VM with bus 0. > > > > We found the virtio-net NIC in bus 4 is not working (can not connect) > > occasionally, as it kick virtio backend failure with error below: > > > > Unassigned mem write fc803004 = 0x1 > > > > > > > > memory-region: pci_bridge_pci > > > > - (prio 0, RW): pci_bridge_pci > > > > fc80-fc803fff (prio 1, RW): virtio-pci > > > > fc80-fc800fff (prio 0, RW): > > virtio-pci-common > > > > fc801000-fc801fff (prio 0, RW): virtio-pci-isr > > > > fc802000-fc802fff (prio 0, RW): > > virtio-pci-device > > > > fc803000-fc803fff (prio 0, RW): > > virtio-pci-notify <- io mem unassigned > > > > … > > > > > > > > We caught an exceptional address changing while this problem happened, > > show as > > follow: > > > > Before pci_bridge_update_mappings： > > > > fc00-fc1f (prio 1, RW): alias > > pci_bridge_pref_mem @pci_bridge_pci fc00-fc1f > > > > fc20-fc3f (prio 1, RW): alias > > pci_bridge_pref_mem @pci_bridge_pci fc20-fc3f > > > > fc40-fc5f (prio 1, RW): alias > > pci_bridge_pref_mem @pci_bridge_pci fc40-fc5f > > > > fc60-fc7f (prio 1, RW): alias > > pci_bridge_pref_mem @pci_bridge_pci fc60-fc7f > > > > fc80-fc9f (prio 1, RW): alias > > pci_bridge_pref_mem @pci_bridge_pci fc80-fc9f > > <- correct Adress Spce > > > > fca0-fcbf (prio 1, RW): alias > > pci_bridge_pref_mem @pci_bridge_pci fca0-fcbf > > > > fcc0-fcdf (prio 1, RW): alias > > pci_bridge_pref_mem @pci_bridge_pci fcc0-fcdf > > > > fce0-fcff (prio 1, RW): alias > > pci_bridge_pref_mem @pci_bridge_pci fce0-fcff > > > > > > > > After pci_bridge_update_mappings： > > > > fda0-fdbf (prio 1, RW): alias > > pci_bridge_mem @pci_bridge_pci fda0-fdbf > > > > fdc0-fddf (prio 1, RW): alias > > pci_bridge_mem @pci_bridge_pci fdc0-fddf > > > > fde0-fdff (prio 1, RW): alias > > pci_bridge_mem @pci_bridge_pci fde0-fdff > > > > fe00-fe1f (prio 1, RW): alias > > pci_bridge_mem @pci_bridge_pci fe00-fe1f > > > > fe20-fe3f (prio 1, RW): alias > > pci_bridge_mem @pci_bridge_pci fe20-fe3f > > > > fe40-fe5f (prio 1, RW): alias > > pci_bridge_mem @pci_bridge_pci fe40-fe5f > > > > fe60-fe7f (prio 1, RW): alias > > pci_bridge_mem @pci_bridge_pci fe60-fe7f > > > > fe80-fe9f (prio 1, RW): alias > > pci_bridge_mem @pci_bridge_pci fe80-fe9f > > > > fc80-fc80 (prio 1, RW): alias > > pci_bridge_pref_mem > > @pci_bridge_pci fc80-fc80 <- Exceptional Adress > Space > > This one is empty though right? > > > > > > > We have figured out why this address becomes this value, according to > > pci spec, pci driver can get BAR address size by writing 0x > > to > > > > the pci register firstly, and then read back the value from this register. > > > OK however as you show below the BAR being sized is the BAR if a bridge. Are > you then adding a bridge device by hotplug? No, I just simply hot plugged a VFIO device to Bus 0, another interesting phenomenon is If I hot plug the device t
Re: [Qemu-devel] [BUG]Unassigned mem write during pci device hot-plug
n Sat, Dec 08, 2018 at 11:58:59AM +, xuyandong wrote: > > Hi all, > > > > > > > > In our test, we configured VM with several pci-bridges and a > > virtio-net nic been attached with bus 4, > > > > After VM is startup, We ping this nic from host to judge if it is > > working normally. Then, we hot add pci devices to this VM with bus 0. > > > > We found the virtio-net NIC in bus 4 is not working (can not connect) > > occasionally, as it kick virtio backend failure with error below: > > > > Unassigned mem write fc803004 = 0x1 > > Thanks for the report. Which guest was used to produce this problem? > > -- > MST I was seeing this problem when I hotplug a VFIO device to guest CentOS 7.4, after that I compiled the latest Linux kernel and it also contains this problem. Thinks, Xu
[Qemu-devel] [BUG]Unassigned mem write during pci device hot-plug
Hi all, In our test, we configured VM with several pci-bridges and a virtio-net nic been attached with bus 4, After VM is startup, We ping this nic from host to judge if it is working normally. Then, we hot add pci devices to this VM with bus 0. We found the virtio-net NIC in bus 4 is not working (can not connect) occasionally, as it kick virtio backend failure with error below: Unassigned mem write fc803004 = 0x1 memory-region: pci_bridge_pci - (prio 0, RW): pci_bridge_pci fc80-fc803fff (prio 1, RW): virtio-pci fc80-fc800fff (prio 0, RW): virtio-pci-common fc801000-fc801fff (prio 0, RW): virtio-pci-isr fc802000-fc802fff (prio 0, RW): virtio-pci-device fc803000-fc803fff (prio 0, RW): virtio-pci-notify <- io mem unassigned … We caught an exceptional address changing while this problem happened, show as follow: Before pci_bridge_update_mappings： fc00-fc1f (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci fc00-fc1f fc20-fc3f (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci fc20-fc3f fc40-fc5f (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci fc40-fc5f fc60-fc7f (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci fc60-fc7f fc80-fc9f (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci fc80-fc9f <- correct Adress Spce fca0-fcbf (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci fca0-fcbf fcc0-fcdf (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci fcc0-fcdf fce0-fcff (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci fce0-fcff After pci_bridge_update_mappings： fda0-fdbf (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci fda0-fdbf fdc0-fddf (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci fdc0-fddf fde0-fdff (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci fde0-fdff fe00-fe1f (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci fe00-fe1f fe20-fe3f (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci fe20-fe3f fe40-fe5f (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci fe40-fe5f fe60-fe7f (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci fe60-fe7f fe80-fe9f (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci fe80-fe9f fc80-fc80 (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci fc80-fc80 <- Exceptional Adress Space We have figured out why this address becomes this value, according to pci spec, pci driver can get BAR address size by writing 0x to the pci register firstly, and then read back the value from this register. We didn't handle this value specially while process pci write in qemu, the function call stack is: Pci_bridge_dev_write_config -> pci_bridge_write_config -> pci_default_write_config (we update the config[address] value here to fc80, which should be 0xfc80 ) -> pci_bridge_update_mappings ->pci_bridge_region_del(br, br->windows); -> pci_bridge_region_init ->pci_bridge_init_alias (here pci_bridge_get_base, we use the wrong value fc80) -> memory_region_transaction_commit So, as we can see, we use the wrong base address in qemu to update the memory regions, though, we update the base address to The correct value after pci driver in VM write the original value back, the virtio NIC in bus 4 may still sends net packets concurrently with The wrong memory region address. We have tried to skip the memory region update action in qemu while detect pci write with 0x value, and it does work, but This seems to be not gently. diff --git a/hw/pci/pci_bridge.c b/hw/pci/pci_bridge.c index b2e50c3..84b405d 100644 --- a/hw/pci/pci_bridge.c +++ b/hw/pci/pci_bridge.c @@ -256,7 +256,8 @@ void pci_bridge_write_config(PCIDevice *d, pci_default_write_config(d, address, val, len); -if (ranges_overlap(address, len, PCI_COMMAND, 2) || +if ( (val != 0x) && +
Re: [Qemu-devel] An emulation failure occurs, if I hotplug vcpus immediately after the VM start
> -Original Message- > From: Paolo Bonzini [mailto:pbonz...@redhat.com] > Sent: Wednesday, June 06, 2018 9:58 PM > To: Gonglei (Arei) ; Igor Mammedov > ; xuyandong > Cc: Zhanghailiang ; wangxin (U) > ; lidonglin ; > k...@vger.kernel.org; qemu-devel@nongnu.org; Huangweidong (C) > > Subject: Re: An emulation failure occurs,if I hotplug vcpus immediately after > the VM start > > On 06/06/2018 15:28, Gonglei (Arei) wrote: > > gonglei: mem.slot: 3, mem.guest_phys_addr=0xc, > > mem.userspace_addr=0x7fc343ec, mem.flags=0, memory_size=0x0 > > gonglei: mem.slot: 3, mem.guest_phys_addr=0xc, > > mem.userspace_addr=0x7fc343ec, mem.flags=0, > memory_size=0x9000 > > > > When the memory region is cleared, the KVM will tell the slot to be > > invalid (which it is set to KVM_MEMSLOT_INVALID). > > > > If SeaBIOS accesses this memory and cause page fault, it will find an > > invalid value according to gfn (by __gfn_to_pfn_memslot), and finally > > it will return an invalid value, and finally it will return a failure. > > > > So, My questions are: > > > > 1) Why don't we hold kvm->slots_lock during page fault processing? > > Because it's protected by SRCU. We don't need kvm->slots_lock on the read > side. > > > 2) How do we assure that vcpus will not access the corresponding > > region when deleting an memory slot? > > We don't. It's generally a guest bug if they do, but the problem here is that > QEMU is splitting a memory region in two parts and that is not atomic. > > One fix could be to add a KVM_SET_USER_MEMORY_REGIONS ioctl that > replaces the entire memory map atomically. > > Paolo After we add a KVM_SET_USER_MEMORY_REGIONS ioctl that replaces the entire memory map atomically, how to use it in address_space_update_topology? Shall we checkout the spilt memory region before " address_space_update_topology_pass(as, old_view, new_view, false); address_space_update_topology_pass(as, old_view, new_view, true); ".
[Qemu-devel] An emulation failure occurs, if I hotplug vcpus immediately after the VM start
Hi there, I am doing some test on qemu vcpu hotplug and I run into some trouble. An emulation failure occurs and qemu prints the following msg: KVM internal error. Suberror: 1 emulation failure EAX= EBX= ECX= EDX=0600 ESI= EDI= EBP= ESP=fff8 EIP=ff53 EFL=00010082 [--S] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES = 9300 CS =f000 000f 9b00 SS = 9300 DS = 9300 FS = 9300 GS = 9300 LDT= 8200 TR = 8b00if GDT= IDT= CR0=6010 CR2= CR3= CR4= DR0= DR1= DR2= DR3= DR6=0ff0 DR7=0400 EFER= Code=31 d2 eb 04 66 83 ca ff 66 89 d0 66 5b 66 c3 66 89 d0 66 c3 66 68 21 8a 00 00 e9 08 d7 66 56 66 53 66 83 ec 0c 66 89 c3 66 e8 ce 7b ff ff 66 89 c6 I notice that guest is still running SeabBIOS in real mode when the vcpu has just been pluged. This emulation failure can be steadly reproduced if I am doing vcpu hotplug during VM launch process. After some digging, I find this KVM internal error shows up because KVM cannot emulate some MMIO (gpa 0xfff53 ). So I am confused, (1) does qemu support vcpu hotplug even if guest is running seabios ? (2) the gpa (0xfff53) is an address of BIOS ROM section, why does kvm confirm it as a mmio address incorrectly?
Re: [Qemu-devel] [PATCH] cpu: skip unpluged cpu when querying cpus
Yes, we should add it at qmp_query_cpus_fast() .
In this case, qmp_query_cpus() may kick a vcpu thread that has
stopped causing the qemu process to exit, qmp_query_cpus_fast() does not
interrupt vCPU threads, but may return invalid vCPU information.
> -Original Message-
> From: Igor Mammedov [mailto:imamm...@redhat.com]
> Sent: Wednesday, April 11, 2018 7:46 PM
> To: linzhecheng <linzhech...@huawei.com>
> Cc: qemu-devel@nongnu.org; xuyandong <xuyando...@huawei.com>;
> pbonz...@redhat.com; wangxin (U) <wangxinxin.w...@huawei.com>;
> r...@twiddle.net; crosthwaite.pe...@gmail.com
> Subject: Re: [Qemu-devel] [PATCH] cpu: skip unpluged cpu when querying
> cpus
>
> On Wed, 11 Apr 2018 19:16:02 +0800
> linzhecheng <linzhech...@huawei.com> wrote:
>
> > From: XuYandong <xuyando...@huawei.com>
> >
> > After vcpu1 thread exiting, vcpu0 thread (received notification) is
> > still waiting for holding qemu_global_mutex in cpu_remove_sync, at this
> moment, vcpu1 is still in global cpus list.
> > If main thread grab qemu_global_mutex in order to handle qmp command
> > "info cpus", qmp_query_cpus visit unpluged vcpu1 will lead qemu process
> to exit.
> Add here exact error or better stack trace in case it crashes.
>
>
> > Signed-off-by: XuYandong <xuyando...@huawei.com>
> > ---
> > cpus.c | 5 +
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/cpus.c b/cpus.c
> > index 2cb0af9..9b3a6c4 100644
> > --- a/cpus.c
> > +++ b/cpus.c
> > @@ -2018,6 +2018,11 @@ CpuInfoList *qmp_query_cpus(Error **errp)
> >
> > CPU_FOREACH(cpu) {
> > CpuInfoList *info;
> > +
> > +if (cpu->unplug) {
> > +continue;
> > +}
> Shouldn't be it done for qmp_query_cpus_fast() as well?
>
> > +
> > #if defined(TARGET_I386)
> > X86CPU *x86_cpu = X86_CPU(cpu);
> > CPUX86State *env = _cpu->env;
