Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 20:46 -0800, Andy Lutomirski wrote: > Me neither.  At least it wouldn't be a regression, but it's still > crappy. > > I think that arm is fine, at least.  I was unable to find an arm QEMU > config that has any problems with my patches. Ok, give me a few days for my headache

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Nov 10, 2015 4:44 PM, "Benjamin Herrenschmidt" wrote: > > On Tue, 2015-11-10 at 15:44 -0800, Andy Lutomirski wrote: > > > > > What about partition <-> partition virtio such as what we could do on > > > PAPR systems. That would have the weak barrier bit. > > > > > > > Is it partition <->

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 15:44 -0800, Andy Lutomirski wrote: > > > What about partition <-> partition virtio such as what we could do on > > PAPR systems. That would have the weak barrier bit. > > > > Is it partition <-> partition, bypassing IOMMU? No. > I think I'd settle for just something that

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, Nov 10, 2015 at 2:27 PM, Benjamin Herrenschmidt wrote: > On Tue, 2015-11-10 at 10:54 -0800, Andy Lutomirski wrote: >> >> Does that work on powerpc on existing kernels? >> >> Anyway, here's another crazy idea: make the quirk assume that the >> IOMMU is bypasses if and only if the weak

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 10:54 -0800, Andy Lutomirski wrote: >  > Does that work on powerpc on existing kernels? > > Anyway, here's another crazy idea: make the quirk assume that the > IOMMU is bypasses if and only if the weak barriers bit is set on > systems that are missing the new DT binding.

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 14:43 +0200, Michael S. Tsirkin wrote: > But not virtio-pci I think - that's broken for that usecase since we use > weaker barriers than required for real IO, as these have measureable > overhead.  We could have a feature "is a real PCI device", > that's completely

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 11:27 +0100, Joerg Roedel wrote: > > You have the same problem when real PCIe devices appear that speak > virtio. I think the only real (still not very nice) solution is to add a > quirk to powerpc platform code that sets noop dma-ops for the existing > virtio

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Nov 10, 2015 2:38 AM, "Benjamin Herrenschmidt" wrote: > > On Mon, 2015-11-09 at 21:35 -0800, Andy Lutomirski wrote: > > > > We could do it the other way around: on powerpc, if a PCI device is in > > that range and doesn't have the "bypass" property at all, then it's > > assumed to bypass the

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, Nov 10, 2015 at 09:37:54PM +1100, Benjamin Herrenschmidt wrote: > On Mon, 2015-11-09 at 21:35 -0800, Andy Lutomirski wrote: > > > > We could do it the other way around: on powerpc, if a PCI device is in > > that range and doesn't have the "bypass" property at all, then it's > > assumed to

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 10:45 +0100, Knut Omang wrote: > Can something be done by means of PCIe capabilities? > ATS (Address Translation Support) seems like a natural choice? Euh no... ATS is something else completely Cheers, Ben. -- To unsubscribe from this list: send the line "unsubscribe

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, 2015-11-09 at 21:35 -0800, Andy Lutomirski wrote: > > We could do it the other way around: on powerpc, if a PCI device is in > that range and doesn't have the "bypass" property at all, then it's > assumed to bypass the IOMMU.  This means that everything that > currently works continues

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, Nov 10, 2015 at 01:04:36PM +1100, Benjamin Herrenschmidt wrote: > The "in absence of the new DT binding" doesn't make that much sense. > > Those platforms use device-trees defined since the dawn of ages by > actual open firmware implementations, they either have no iommu > representation

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 13:04 +1100, Benjamin Herrenschmidt wrote: > On Mon, 2015-11-09 at 16:46 -0800, Andy Lutomirski wrote: > > The problem here is that in some of the problematic cases the > > virtio > > driver may not even be loaded. If someone runs an L1 guest with an > > IOMMU-bypassing

Re: [PATCH v4 0/6] virtio core DMA API conversion

On 2015-11-10 03:18, Andy Lutomirski wrote: > On Mon, Nov 9, 2015 at 6:04 PM, Benjamin Herrenschmidt >> I thus go back to my original statement, it's a LOT easier to handle if >> the device itself is self describing, indicating whether it is set to >> bypass a host iommu or not. For L1->L2, well,

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Nov 10, 2015 2:38 AM, "Benjamin Herrenschmidt" wrote: > > On Mon, 2015-11-09 at 21:35 -0800, Andy Lutomirski wrote: > > > > We could do it the other way around: on powerpc, if a PCI device is in > > that range and doesn't have the "bypass" property at all, then it's >

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 14:43 +0200, Michael S. Tsirkin wrote: > But not virtio-pci I think - that's broken for that usecase since we use > weaker barriers than required for real IO, as these have measureable > overhead.  We could have a feature "is a real PCI device", > that's completely

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 11:27 +0100, Joerg Roedel wrote: > > You have the same problem when real PCIe devices appear that speak > virtio. I think the only real (still not very nice) solution is to add a > quirk to powerpc platform code that sets noop dma-ops for the existing > virtio

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 10:54 -0800, Andy Lutomirski wrote: >  > Does that work on powerpc on existing kernels? > > Anyway, here's another crazy idea: make the quirk assume that the > IOMMU is bypasses if and only if the weak barriers bit is set on > systems that are missing the new DT binding.

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, Nov 10, 2015 at 2:27 PM, Benjamin Herrenschmidt wrote: > On Tue, 2015-11-10 at 10:54 -0800, Andy Lutomirski wrote: >> >> Does that work on powerpc on existing kernels? >> >> Anyway, here's another crazy idea: make the quirk assume that the >> IOMMU is bypasses if

Re: [PATCH v4 0/6] virtio core DMA API conversion

On 2015-11-10 03:18, Andy Lutomirski wrote: > On Mon, Nov 9, 2015 at 6:04 PM, Benjamin Herrenschmidt >> I thus go back to my original statement, it's a LOT easier to handle if >> the device itself is self describing, indicating whether it is set to >> bypass a host iommu or not. For L1->L2, well,

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, 2015-11-09 at 21:35 -0800, Andy Lutomirski wrote: > > We could do it the other way around: on powerpc, if a PCI device is in > that range and doesn't have the "bypass" property at all, then it's > assumed to bypass the IOMMU.  This means that everything that > currently works continues

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 13:04 +1100, Benjamin Herrenschmidt wrote: > On Mon, 2015-11-09 at 16:46 -0800, Andy Lutomirski wrote: > > The problem here is that in some of the problematic cases the > > virtio > > driver may not even be loaded. If someone runs an L1 guest with an > > IOMMU-bypassing

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, Nov 10, 2015 at 01:04:36PM +1100, Benjamin Herrenschmidt wrote: > The "in absence of the new DT binding" doesn't make that much sense. > > Those platforms use device-trees defined since the dawn of ages by > actual open firmware implementations, they either have no iommu > representation

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, Nov 10, 2015 at 09:37:54PM +1100, Benjamin Herrenschmidt wrote: > On Mon, 2015-11-09 at 21:35 -0800, Andy Lutomirski wrote: > > > > We could do it the other way around: on powerpc, if a PCI device is in > > that range and doesn't have the "bypass" property at all, then it's > > assumed to

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 15:44 -0800, Andy Lutomirski wrote: > > > What about partition <-> partition virtio such as what we could do on > > PAPR systems. That would have the weak barrier bit. > > > > Is it partition <-> partition, bypassing IOMMU? No. > I think I'd settle for just something that

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 10:45 +0100, Knut Omang wrote: > Can something be done by means of PCIe capabilities? > ATS (Address Translation Support) seems like a natural choice? Euh no... ATS is something else completely Cheers, Ben. -- To unsubscribe from this list: send the line "unsubscribe

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Nov 10, 2015 4:44 PM, "Benjamin Herrenschmidt" wrote: > > On Tue, 2015-11-10 at 15:44 -0800, Andy Lutomirski wrote: > > > > > What about partition <-> partition virtio such as what we could do on > > > PAPR systems. That would have the weak barrier bit. > > > > > > >

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Tue, 2015-11-10 at 20:46 -0800, Andy Lutomirski wrote: > Me neither.  At least it wouldn't be a regression, but it's still > crappy. > > I think that arm is fine, at least.  I was unable to find an arm QEMU > config that has any problems with my patches. Ok, give me a few days for my headache

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, 2015-11-09 at 18:18 -0800, Andy Lutomirski wrote: > > Which leaves the special case of Xen, where even preexisting devices > don't bypass the IOMMU.  Can we keep this specific to powerpc and > sparc?  On x86, this problem is basically nonexistent, since the IOMMU > is properly

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, Nov 9, 2015 at 9:28 PM, Benjamin Herrenschmidt wrote: > On Mon, 2015-11-09 at 18:18 -0800, Andy Lutomirski wrote: >> >> /* Qumranet donated their vendor ID for devices 0x1000 thru 0x10FF. >> */ >> static const struct pci_device_id virtio_pci_id_table[] = { >> { PCI_DEVICE(0x1af4,

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, Nov 9, 2015 at 9:26 PM, Benjamin Herrenschmidt wrote: > On Mon, 2015-11-09 at 18:18 -0800, Andy Lutomirski wrote: >> >> Which leaves the special case of Xen, where even preexisting devices >> don't bypass the IOMMU. Can we keep this specific to powerpc and >> sparc? On x86, this problem

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, 2015-11-09 at 18:18 -0800, Andy Lutomirski wrote: > > /* Qumranet donated their vendor ID for devices 0x1000 thru 0x10FF. > */ > static const struct pci_device_id virtio_pci_id_table[] = { >     { PCI_DEVICE(0x1af4, PCI_ANY_ID) }, >     { 0 } > }; > > Can we match on that range?

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, 2015-11-09 at 16:46 -0800, Andy Lutomirski wrote: > The problem here is that in some of the problematic cases the virtio > driver may not even be loaded.  If someone runs an L1 guest with an > IOMMU-bypassing virtio device and assigns it to L2 using vfio, then > *boom* L1 crashes.  (Same

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, Nov 9, 2015 at 6:04 PM, Benjamin Herrenschmidt wrote: > On Mon, 2015-11-09 at 16:46 -0800, Andy Lutomirski wrote: >> The problem here is that in some of the problematic cases the virtio >> driver may not even be loaded. If someone runs an L1 guest with an >> IOMMU-bypassing virtio device

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, Nov 9, 2015 at 2:58 PM, Benjamin Herrenschmidt wrote: > So ... > > I've finally tried to sort that out for powerpc and I can't find a way > to make that work that isn't a complete pile of stinking shit. > > I'm very tempted to go back to my original idea: virtio itself should > indicate

Re: [PATCH v4 0/6] virtio core DMA API conversion

So ... I've finally tried to sort that out for powerpc and I can't find a way to make that work that isn't a complete pile of stinking shit. I'm very tempted to go back to my original idea: virtio itself should indicate it's "bypassing ability" via the virtio config space or some other bit (like

Re: [PATCH v4 0/6] virtio core DMA API conversion

On 09/11/2015 13:15, Michael S. Tsirkin wrote: > Well that's not exactly true. I think we would like to make > it possible to put virtio devices behind an IOMMU on x86, > but if this means existing guests break, then many people won't be able > to use this option: having to find out which kernel

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Thu, Oct 29, 2015 at 06:09:45PM -0700, Andy Lutomirski wrote: > This switches virtio to use the DMA API unconditionally. I'm sure > it breaks things, but it seems to work on x86 using virtio-pci, with > and without Xen, and using both the modern 1.0 variant and the > legacy variant. > > This

Re: [PATCH v4 0/6] virtio core DMA API conversion

On 09/11/2015 13:15, Michael S. Tsirkin wrote: > Well that's not exactly true. I think we would like to make > it possible to put virtio devices behind an IOMMU on x86, > but if this means existing guests break, then many people won't be able > to use this option: having to find out which kernel

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Thu, Oct 29, 2015 at 06:09:45PM -0700, Andy Lutomirski wrote: > This switches virtio to use the DMA API unconditionally. I'm sure > it breaks things, but it seems to work on x86 using virtio-pci, with > and without Xen, and using both the modern 1.0 variant and the > legacy variant. > > This

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, 2015-11-09 at 18:18 -0800, Andy Lutomirski wrote: > > /* Qumranet donated their vendor ID for devices 0x1000 thru 0x10FF. > */ > static const struct pci_device_id virtio_pci_id_table[] = { >     { PCI_DEVICE(0x1af4, PCI_ANY_ID) }, >     { 0 } > }; > > Can we match on that range?

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, 2015-11-09 at 16:46 -0800, Andy Lutomirski wrote: > The problem here is that in some of the problematic cases the virtio > driver may not even be loaded.  If someone runs an L1 guest with an > IOMMU-bypassing virtio device and assigns it to L2 using vfio, then > *boom* L1 crashes.  (Same

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, Nov 9, 2015 at 9:28 PM, Benjamin Herrenschmidt wrote: > On Mon, 2015-11-09 at 18:18 -0800, Andy Lutomirski wrote: >> >> /* Qumranet donated their vendor ID for devices 0x1000 thru 0x10FF. >> */ >> static const struct pci_device_id virtio_pci_id_table[] = { >>

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, Nov 9, 2015 at 9:26 PM, Benjamin Herrenschmidt wrote: > On Mon, 2015-11-09 at 18:18 -0800, Andy Lutomirski wrote: >> >> Which leaves the special case of Xen, where even preexisting devices >> don't bypass the IOMMU. Can we keep this specific to powerpc and >>

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, Nov 9, 2015 at 6:04 PM, Benjamin Herrenschmidt wrote: > On Mon, 2015-11-09 at 16:46 -0800, Andy Lutomirski wrote: >> The problem here is that in some of the problematic cases the virtio >> driver may not even be loaded. If someone runs an L1 guest with an >>

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, 2015-11-09 at 18:18 -0800, Andy Lutomirski wrote: > > Which leaves the special case of Xen, where even preexisting devices > don't bypass the IOMMU.  Can we keep this specific to powerpc and > sparc?  On x86, this problem is basically nonexistent, since the IOMMU > is properly

Re: [PATCH v4 0/6] virtio core DMA API conversion

So ... I've finally tried to sort that out for powerpc and I can't find a way to make that work that isn't a complete pile of stinking shit. I'm very tempted to go back to my original idea: virtio itself should indicate it's "bypassing ability" via the virtio config space or some other bit (like

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Mon, Nov 9, 2015 at 2:58 PM, Benjamin Herrenschmidt wrote: > So ... > > I've finally tried to sort that out for powerpc and I can't find a way > to make that work that isn't a complete pile of stinking shit. > > I'm very tempted to go back to my original idea: virtio

Re: [PATCH v4 0/6] virtio core DMA API conversion

Am 30.10.2015 um 02:09 schrieb Andy Lutomirski: > This switches virtio to use the DMA API unconditionally. I'm sure > it breaks things, but it seems to work on x86 using virtio-pci, with > and without Xen, and using both the modern 1.0 variant and the > legacy variant. > > This appears to work

Re: [PATCH v4 0/6] virtio core DMA API conversion

Am 30.10.2015 um 02:09 schrieb Andy Lutomirski: > This switches virtio to use the DMA API unconditionally. I'm sure > it breaks things, but it seems to work on x86 using virtio-pci, with > and without Xen, and using both the modern 1.0 variant and the > legacy variant. > > This appears to work

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Thu, Oct 29, 2015 at 6:09 PM, Andy Lutomirski wrote: > This switches virtio to use the DMA API unconditionally. I'm sure > it breaks things, but it seems to work on x86 using virtio-pci, with > and without Xen, and using both the modern 1.0 variant and the > legacy variant. ... > Andy

Re: [PATCH v4 0/6] virtio core DMA API conversion

On Thu, Oct 29, 2015 at 6:09 PM, Andy Lutomirski wrote: > This switches virtio to use the DMA API unconditionally. I'm sure > it breaks things, but it seems to work on x86 using virtio-pci, with > and without Xen, and using both the modern 1.0 variant and the > legacy variant.