Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-09-10 Thread Gerd Hoffmann
> > this to set the VIRTIO_F_IOMMU_PLATFORM flag. But for example > > QEMU has the use of iommu_platform attribute disabled for virtio-gpu > > device. So would also like to move towards not having to specify > > the VIRTIO_F_IOMMU_PLATFORM flag. > > Specifying VIRTIO_F_IOMMU_PLATFORM is the

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-09-10 Thread Christoph Hellwig
On Thu, Sep 06, 2018 at 07:09:09PM -0500, Jiandi An wrote: > For virtio device we have to pass in iommu_platform=true flag for > this to set the VIRTIO_F_IOMMU_PLATFORM flag. But for example > QEMU has the use of iommu_platform attribute disabled for virtio-gpu > device. So would also like to

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-08 Thread Christoph Hellwig
On Thu, Aug 09, 2018 at 08:13:32AM +1000, Benjamin Herrenschmidt wrote: > > > - if (xen_domain()) > > > + if (xen_domain() || pseries_secure_vm()) > > > return true; > > > > I don't think it's pseries specific actually. E.g. I suspect AMD SEV > > might benefit from the same kind of

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-08 Thread Benjamin Herrenschmidt
On Thu, 2018-08-09 at 08:13 +1000, Benjamin Herrenschmidt wrote: > > For completeness, virtio could also have its own bounce buffer > > outside of DMA API one. I don't see lots of benefits to this > > though. > > Not fan of that either... To elaborate a bit ... For our secure VMs, we will need

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-08 Thread Benjamin Herrenschmidt
On Wed, 2018-08-08 at 23:31 +0300, Michael S. Tsirkin wrote: > On Wed, Aug 08, 2018 at 11:18:13PM +1000, Benjamin Herrenschmidt wrote: > > Sure, but all of this is just the configuration of the iommu. But I > > think we agree here, and your point remains valid, indeed my proposed > > hack: > > >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-08 Thread Michael S. Tsirkin
On Wed, Aug 08, 2018 at 11:18:13PM +1000, Benjamin Herrenschmidt wrote: > Sure, but all of this is just the configuration of the iommu. But I > think we agree here, and your point remains valid, indeed my proposed > hack: > > > if ((flags & VIRTIO_F_IOMMU_PLATFORM) ||

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-08 Thread Benjamin Herrenschmidt
On Wed, 2018-08-08 at 05:30 -0700, Christoph Hellwig wrote: > On Wed, Aug 08, 2018 at 08:07:49PM +1000, Benjamin Herrenschmidt wrote: > > Qemu virtio bypasses that iommu when the VIRTIO_F_IOMMU_PLATFORM flag > > is not set (default) but there's nothing in the device-tree to tell the > > guest

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-08 Thread Christoph Hellwig
On Wed, Aug 08, 2018 at 08:07:49PM +1000, Benjamin Herrenschmidt wrote: > Qemu virtio bypasses that iommu when the VIRTIO_F_IOMMU_PLATFORM flag > is not set (default) but there's nothing in the device-tree to tell the > guest about this since it's a violation of our pseries architecture, so > we

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-08 Thread Benjamin Herrenschmidt
On Tue, 2018-08-07 at 23:31 -0700, Christoph Hellwig wrote: > > You don't need to set them the time you go secure. You just need to > set the flag from the beginning on any VM you might want to go secure. > Or for simplicity just any VM - if the DT/ACPI tables exposed by > qemu are good enough

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-08 Thread Christoph Hellwig
On Wed, Aug 08, 2018 at 06:32:45AM +1000, Benjamin Herrenschmidt wrote: > As for the flag itself, while we could set it from qemu when we get > notified that the guest is going secure, both Michael and I think it's > rather gross, it requires qemu to go iterate all virtio devices and > "poke"

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-07 Thread Benjamin Herrenschmidt
On Tue, 2018-08-07 at 06:55 -0700, Christoph Hellwig wrote: > On Tue, Aug 07, 2018 at 04:42:44PM +1000, Benjamin Herrenschmidt wrote: > > Note that I can make it so that the same DMA ops (basically standard > > swiotlb ops without arch hacks) work for both "direct virtio" and > > "normal PCI"

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-07 Thread Christoph Hellwig
On Tue, Aug 07, 2018 at 04:42:44PM +1000, Benjamin Herrenschmidt wrote: > Note that I can make it so that the same DMA ops (basically standard > swiotlb ops without arch hacks) work for both "direct virtio" and > "normal PCI" devices. > > The trick is simply in the arch to setup the iommu to map

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-07 Thread Benjamin Herrenschmidt
On Mon, 2018-08-06 at 23:27 -0700, Christoph Hellwig wrote: > On Tue, Aug 07, 2018 at 08:13:56AM +1000, Benjamin Herrenschmidt wrote: > > It would be indeed ideal if all we had to do was setup some kind of > > bus_dma_mask on all PCI devices and have virtio automagically insert > > swiotlb when

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-07 Thread Benjamin Herrenschmidt
On Mon, 2018-08-06 at 23:21 -0700, Christoph Hellwig wrote: > On Tue, Aug 07, 2018 at 05:52:12AM +1000, Benjamin Herrenschmidt wrote: > > > It is your job to write a coherent interface specification that does > > > not depend on the used components. The hypervisor might be PAPR, > > > Linux +

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-07 Thread Christoph Hellwig
On Tue, Aug 07, 2018 at 02:45:25AM +0300, Michael S. Tsirkin wrote: > > > I think that's where Christoph might have specific ideas about it. > > > > OK well, assuming Christoph can solve the direct case in a way that > > also work for the virtio !iommu case, we still want some bit of logic > >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-07 Thread Christoph Hellwig
On Tue, Aug 07, 2018 at 08:13:56AM +1000, Benjamin Herrenschmidt wrote: > It would be indeed ideal if all we had to do was setup some kind of > bus_dma_mask on all PCI devices and have virtio automagically insert > swiotlb when necessary. For 4.20 I plan to remove the swiotlb ops and instead do

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-07 Thread Christoph Hellwig
On Tue, Aug 07, 2018 at 05:52:12AM +1000, Benjamin Herrenschmidt wrote: > > It is your job to write a coherent interface specification that does > > not depend on the used components. The hypervisor might be PAPR, > > Linux + qemu, VMware, Hyperv or something so secret that you'd have > > to

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-07 Thread Christoph Hellwig
On Tue, Aug 07, 2018 at 12:46:34AM +0300, Michael S. Tsirkin wrote: > Well we have the RFC for that - the switch to using DMA ops unconditionally > isn't > problematic itself IMHO, for now that RFC is blocked > by its perfromance overhead for now but Christoph says > he's trying to remove that

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-07 Thread Christoph Hellwig
On Tue, Aug 07, 2018 at 07:26:35AM +1000, Benjamin Herrenschmidt wrote: > > I think Christoph merely objects to the specific implementation. If > > instead you do something like tweak dev->bus_dma_mask for the virtio > > device I think he won't object. > > Well, we don't have "bus_dma_mask" yet

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-07 Thread Christoph Hellwig
On Mon, Aug 06, 2018 at 11:35:39PM +0300, Michael S. Tsirkin wrote: > > As I said replying to Christoph, we are "leaking" into the interface > > something here that is really what's the VM is doing to itself, which > > is to stash its memory away in an inaccessible place. > > > > Cheers, > > Ben.

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Benjamin Herrenschmidt
On Tue, 2018-08-07 at 02:45 +0300, Michael S. Tsirkin wrote: > > OK well, assuming Christoph can solve the direct case in a way that > > also work for the virtio !iommu case, we still want some bit of logic > > somewhere that will "switch" to swiotlb based ops if the DMA mask is > > limited. > >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Michael S. Tsirkin
On Tue, Aug 07, 2018 at 08:13:56AM +1000, Benjamin Herrenschmidt wrote: > On Tue, 2018-08-07 at 00:46 +0300, Michael S. Tsirkin wrote: > > On Tue, Aug 07, 2018 at 07:26:35AM +1000, Benjamin Herrenschmidt wrote: > > > On Mon, 2018-08-06 at 23:35 +0300, Michael S. Tsirkin wrote: > > > > > As I said

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Benjamin Herrenschmidt
On Mon, 2018-08-06 at 23:35 +0300, Michael S. Tsirkin wrote: > On Tue, Aug 07, 2018 at 05:56:59AM +1000, Benjamin Herrenschmidt wrote: > > On Mon, 2018-08-06 at 16:46 +0300, Michael S. Tsirkin wrote: > > > > > > > Right, we'll need some quirk to disable balloons in the guest I > > > > suppose. >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Benjamin Herrenschmidt
On Tue, 2018-08-07 at 08:13 +1000, Benjamin Herrenschmidt wrote: > > OK well, assuming Christoph can solve the direct case in a way that > also work for the virtio !iommu case, we still want some bit of logic > somewhere that will "switch" to swiotlb based ops if the DMA mask is > limited. > >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Benjamin Herrenschmidt
On Tue, 2018-08-07 at 00:46 +0300, Michael S. Tsirkin wrote: > On Tue, Aug 07, 2018 at 07:26:35AM +1000, Benjamin Herrenschmidt wrote: > > On Mon, 2018-08-06 at 23:35 +0300, Michael S. Tsirkin wrote: > > > > As I said replying to Christoph, we are "leaking" into the interface > > > > something

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Michael S. Tsirkin
On Tue, Aug 07, 2018 at 07:26:35AM +1000, Benjamin Herrenschmidt wrote: > On Mon, 2018-08-06 at 23:35 +0300, Michael S. Tsirkin wrote: > > > As I said replying to Christoph, we are "leaking" into the interface > > > something here that is really what's the VM is doing to itself, which > > > is to

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Benjamin Herrenschmidt
On Mon, 2018-08-06 at 23:35 +0300, Michael S. Tsirkin wrote: > > As I said replying to Christoph, we are "leaking" into the interface > > something here that is really what's the VM is doing to itself, which > > is to stash its memory away in an inaccessible place. > > > > Cheers, > > Ben. > > I

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Michael S. Tsirkin
On Tue, Aug 07, 2018 at 05:56:59AM +1000, Benjamin Herrenschmidt wrote: > On Mon, 2018-08-06 at 16:46 +0300, Michael S. Tsirkin wrote: > > > > > Right, we'll need some quirk to disable balloons in the guest I > > > suppose. > > > > > > Passing something from libvirt is cumbersome because the

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Benjamin Herrenschmidt
On Mon, 2018-08-06 at 16:46 +0300, Michael S. Tsirkin wrote: > > > Right, we'll need some quirk to disable balloons in the guest I > > suppose. > > > > Passing something from libvirt is cumbersome because the end user may > > not even need to know about secure VMs. There are use cases where the

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Benjamin Herrenschmidt
On Mon, 2018-08-06 at 02:42 -0700, Christoph Hellwig wrote: > On Mon, Aug 06, 2018 at 07:16:47AM +1000, Benjamin Herrenschmidt wrote: > > Who would set this bit ? qemu ? Under what circumstances ? > > I don't really care who sets what. The implementation might not even > involved qemu. > > It

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Christoph Hellwig
On Mon, Aug 06, 2018 at 07:13:32PM +0300, Michael S. Tsirkin wrote: > Oh that makes sense then. Could you post a pointer pls so > this patchset is rebased on top (there are things to > change about 4/4 but 1-3 could go in if they don't add > overhead)? The dma mapping direct calls will need a

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Michael S. Tsirkin
On Mon, Aug 06, 2018 at 09:10:40AM -0700, Christoph Hellwig wrote: > On Mon, Aug 06, 2018 at 07:06:05PM +0300, Michael S. Tsirkin wrote: > > > I've done something very similar in the thread I posted a few years > > > ago. > > > > Right so that was before spectre where a virtual call was cheaper

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Christoph Hellwig
On Mon, Aug 06, 2018 at 07:06:05PM +0300, Michael S. Tsirkin wrote: > > I've done something very similar in the thread I posted a few years > > ago. > > Right so that was before spectre where a virtual call was cheaper :( Sorry, I meant days, not years. The whole point of the thread was the

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Michael S. Tsirkin
On Mon, Aug 06, 2018 at 08:24:06AM -0700, Christoph Hellwig wrote: > On Mon, Aug 06, 2018 at 04:36:43PM +0300, Michael S. Tsirkin wrote: > > On Mon, Aug 06, 2018 at 02:32:28PM +0530, Anshuman Khandual wrote: > > > On 08/05/2018 05:54 AM, Michael S. Tsirkin wrote: > > > > On Fri, Aug 03, 2018 at

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Christoph Hellwig
On Mon, Aug 06, 2018 at 04:36:43PM +0300, Michael S. Tsirkin wrote: > On Mon, Aug 06, 2018 at 02:32:28PM +0530, Anshuman Khandual wrote: > > On 08/05/2018 05:54 AM, Michael S. Tsirkin wrote: > > > On Fri, Aug 03, 2018 at 08:21:26PM -0500, Benjamin Herrenschmidt wrote: > > >> On Fri, 2018-08-03 at

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Will Deacon
Hi Michael, On Sun, Aug 05, 2018 at 03:27:42AM +0300, Michael S. Tsirkin wrote: > On Wed, Aug 01, 2018 at 09:16:38AM +0100, Will Deacon wrote: > > On Tue, Jul 31, 2018 at 03:36:22PM -0500, Benjamin Herrenschmidt wrote: > > > On Tue, 2018-07-31 at 10:30 -0700, Christoph Hellwig wrote: > > > > >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Michael S. Tsirkin
On Sun, Aug 05, 2018 at 02:52:54PM +1000, Benjamin Herrenschmidt wrote: > On Sun, 2018-08-05 at 03:22 +0300, Michael S. Tsirkin wrote: > > I see the allure of this, but I think down the road you will > > discover passing a flag in libvirt XML saying > > "please use a secure mode" or whatever is a

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Christoph Hellwig
On Mon, Aug 06, 2018 at 07:16:47AM +1000, Benjamin Herrenschmidt wrote: > Who would set this bit ? qemu ? Under what circumstances ? I don't really care who sets what. The implementation might not even involved qemu. It is your job to write a coherent interface specification that does not

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-06 Thread Anshuman Khandual
On 08/05/2018 05:54 AM, Michael S. Tsirkin wrote: > On Fri, Aug 03, 2018 at 08:21:26PM -0500, Benjamin Herrenschmidt wrote: >> On Fri, 2018-08-03 at 22:08 +0300, Michael S. Tsirkin wrote: >> Please go through these patches and review whether this approach broadly >> makes sense. I will

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-05 Thread Benjamin Herrenschmidt
On Mon, 2018-08-06 at 07:16 +1000, Benjamin Herrenschmidt wrote: > I'm trying to understand because the limitation is not a device side > limitation, it's not a qemu limitation, it's actually more of a VM > limitation. It has most of its memory pages made inaccessible for > security reasons. The

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-05 Thread Benjamin Herrenschmidt
On Sun, 2018-08-05 at 00:29 -0700, Christoph Hellwig wrote: > On Sun, Aug 05, 2018 at 11:10:15AM +1000, Benjamin Herrenschmidt wrote: > > - One you have rejected, which is to have a way for "no-iommu" virtio > > (which still doesn't use an iommu on the qemu side and doesn't need > > to), to be

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-05 Thread Christoph Hellwig
On Sun, Aug 05, 2018 at 11:10:15AM +1000, Benjamin Herrenschmidt wrote: > - One you have rejected, which is to have a way for "no-iommu" virtio > (which still doesn't use an iommu on the qemu side and doesn't need > to), to be forced to use some custom DMA ops on the VM side. > > - One, which

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-05 Thread Christoph Hellwig
On Sun, Aug 05, 2018 at 03:09:55AM +0300, Michael S. Tsirkin wrote: > So in this case however I'm not sure what exactly do we want to add. It > seems that from point of view of the device, there is nothing special - > it just gets a PA and writes there. It also seems that guest does not > need to

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-04 Thread Benjamin Herrenschmidt
On Sun, 2018-08-05 at 03:22 +0300, Michael S. Tsirkin wrote: > I see the allure of this, but I think down the road you will > discover passing a flag in libvirt XML saying > "please use a secure mode" or whatever is a good idea. > > Even thought it is probably not required to address this >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-04 Thread Benjamin Herrenschmidt
On Sat, 2018-08-04 at 01:21 -0700, Christoph Hellwig wrote: > No matter if you like it or not (I don't!) virtio is defined to bypass > dma translations, it is very clearly stated in the spec. It has some > ill-defined bits to bypass it, so if you want the dma mapping API > to be used you'll have

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-04 Thread Benjamin Herrenschmidt
On Sun, 2018-08-05 at 03:09 +0300, Michael S. Tsirkin wrote: > It seems that the fact that within guest it's implemented using a bounce > buffer and that it's easiest to do by switching virtio to use the DMA API > isn't something virtio spec concerns itself with. Right, this is my reasoning as

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-04 Thread Michael S. Tsirkin
On Wed, Aug 01, 2018 at 09:16:38AM +0100, Will Deacon wrote: > On Tue, Jul 31, 2018 at 03:36:22PM -0500, Benjamin Herrenschmidt wrote: > > On Tue, 2018-07-31 at 10:30 -0700, Christoph Hellwig wrote: > > > > However the question people raise is that DMA API is already full of > > > > arch-specific

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-04 Thread Michael S. Tsirkin
On Fri, Aug 03, 2018 at 08:22:11PM -0500, Benjamin Herrenschmidt wrote: > (Appologies if you got this twice, my mailer had a brain fart and I don't > know if the first one got through & am about to disappear in a plane for 17h) I got like 3 of these. I hope that's true for everyone as I replied

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-04 Thread Michael S. Tsirkin
On Fri, Aug 03, 2018 at 08:21:26PM -0500, Benjamin Herrenschmidt wrote: > On Fri, 2018-08-03 at 22:08 +0300, Michael S. Tsirkin wrote: > > > > > Please go through these patches and review whether this approach > > > > > broadly > > > > > makes sense. I will appreciate suggestions, inputs,

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-04 Thread Michael S. Tsirkin
On Fri, Aug 03, 2018 at 08:16:21PM -0500, Benjamin Herrenschmidt wrote: > On Fri, 2018-08-03 at 22:07 +0300, Michael S. Tsirkin wrote: > > On Fri, Aug 03, 2018 at 10:58:36AM -0500, Benjamin Herrenschmidt wrote: > > > On Fri, 2018-08-03 at 00:05 -0700, Christoph Hellwig wrote: > > > > > 2- Make

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-04 Thread Michael S. Tsirkin
On Sat, Aug 04, 2018 at 01:15:00AM -0700, Christoph Hellwig wrote: > On Fri, Aug 03, 2018 at 10:17:32PM +0300, Michael S. Tsirkin wrote: > > It seems reasonable to teach a platform to override dma-range > > for a specific device e.g. in case it knows about bugs in ACPI. > > A platform will be

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-04 Thread Christoph Hellwig
On Fri, Aug 03, 2018 at 01:58:46PM -0500, Benjamin Herrenschmidt wrote: > You are saying something along the lines of "I don't like an > instruction in your ISA, let's not support your entire CPU architecture > in Linux". No. I'm saying if you can't describe your architecture in the virtio spec

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-04 Thread Christoph Hellwig
On Fri, Aug 03, 2018 at 10:17:32PM +0300, Michael S. Tsirkin wrote: > It seems reasonable to teach a platform to override dma-range > for a specific device e.g. in case it knows about bugs in ACPI. A platform will be able override dma-range using the dev->bus_dma_mask field starting in 4.19. But

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Benjamin Herrenschmidt
On Fri, 2018-08-03 at 22:08 +0300, Michael S. Tsirkin wrote: > > > > Please go through these patches and review whether this approach broadly > > > > makes sense. I will appreciate suggestions, inputs, comments regarding > > > > the patches or the approach in general. Thank you. > > > > > > Jason

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Benjamin Herrenschmidt
On Fri, 2018-08-03 at 22:07 +0300, Michael S. Tsirkin wrote: > On Fri, Aug 03, 2018 at 10:58:36AM -0500, Benjamin Herrenschmidt wrote: > > On Fri, 2018-08-03 at 00:05 -0700, Christoph Hellwig wrote: > > > > 2- Make virtio use the DMA API with our custom platform-provided > > > > swiotlb

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Benjamin Herrenschmidt
On Fri, 2018-08-03 at 22:07 +0300, Michael S. Tsirkin wrote: > On Fri, Aug 03, 2018 at 10:58:36AM -0500, Benjamin Herrenschmidt wrote: > > On Fri, 2018-08-03 at 00:05 -0700, Christoph Hellwig wrote: > > > > 2- Make virtio use the DMA API with our custom platform-provided > > > > swiotlb

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Benjamin Herrenschmidt
On Fri, 2018-08-03 at 22:07 +0300, Michael S. Tsirkin wrote: > On Fri, Aug 03, 2018 at 10:58:36AM -0500, Benjamin Herrenschmidt wrote: > > On Fri, 2018-08-03 at 00:05 -0700, Christoph Hellwig wrote: > > > > 2- Make virtio use the DMA API with our custom platform-provided > > > > swiotlb

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Benjamin Herrenschmidt
On Fri, 2018-08-03 at 22:07 +0300, Michael S. Tsirkin wrote: > On Fri, Aug 03, 2018 at 10:58:36AM -0500, Benjamin Herrenschmidt wrote: > > On Fri, 2018-08-03 at 00:05 -0700, Christoph Hellwig wrote: > > > > 2- Make virtio use the DMA API with our custom platform-provided > > > > swiotlb

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Michael S. Tsirkin
On Fri, Aug 03, 2018 at 12:05:07AM -0700, Christoph Hellwig wrote: > On Thu, Aug 02, 2018 at 04:13:09PM -0500, Benjamin Herrenschmidt wrote: > > So let's differenciate the two problems of having an IOMMU (real or > > emulated) which indeeds adds overhead etc... and using the DMA API. > > > > At

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Michael S. Tsirkin
On Fri, Aug 03, 2018 at 10:41:41AM +0800, Jason Wang wrote: > > > On 2018年08月03日 04:55, Michael S. Tsirkin wrote: > > On Fri, Jul 20, 2018 at 09:29:37AM +0530, Anshuman Khandual wrote: > > > This patch series is the follow up on the discussions we had before about > > > the RFC titled [RFC,V2]

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Benjamin Herrenschmidt
On Fri, 2018-08-03 at 09:02 -0700, Christoph Hellwig wrote: > On Fri, Aug 03, 2018 at 10:58:36AM -0500, Benjamin Herrenschmidt wrote: > > On Fri, 2018-08-03 at 00:05 -0700, Christoph Hellwig wrote: > > > > 2- Make virtio use the DMA API with our custom platform-provided > > > > swiotlb callbacks

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Christoph Hellwig
On Fri, Aug 03, 2018 at 10:58:36AM -0500, Benjamin Herrenschmidt wrote: > On Fri, 2018-08-03 at 00:05 -0700, Christoph Hellwig wrote: > > > 2- Make virtio use the DMA API with our custom platform-provided > > > swiotlb callbacks when needed, that is when not using IOMMU *and* > > > running on a

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Benjamin Herrenschmidt
On Fri, 2018-08-03 at 00:05 -0700, Christoph Hellwig wrote: > > 2- Make virtio use the DMA API with our custom platform-provided > > swiotlb callbacks when needed, that is when not using IOMMU *and* > > running on a secure VM in our case. > > And total NAK the customer platform-provided part of

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Christoph Hellwig
On Thu, Aug 02, 2018 at 11:53:08PM +0300, Michael S. Tsirkin wrote: > > We don't need cache flushing tricks. > > You don't but do real devices on same platform need them? IBMs power plaforms are always cache coherent. There are some powerpc platforms have not cache coherent DMA, but I guess

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-03 Thread Christoph Hellwig
On Thu, Aug 02, 2018 at 04:13:09PM -0500, Benjamin Herrenschmidt wrote: > So let's differenciate the two problems of having an IOMMU (real or > emulated) which indeeds adds overhead etc... and using the DMA API. > > At the moment, virtio does this all over the place: > > if (use_dma_api) >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Jason Wang
On 2018年08月03日 04:55, Michael S. Tsirkin wrote: On Fri, Jul 20, 2018 at 09:29:37AM +0530, Anshuman Khandual wrote: This patch series is the follow up on the discussions we had before about the RFC titled [RFC,V2] virtio: Add platform specific DMA API translation for virito devices

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Michael S. Tsirkin
On Thu, Aug 02, 2018 at 04:13:09PM -0500, Benjamin Herrenschmidt wrote: > On Thu, 2018-08-02 at 23:52 +0300, Michael S. Tsirkin wrote: > > > Yes, this is the purpose of Anshuman original patch (I haven't looked > > > at the details of the patch in a while but that's what I told him to > > >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Benjamin Herrenschmidt
On Thu, 2018-08-02 at 23:52 +0300, Michael S. Tsirkin wrote: > > Yes, this is the purpose of Anshuman original patch (I haven't looked > > at the details of the patch in a while but that's what I told him to > > implement ;-) : > > > > - Make virtio always use DMA ops to simplify the code path

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Michael S. Tsirkin
On Fri, Jul 20, 2018 at 09:29:37AM +0530, Anshuman Khandual wrote: > This patch series is the follow up on the discussions we had before about > the RFC titled [RFC,V2] virtio: Add platform specific DMA API translation > for virito devices (https://patchwork.kernel.org/patch/10417371/). There >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Michael S. Tsirkin
On Thu, Aug 02, 2018 at 10:33:05AM -0500, Benjamin Herrenschmidt wrote: > On Thu, 2018-08-02 at 00:56 +0300, Michael S. Tsirkin wrote: > > > but it's not, VMs are > > > created in "legacy" mode all the times and we don't know at VM creation > > > time whether it will become a secure VM or not. The

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Michael S. Tsirkin
On Thu, Aug 02, 2018 at 12:53:41PM -0500, Benjamin Herrenschmidt wrote: > On Thu, 2018-08-02 at 20:19 +0300, Michael S. Tsirkin wrote: > > > > I see. So yes, given that device does not know or care, using > > virtio features is an awkward fit. > > > > So let's say as a quick fix for you maybe we

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Benjamin Herrenschmidt
On Thu, 2018-08-02 at 20:19 +0300, Michael S. Tsirkin wrote: > > I see. So yes, given that device does not know or care, using > virtio features is an awkward fit. > > So let's say as a quick fix for you maybe we could generalize the > xen_domain hack, instead of just checking xen_domain check

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Michael S. Tsirkin
On Thu, Aug 02, 2018 at 11:01:26AM -0500, Benjamin Herrenschmidt wrote: > On Thu, 2018-08-02 at 18:41 +0300, Michael S. Tsirkin wrote: > > > > > I don't completely agree: > > > > > > 1 - VIRTIO_F_IOMMU_PLATFORM is a property of the "other side", ie qemu > > > for example. It indicates that the

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Benjamin Herrenschmidt
On Thu, 2018-08-02 at 18:41 +0300, Michael S. Tsirkin wrote: > > > I don't completely agree: > > > > 1 - VIRTIO_F_IOMMU_PLATFORM is a property of the "other side", ie qemu > > for example. It indicates that the peer bypasses the normal platform > > iommu. The platform code in the guest has no

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Michael S. Tsirkin
On Thu, Aug 02, 2018 at 10:24:57AM -0500, Benjamin Herrenschmidt wrote: > On Wed, 2018-08-01 at 01:36 -0700, Christoph Hellwig wrote: > > We just need to figure out how to deal with devices that deviate > > from the default. One things is that VIRTIO_F_IOMMU_PLATFORM really > > should become

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Benjamin Herrenschmidt
On Thu, 2018-08-02 at 00:56 +0300, Michael S. Tsirkin wrote: > > but it's not, VMs are > > created in "legacy" mode all the times and we don't know at VM creation > > time whether it will become a secure VM or not. The way our secure VMs > > work is that they start as a normal VM, load a secure

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-02 Thread Benjamin Herrenschmidt
On Wed, 2018-08-01 at 01:36 -0700, Christoph Hellwig wrote: > We just need to figure out how to deal with devices that deviate > from the default. One things is that VIRTIO_F_IOMMU_PLATFORM really > should become VIRTIO_F_PLATFORM_DMA to cover the cases of non-iommu > dma tweaks (offsets, cache

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-01 Thread Michael S. Tsirkin
On Wed, Aug 01, 2018 at 10:05:35AM +0100, Will Deacon wrote: > Hi Christoph, > > On Wed, Aug 01, 2018 at 01:36:39AM -0700, Christoph Hellwig wrote: > > On Wed, Aug 01, 2018 at 09:16:38AM +0100, Will Deacon wrote: > > > On arm/arm64, the problem we have is that legacy virtio devices on the > > >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-01 Thread Michael S. Tsirkin
On Wed, Aug 01, 2018 at 01:36:39AM -0700, Christoph Hellwig wrote: > On Wed, Aug 01, 2018 at 09:16:38AM +0100, Will Deacon wrote: > > On arm/arm64, the problem we have is that legacy virtio devices on the MMIO > > transport (so definitely not PCI) have historically been advertised by qemu > > as

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-01 Thread Michael S. Tsirkin
On Tue, Jul 31, 2018 at 03:36:22PM -0500, Benjamin Herrenschmidt wrote: > On Tue, 2018-07-31 at 10:30 -0700, Christoph Hellwig wrote: > > > However the question people raise is that DMA API is already full of > > > arch-specific tricks the likes of which are outlined in your post linked > > >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-01 Thread Will Deacon
Hi Christoph, On Wed, Aug 01, 2018 at 01:36:39AM -0700, Christoph Hellwig wrote: > On Wed, Aug 01, 2018 at 09:16:38AM +0100, Will Deacon wrote: > > On arm/arm64, the problem we have is that legacy virtio devices on the MMIO > > transport (so definitely not PCI) have historically been advertised

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-01 Thread Christoph Hellwig
On Wed, Aug 01, 2018 at 09:16:38AM +0100, Will Deacon wrote: > On arm/arm64, the problem we have is that legacy virtio devices on the MMIO > transport (so definitely not PCI) have historically been advertised by qemu > as not being cache coherent, but because the virtio core has bypassed DMA > ops

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-08-01 Thread Will Deacon
On Tue, Jul 31, 2018 at 03:36:22PM -0500, Benjamin Herrenschmidt wrote: > On Tue, 2018-07-31 at 10:30 -0700, Christoph Hellwig wrote: > > > However the question people raise is that DMA API is already full of > > > arch-specific tricks the likes of which are outlined in your post linked > > >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-31 Thread Benjamin Herrenschmidt
On Tue, 2018-07-31 at 10:30 -0700, Christoph Hellwig wrote: > > However the question people raise is that DMA API is already full of > > arch-specific tricks the likes of which are outlined in your post linked > > above. How is this one much worse? > > None of these warts is visible to the

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-30 Thread Michael S. Tsirkin
On Mon, Jul 30, 2018 at 04:18:02AM -0700, Christoph Hellwig wrote: > On Mon, Jul 30, 2018 at 01:28:03PM +0300, Michael S. Tsirkin wrote: > > Let me reply to the "crappy" part first: > > So virtio devices can run on another CPU or on a PCI bus. Configuration > > can happen over mupltiple

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-30 Thread Christoph Hellwig
On Mon, Jul 30, 2018 at 01:28:03PM +0300, Michael S. Tsirkin wrote: > Let me reply to the "crappy" part first: > So virtio devices can run on another CPU or on a PCI bus. Configuration > can happen over mupltiple transports. There is a discovery protocol to > figure out where it is. It has some

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-30 Thread Michael S. Tsirkin
On Mon, Jul 30, 2018 at 02:34:14AM -0700, Christoph Hellwig wrote: > We really need to distinguish between legacy virtual crappy > virtio (and that includes v1) that totally ignores the bus it pretends > to be on, and sane virtio (to be defined) that sit on a real (or > properly emulated including

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-30 Thread Christoph Hellwig
On Fri, Jul 27, 2018 at 10:58:05AM +0100, Will Deacon wrote: > > I just wanted to say that this patch series provides a means for us to > force the coherent DMA ops for legacy virtio devices on arm64, which in turn > means that we can enable the SMMU with legacy devices in our fastmodel >

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-28 Thread Anshuman Khandual
On 07/27/2018 05:01 PM, Michael S. Tsirkin wrote: > On Wed, Jul 25, 2018 at 08:56:23AM +0530, Anshuman Khandual wrote: >> Results with and without the patches are similar. > > Thanks! And another thing to try is virtio-net with > a fast NIC backend (40G and up). Unfortunately > at this point

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-27 Thread Michael S. Tsirkin
On Wed, Jul 25, 2018 at 08:56:23AM +0530, Anshuman Khandual wrote: > Results with and without the patches are similar. Thanks! And another thing to try is virtio-net with a fast NIC backend (40G and up). Unfortunately at this point loopback tests stress the host scheduler too much. -- MST

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-27 Thread Anshuman Khandual
On 07/27/2018 03:28 PM, Will Deacon wrote: > Hi Anshuman, > > On Fri, Jul 20, 2018 at 09:29:37AM +0530, Anshuman Khandual wrote: >> This patch series is the follow up on the discussions we had before about >> the RFC titled [RFC,V2] virtio: Add platform specific DMA API translation >> for virito

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-27 Thread Will Deacon
Hi Anshuman, On Fri, Jul 20, 2018 at 09:29:37AM +0530, Anshuman Khandual wrote: > This patch series is the follow up on the discussions we had before about > the RFC titled [RFC,V2] virtio: Add platform specific DMA API translation > for virito devices

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-24 Thread Anshuman Khandual
On 07/23/2018 02:38 PM, Michael S. Tsirkin wrote: > On Mon, Jul 23, 2018 at 11:58:23AM +0530, Anshuman Khandual wrote: >> On 07/20/2018 06:46 PM, Michael S. Tsirkin wrote: >>> On Fri, Jul 20, 2018 at 09:29:37AM +0530, Anshuman Khandual wrote: This patch series is the follow up on the

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-23 Thread Michael S. Tsirkin
On Mon, Jul 23, 2018 at 11:58:23AM +0530, Anshuman Khandual wrote: > On 07/20/2018 06:46 PM, Michael S. Tsirkin wrote: > > On Fri, Jul 20, 2018 at 09:29:37AM +0530, Anshuman Khandual wrote: > >> This patch series is the follow up on the discussions we had before about > >> the RFC titled [RFC,V2]

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-23 Thread Anshuman Khandual
On 07/20/2018 06:46 PM, Michael S. Tsirkin wrote: > On Fri, Jul 20, 2018 at 09:29:37AM +0530, Anshuman Khandual wrote: >> This patch series is the follow up on the discussions we had before about >> the RFC titled [RFC,V2] virtio: Add platform specific DMA API translation >> for virito devices

Re: [RFC 0/4] Virtio uses DMA API for all devices

2018-07-20 Thread Michael S. Tsirkin
On Fri, Jul 20, 2018 at 09:29:37AM +0530, Anshuman Khandual wrote: > This patch series is the follow up on the discussions we had before about > the RFC titled [RFC,V2] virtio: Add platform specific DMA API translation > for virito devices (https://patchwork.kernel.org/patch/10417371/). There >