RE: [PATCH 2/2] virtio: let virtio use DMA API when guest RAM is protected
On Thu, Feb 20, 2020 at 03:55:14PM -0500, Michael S. Tsirkin wrote: > On Thu, Feb 20, 2020 at 05:06:06PM +0100, Halil Pasic wrote: > > Currently the advanced guest memory protection technologies (AMD SEV, > > powerpc secure guest technology and s390 Protected VMs) abuse the > > VIRTIO_F_IOMMU_PLATFORM flag to make virtio core use the DMA API, which > > is in turn necessary, to make IO work with guest memory protection. > > > > But VIRTIO_F_IOMMU_PLATFORM a.k.a. VIRTIO_F_ACCESS_PLATFORM is really a > > different beast: with virtio devices whose implementation runs on an SMP > > CPU we are still fine with doing all the usual optimizations, it is just > > that we need to make sure that the memory protection mechanism does not > > get in the way. The VIRTIO_F_ACCESS_PLATFORM mandates more work on the > > side of the guest (and possibly he host side as well) than we actually > > need. > > > > An additional benefit of teaching the guest to make the right decision > > (and use DMA API) on it's own is: removing the need, to mandate special > > VM configuration for guests that may run with protection. This is > > especially interesting for s390 as VIRTIO_F_IOMMU_PLATFORM pushes all > > the virtio control structures into the first 2G of guest memory: > > something we don't necessarily want to do per-default. > > > > Signed-off-by: Halil Pasic > > Tested-by: Ram Pai > > Tested-by: Michael Mueller > > This might work for you but it's fragile, since without > VIRTIO_F_ACCESS_PLATFORM hypervisor assumes it gets > GPA's, not DMA addresses. > > > > IOW this looks like another iteration of: > > virtio: Support encrypted memory on powerpc secure guests > > which I was under the impression was abandoned as unnecessary. It has been abondoned on powerpc. We enabled VIRTIO_F_ACCESS_PLATFORM; by default, flag on powerpc. We would like to enable secure guests on powerpc without this flag aswell enabled, but past experience has educated us that its not a easy path. However if Halil makes some inroads in this path for s390, we will like to support him. RP ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH 2/2] virtio_ring: Use DMA API if memory is encrypted
On Tue, Oct 15, 2019 at 09:35:01AM +0200, Christoph Hellwig wrote:
> On Fri, Oct 11, 2019 at 06:25:19PM -0700, Ram Pai wrote:
> > From: Thiago Jung Bauermann
> >
> > Normally, virtio enables DMA API with VIRTIO_F_IOMMU_PLATFORM, which must
> > be set by both device and guest driver. However, as a hack, when DMA API
> > returns physical addresses, guest driver can use the DMA API; even though
> > device does not set VIRTIO_F_IOMMU_PLATFORM and just uses physical
> > addresses.
>
> Sorry, but this is a complete bullshit hack. Driver must always use
> the DMA API if they do DMA, and if virtio devices use physical addresses
> that needs to be returned through the platform firmware interfaces for
> the dma setup. If you don't do that yet (which based on previous
> informations you don't), you need to fix it, and we can then quirk
> old implementations that already are out in the field.
>
> In other words: we finally need to fix that virtio mess and not pile
> hacks on top of hacks.
So force all virtio devices to use DMA API, except when
VIRTIO_F_IOMMU_PLATFORM is not enabled?
Any help detailing the idea, will enable us fix this issue once for all.
Will something like below work? It removes the prior hacks, and
always uses DMA API; except when VIRTIO_F_IOMMU_PLATFORM is not enabled.
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index c8be1c4..b593d3d 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -240,22 +240,10 @@ static inline bool virtqueue_use_indirect(struct
virtqueue *_vq,
static bool vring_use_dma_api(struct virtio_device *vdev)
{
- if (!virtio_has_iommu_quirk(vdev))
- return true;
-
- /* Otherwise, we are left to guess. */
- /*
-* In theory, it's possible to have a buggy QEMU-supposed
-* emulated Q35 IOMMU and Xen enabled at the same time. On
-* such a configuration, virtio has never worked and will
-* not work without an even larger kludge. Instead, enable
-* the DMA API if we're a Xen guest, which at least allows
-* all of the sensible Xen configurations to work correctly.
-*/
- if (xen_domain())
- return true;
+ if (virtio_has_iommu_quirk(vdev))
+ return false;
- return false;
+ return true;
}
size_t virtio_max_dma_size(struct virtio_device *vdev)
--
Ram Pai
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH 1/2] dma-mapping: Add dma_addr_is_phys_addr()
On Mon, Oct 14, 2019 at 11:29:24AM +0100, Robin Murphy wrote: > On 14/10/2019 05:51, David Gibson wrote: > >On Fri, Oct 11, 2019 at 06:25:18PM -0700, Ram Pai wrote: > >>From: Thiago Jung Bauermann > >> > >>In order to safely use the DMA API, virtio needs to know whether DMA > >>addresses are in fact physical addresses and for that purpose, > >>dma_addr_is_phys_addr() is introduced. > >> > >>cc: Benjamin Herrenschmidt > >>cc: David Gibson > >>cc: Michael Ellerman > >>cc: Paul Mackerras > >>cc: Michael Roth > >>cc: Alexey Kardashevskiy > >>cc: Paul Burton > >>cc: Robin Murphy > >>cc: Bartlomiej Zolnierkiewicz > >>cc: Marek Szyprowski > >>cc: Christoph Hellwig > >>Suggested-by: Michael S. Tsirkin > >>Signed-off-by: Ram Pai > >>Signed-off-by: Thiago Jung Bauermann > > > >The change itself looks ok, so > > > >Reviewed-by: David Gibson > > > >However, I would like to see the commit message (and maybe the inline > >comments) expanded a bit on what the distinction here is about. Some > >of the text from the next patch would be suitable, about DMA addresses > >usually being in a different address space but not in the case of > >bounce buffering. > > Right, this needs a much tighter definition. "DMA address happens to > be a valid physical address" is true of various IOMMU setups too, > but I can't believe it's meaningful in such cases. The definition by itself is meaningful AFAICT. At its core its just indicating whether DMA addresses are physical addresses or not. However its up to the caller to use it meaningfully. For non-virtio caller, dma_addr_is_phys_addr() by itself may or may not be meaningful. But for a virtio caller, this information when paired with mem_encrypt_active() is meaningful. For IOMMU setups DMA API will get used regardless of "DMA address happens to be a valid physical address" > > If what you actually want is "DMA is direct or SWIOTLB" - i.e. "DMA > address is physical address of DMA data (not necessarily the > original buffer)" - wouldn't dma_is_direct() suffice? This may also work, I think. MST: thoughts? RP ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH 0/2] virtio: Support encrypted memory on powerpc secure guests
Hmm.. git-send-email forgot to CC Michael Tsirkin, and Thiago; the original author, who is on vacation. Adding them now. RP On Fri, Oct 11, 2019 at 06:25:17PM -0700, Ram Pai wrote: > **We would like the patches to be merged through the virtio tree. Please > review, and ack merging the DMA mapping change through that tree. Thanks!** > > The memory of powerpc secure guests can't be accessed by the hypervisor / > virtio device except for a few memory regions designated as 'shared'. > > At the moment, Linux uses bounce-buffering to communicate with the > hypervisor, with a bounce buffer marked as shared. This is how the DMA API > is implemented on this platform. > > In particular, the most convenient way to use virtio on this platform is by > making virtio use the DMA API: in fact, this is exactly what happens if the > virtio device exposes the flag VIRTIO_F_ACCESS_PLATFORM. However, bugs in > the > hypervisor on the powerpc platform do not allow setting this flag, with some > hypervisors already in the field that don't set this flag. At the moment they > are forced to use emulated devices when guest is in secure mode; virtio is > only useful when guest is not secure. > > Normally, both device and driver must support VIRTIO_F_ACCESS_PLATFORM: > if one of them doesn't, the other mustn't assume it for communication > to work. > > However, a guest-side work-around is possible to enable virtio > for these hypervisors with guest in secure mode: it so happens that on > powerpc secure platform the DMA address is actually a physical address - > that of the bounce buffer. For these platforms we can make the virtio > driver go through the DMA API even though the device itself ignores > the DMA API. > > These patches implement this work around for virtio: we detect that > - secure guest mode is enabled - so we know that since we don't share >most memory and Hypervisor has not enabled VIRTIO_F_ACCESS_PLATFORM, >regular virtio code won't work. > - DMA API is giving us addresses that are actually also physical >addresses. > - Hypervisor has not enabled VIRTIO_F_ACCESS_PLATFORM. > > and if all conditions are true, we force all data through the bounce > buffer. > > To put it another way, from hypervisor's point of view DMA API is > not required: hypervisor would be happy to get access to all of guest > memory. That's why it does not set VIRTIO_F_ACCESS_PLATFORM. However, > guest decides that it does not trust the hypervisor and wants to force > a bounce buffer for its own reasons. > > > Thiago Jung Bauermann (2): > dma-mapping: Add dma_addr_is_phys_addr() > virtio_ring: Use DMA API if memory is encrypted > > arch/powerpc/include/asm/dma-mapping.h | 21 + > arch/powerpc/platforms/pseries/Kconfig | 1 + > drivers/virtio/virtio.c| 18 ++ > drivers/virtio/virtio_ring.c | 8 > include/linux/dma-mapping.h| 20 > include/linux/virtio_config.h | 14 ++ > kernel/dma/Kconfig | 3 +++ > 7 files changed, 85 insertions(+) > > -- > 1.8.3.1 -- Ram Pai ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH 2/2] virtio_ring: Use DMA API if memory is encrypted
From: Thiago Jung Bauermann
Normally, virtio enables DMA API with VIRTIO_F_IOMMU_PLATFORM, which must
be set by both device and guest driver. However, as a hack, when DMA API
returns physical addresses, guest driver can use the DMA API; even though
device does not set VIRTIO_F_IOMMU_PLATFORM and just uses physical
addresses.
Doing this works-around POWER secure guests for which only the bounce
buffer is accessible to the device, but which don't set
VIRTIO_F_IOMMU_PLATFORM due to a set of hypervisor and architectural bugs.
To guard against platform changes, breaking any of these assumptions down
the road, we check at probe time and fail if that's not the case.
cc: Benjamin Herrenschmidt
cc: David Gibson
cc: Michael Ellerman
cc: Paul Mackerras
cc: Michael Roth
cc: Alexey Kardashevskiy
cc: Jason Wang
cc: Christoph Hellwig
Suggested-by: Michael S. Tsirkin
Signed-off-by: Ram Pai
Signed-off-by: Thiago Jung Bauermann
---
drivers/virtio/virtio.c | 18 ++
drivers/virtio/virtio_ring.c | 8
include/linux/virtio_config.h | 14 ++
3 files changed, 40 insertions(+)
diff --git a/drivers/virtio/virtio.c b/drivers/virtio/virtio.c
index a977e32..77a3baf 100644
--- a/drivers/virtio/virtio.c
+++ b/drivers/virtio/virtio.c
@@ -4,6 +4,7 @@
#include
#include
#include
+#include
#include
/* Unique numbering for virtio devices. */
@@ -245,6 +246,23 @@ static int virtio_dev_probe(struct device *_d)
if (err)
goto err;
+ /*
+* If memory is encrypted, but VIRTIO_F_IOMMU_PLATFORM is not set, then
+* the device is broken: DMA API is required for these platforms, but
+* the only way using the DMA API is going to work at all is if the
+* device is ready for it. So we need a flag on the virtio device,
+* exposed by the hypervisor (or hardware for hw virtio devices) that
+* says: hey, I'm real, don't take a shortcut.
+*
+* There's one exception where guest can make things work, and that is
+* when DMA API is guaranteed to always return physical addresses.
+*/
+ if (mem_encrypt_active() && !virtio_can_use_dma_api(dev)) {
+ dev_err(_d, "virtio: device unable to access encrypted
memory\n");
+ err = -EINVAL;
+ goto err;
+ }
+
err = drv->probe(dev);
if (err)
goto err;
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index c8be1c4..9c56b61 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -255,6 +255,14 @@ static bool vring_use_dma_api(struct virtio_device *vdev)
if (xen_domain())
return true;
+ /*
+* Also, if guest memory is encrypted the host can't access it
+* directly. We need to either use an IOMMU or do bounce buffering.
+* Both are done via the DMA API.
+*/
+ if (mem_encrypt_active() && virtio_can_use_dma_api(vdev))
+ return true;
+
return false;
}
diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index bb4cc49..57bc25c 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -4,6 +4,7 @@
#include
#include
+#include
#include
#include
#include
@@ -174,6 +175,19 @@ static inline bool virtio_has_iommu_quirk(const struct
virtio_device *vdev)
return !virtio_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM);
}
+/**
+ * virtio_can_use_dma_api - determine whether the DMA API can be used
+ * @vdev: the device
+ *
+ * The DMA API can be used either when the device doesn't have the IOMMU quirk,
+ * or when the DMA API is guaranteed to always return physical addresses.
+ */
+static inline bool virtio_can_use_dma_api(const struct virtio_device *vdev)
+{
+ return !virtio_has_iommu_quirk(vdev) ||
+ dma_addr_is_phys_addr(vdev->dev.parent);
+}
+
static inline
struct virtqueue *virtio_find_single_vq(struct virtio_device *vdev,
vq_callback_t *c, const char *n)
--
1.8.3.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH 1/2] dma-mapping: Add dma_addr_is_phys_addr()
From: Thiago Jung Bauermann
In order to safely use the DMA API, virtio needs to know whether DMA
addresses are in fact physical addresses and for that purpose,
dma_addr_is_phys_addr() is introduced.
cc: Benjamin Herrenschmidt
cc: David Gibson
cc: Michael Ellerman
cc: Paul Mackerras
cc: Michael Roth
cc: Alexey Kardashevskiy
cc: Paul Burton
cc: Robin Murphy
cc: Bartlomiej Zolnierkiewicz
cc: Marek Szyprowski
cc: Christoph Hellwig
Suggested-by: Michael S. Tsirkin
Signed-off-by: Ram Pai
Signed-off-by: Thiago Jung Bauermann
---
arch/powerpc/include/asm/dma-mapping.h | 21 +
arch/powerpc/platforms/pseries/Kconfig | 1 +
include/linux/dma-mapping.h| 20
kernel/dma/Kconfig | 3 +++
4 files changed, 45 insertions(+)
diff --git a/arch/powerpc/include/asm/dma-mapping.h
b/arch/powerpc/include/asm/dma-mapping.h
index 565d6f7..f92c0a4b 100644
--- a/arch/powerpc/include/asm/dma-mapping.h
+++ b/arch/powerpc/include/asm/dma-mapping.h
@@ -5,6 +5,8 @@
#ifndef _ASM_DMA_MAPPING_H
#define _ASM_DMA_MAPPING_H
+#include
+
static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
/* We don't handle the NULL dev case for ISA for now. We could
@@ -15,4 +17,23 @@ static inline const struct dma_map_ops
*get_arch_dma_ops(struct bus_type *bus)
return NULL;
}
+#ifdef CONFIG_ARCH_HAS_DMA_ADDR_IS_PHYS_ADDR
+/**
+ * dma_addr_is_phys_addr - check whether a device DMA address is a physical
+ * address
+ * @dev: device to check
+ *
+ * Returns %true if any DMA address for this device happens to also be a valid
+ * physical address (not necessarily of the same page).
+ */
+static inline bool dma_addr_is_phys_addr(struct device *dev)
+{
+ /*
+* Secure guests always use the SWIOTLB, therefore DMA addresses are
+* actually the physical address of the bounce buffer.
+*/
+ return is_secure_guest();
+}
+#endif
+
#endif /* _ASM_DMA_MAPPING_H */
diff --git a/arch/powerpc/platforms/pseries/Kconfig
b/arch/powerpc/platforms/pseries/Kconfig
index 9e35cdd..0108150 100644
--- a/arch/powerpc/platforms/pseries/Kconfig
+++ b/arch/powerpc/platforms/pseries/Kconfig
@@ -152,6 +152,7 @@ config PPC_SVM
select SWIOTLB
select ARCH_HAS_MEM_ENCRYPT
select ARCH_HAS_FORCE_DMA_UNENCRYPTED
+ select ARCH_HAS_DMA_ADDR_IS_PHYS_ADDR
help
There are certain POWER platforms which support secure guests using
the Protected Execution Facility, with the help of an Ultravisor
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index f7d1eea..6df5664 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -693,6 +693,26 @@ static inline bool dma_addressing_limited(struct device
*dev)
dma_get_required_mask(dev);
}
+#ifndef CONFIG_ARCH_HAS_DMA_ADDR_IS_PHYS_ADDR
+/**
+ * dma_addr_is_phys_addr - check whether a device DMA address is a physical
+ * address
+ * @dev: device to check
+ *
+ * Returns %true if any DMA address for this device happens to also be a valid
+ * physical address (not necessarily of the same page).
+ */
+static inline bool dma_addr_is_phys_addr(struct device *dev)
+{
+ /*
+* Except in very specific setups, DMA addresses exist in a different
+* address space from CPU physical addresses and cannot be directly used
+* to reference system memory.
+*/
+ return false;
+}
+#endif
+
#ifdef CONFIG_ARCH_HAS_SETUP_DMA_OPS
void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
const struct iommu_ops *iommu, bool coherent);
diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig
index 9decbba..6209b46 100644
--- a/kernel/dma/Kconfig
+++ b/kernel/dma/Kconfig
@@ -51,6 +51,9 @@ config ARCH_HAS_DMA_MMAP_PGPROT
config ARCH_HAS_FORCE_DMA_UNENCRYPTED
bool
+config ARCH_HAS_DMA_ADDR_IS_PHYS_ADDR
+ bool
+
config DMA_NONCOHERENT_CACHE_SYNC
bool
--
1.8.3.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH 0/2] virtio: Support encrypted memory on powerpc secure guests
**We would like the patches to be merged through the virtio tree. Please review, and ack merging the DMA mapping change through that tree. Thanks!** The memory of powerpc secure guests can't be accessed by the hypervisor / virtio device except for a few memory regions designated as 'shared'. At the moment, Linux uses bounce-buffering to communicate with the hypervisor, with a bounce buffer marked as shared. This is how the DMA API is implemented on this platform. In particular, the most convenient way to use virtio on this platform is by making virtio use the DMA API: in fact, this is exactly what happens if the virtio device exposes the flag VIRTIO_F_ACCESS_PLATFORM. However, bugs in the hypervisor on the powerpc platform do not allow setting this flag, with some hypervisors already in the field that don't set this flag. At the moment they are forced to use emulated devices when guest is in secure mode; virtio is only useful when guest is not secure. Normally, both device and driver must support VIRTIO_F_ACCESS_PLATFORM: if one of them doesn't, the other mustn't assume it for communication to work. However, a guest-side work-around is possible to enable virtio for these hypervisors with guest in secure mode: it so happens that on powerpc secure platform the DMA address is actually a physical address - that of the bounce buffer. For these platforms we can make the virtio driver go through the DMA API even though the device itself ignores the DMA API. These patches implement this work around for virtio: we detect that - secure guest mode is enabled - so we know that since we don't share most memory and Hypervisor has not enabled VIRTIO_F_ACCESS_PLATFORM, regular virtio code won't work. - DMA API is giving us addresses that are actually also physical addresses. - Hypervisor has not enabled VIRTIO_F_ACCESS_PLATFORM. and if all conditions are true, we force all data through the bounce buffer. To put it another way, from hypervisor's point of view DMA API is not required: hypervisor would be happy to get access to all of guest memory. That's why it does not set VIRTIO_F_ACCESS_PLATFORM. However, guest decides that it does not trust the hypervisor and wants to force a bounce buffer for its own reasons. Thiago Jung Bauermann (2): dma-mapping: Add dma_addr_is_phys_addr() virtio_ring: Use DMA API if memory is encrypted arch/powerpc/include/asm/dma-mapping.h | 21 + arch/powerpc/platforms/pseries/Kconfig | 1 + drivers/virtio/virtio.c| 18 ++ drivers/virtio/virtio_ring.c | 8 include/linux/dma-mapping.h| 20 include/linux/virtio_config.h | 14 ++ kernel/dma/Kconfig | 3 +++ 7 files changed, 85 insertions(+) -- 1.8.3.1 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted
On Tue, Aug 13, 2019 at 08:45:37AM -0700, Ram Pai wrote: > On Wed, Aug 14, 2019 at 12:24:39AM +1000, David Gibson wrote: > > On Tue, Aug 13, 2019 at 03:26:17PM +0200, Christoph Hellwig wrote: > > > On Mon, Aug 12, 2019 at 07:51:56PM +1000, David Gibson wrote: > > > > AFAICT we already kind of abuse this for the VIRTIO_F_IOMMU_PLATFORM, > > > > because to handle for cases where it *is* a device limitation, we > > > > assume that if the hypervisor presents VIRTIO_F_IOMMU_PLATFORM then > > > > the guest *must* select it. > > > > > > > > What we actually need here is for the hypervisor to present > > > > VIRTIO_F_IOMMU_PLATFORM as available, but not required. Then we need > > > > a way for the platform core code to communicate to the virtio driver > > > > that *it* requires the IOMMU to be used, so that the driver can select > > > > or not the feature bit on that basis. > > > > > > I agree with the above, but that just brings us back to the original > > > issue - the whole bypass of the DMA OPS should be an option that the > > > device can offer, not the other way around. And we really need to > > > fix that root cause instead of doctoring around it. > > > > I'm not exactly sure what you mean by "device" in this context. Do > > you mean the hypervisor (qemu) side implementation? > > > > You're right that this was the wrong way around to begin with, but as > > well as being hard to change now, I don't see how it really addresses > > the current problem. The device could default to IOMMU and allow > > bypass, but the driver would still need to get information from the > > platform to know that it *can't* accept that option in the case of a > > secure VM. Reversed sense, but the same basic problem. > > > > The hypervisor does not, and can not be aware of the secure VM > > restrictions - only the guest side platform code knows that. > > This statement is almost entirely right. I will rephrase it to make it > entirely right. > > The hypervisor does not, and can not be aware of the secure VM > requirement that it needs to do some special processing that has nothing > to do with DMA address translation - only the guest side platform code > know that. > > BTW: I do not consider 'bounce buffering' as 'DMA address translation'. > DMA address translation, translates CPU address to DMA address. Bounce > buffering moves the data from one buffer at a given CPU address to > another buffer at a different CPU address. Unfortunately the current > DMA ops conflates the two. The need to do 'DMA address translation' > is something the device can enforce. But the need to do bounce > buffering, is something that the device should not be aware and should be > entirely a decision made locally by the kernel/driver in the secure VM. Christoph, Since we have not heard back from you, I am not sure where you stand on this issue now. One of the three things are possible.. (a) our above explaination did not make sense and hence you decided to ignore it. (b) our above above made some sense and need more time to think and respond. (c) you totally forgot about this. I hope it is (b). We want a solution that works for all, and your inputs are important to us. Thanks, RP ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted
On Wed, Aug 14, 2019 at 12:24:39AM +1000, David Gibson wrote: > On Tue, Aug 13, 2019 at 03:26:17PM +0200, Christoph Hellwig wrote: > > On Mon, Aug 12, 2019 at 07:51:56PM +1000, David Gibson wrote: > > > AFAICT we already kind of abuse this for the VIRTIO_F_IOMMU_PLATFORM, > > > because to handle for cases where it *is* a device limitation, we > > > assume that if the hypervisor presents VIRTIO_F_IOMMU_PLATFORM then > > > the guest *must* select it. > > > > > > What we actually need here is for the hypervisor to present > > > VIRTIO_F_IOMMU_PLATFORM as available, but not required. Then we need > > > a way for the platform core code to communicate to the virtio driver > > > that *it* requires the IOMMU to be used, so that the driver can select > > > or not the feature bit on that basis. > > > > I agree with the above, but that just brings us back to the original > > issue - the whole bypass of the DMA OPS should be an option that the > > device can offer, not the other way around. And we really need to > > fix that root cause instead of doctoring around it. > > I'm not exactly sure what you mean by "device" in this context. Do > you mean the hypervisor (qemu) side implementation? > > You're right that this was the wrong way around to begin with, but as > well as being hard to change now, I don't see how it really addresses > the current problem. The device could default to IOMMU and allow > bypass, but the driver would still need to get information from the > platform to know that it *can't* accept that option in the case of a > secure VM. Reversed sense, but the same basic problem. > > The hypervisor does not, and can not be aware of the secure VM > restrictions - only the guest side platform code knows that. This statement is almost entirely right. I will rephrase it to make it entirely right. The hypervisor does not, and can not be aware of the secure VM requirement that it needs to do some special processing that has nothing to do with DMA address translation - only the guest side platform code know that. BTW: I do not consider 'bounce buffering' as 'DMA address translation'. DMA address translation, translates CPU address to DMA address. Bounce buffering moves the data from one buffer at a given CPU address to another buffer at a different CPU address. Unfortunately the current DMA ops conflates the two. The need to do 'DMA address translation' is something the device can enforce. But the need to do bounce buffering, is something that the device should not be aware and should be entirely a decision made locally by the kernel/driver in the secure VM. RP > > -- > David Gibson | I'll have my music baroque, and my code > david AT gibson.dropbear.id.au| minimalist, thank you. NOT _the_ > _other_ > | _way_ _around_! > http://www.ozlabs.org/~dgibson -- Ram Pai ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted
On Mon, Aug 12, 2019 at 02:13:24PM +0200, Christoph Hellwig wrote: > On Sat, Aug 10, 2019 at 11:46:21PM -0700, Ram Pai wrote: > > If the hypervisor (hardware for hw virtio devices) does not mandate a > > DMA API, why is it illegal for the driver to request, special handling > > of its i/o buffers? Why are we associating this special handling to > > always mean, some DMA address translation? Can't there be > > any other kind of special handling needs, that has nothing to do with > > DMA address translation? > > I don't think it is illegal per se. It is however completely broken > if we do that decision on a system weide scale rather than properly > requesting it through a per-device flag in the normal virtio framework. if the decision has to be system-wide; for reasons known locally only to the kernel/driver, something that is independent of any device-flag, what would be the mechanism? RP ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted
On Sun, Aug 11, 2019 at 07:56:07AM +0200, Christoph Hellwig wrote: > sev_active() is gone now in linux-next, at least as a global API. > > And once again this is entirely going in the wrong direction. The only > way using the DMA API is going to work at all is if the device is ready > for it. So we need a flag on the virtio device, exposed by the > hypervisor (or hardware for hw virtio devices) that says: hey, I'm real, > don't take a shortcut. > > And that means on power and s390 qemu will always have to set thos if > you want to be ready for the ultravisor and co games. It's not like we > haven't been through this a few times before, have we? We have been through this so many times, but I dont think, we ever understood each other. I have a fundamental question, the answer to which was never clear. Here it is... If the hypervisor (hardware for hw virtio devices) does not mandate a DMA API, why is it illegal for the driver to request, special handling of its i/o buffers? Why are we associating this special handling to always mean, some DMA address translation? Can't there be any other kind of special handling needs, that has nothing to do with DMA address translation? -- Ram Pai ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted
On Sat, Aug 10, 2019 at 02:57:17PM -0400, Michael S. Tsirkin wrote: > On Tue, Jan 29, 2019 at 03:08:12PM -0200, Thiago Jung Bauermann wrote: > > > > Hello, > > > > With Christoph's rework of the DMA API that recently landed, the patch > > below is the only change needed in virtio to make it work in a POWER > > secure guest under the ultravisor. > > > > The other change we need (making sure the device's dma_map_ops is NULL > > so that the dma-direct/swiotlb code is used) can be made in > > powerpc-specific code. > > > > Of course, I also have patches (soon to be posted as RFC) which hook up > > to the powerpc secure guest support code. > > > > What do you think? > > > > >From d0629a36a75c678b4a72b853f8f7f8c17eedd6b3 Mon Sep 17 00:00:00 2001 > > From: Thiago Jung Bauermann > > Date: Thu, 24 Jan 2019 22:08:02 -0200 > > Subject: [RFC PATCH] virtio_ring: Use DMA API if guest memory is encrypted > > > > The host can't access the guest memory when it's encrypted, so using > > regular memory pages for the ring isn't an option. Go through the DMA API. > > > > Signed-off-by: Thiago Jung Bauermann > > --- > > drivers/virtio/virtio_ring.c | 5 - > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c > > index cd7e755484e3..321a27075380 100644 > > --- a/drivers/virtio/virtio_ring.c > > +++ b/drivers/virtio/virtio_ring.c > > @@ -259,8 +259,11 @@ static bool vring_use_dma_api(struct virtio_device > > *vdev) > > * not work without an even larger kludge. Instead, enable > > * the DMA API if we're a Xen guest, which at least allows > > * all of the sensible Xen configurations to work correctly. > > +* > > +* Also, if guest memory is encrypted the host can't access > > +* it directly. In this case, we'll need to use the DMA API. > > */ > > - if (xen_domain()) > > + if (xen_domain() || sev_active()) > > return true; > > > > return false; > > So I gave this lots of thought, and I'm coming round to > basically accepting something very similar to this patch. > > But not exactly like this :). > > Let's see what are the requirements. > > If > > 1. We do not trust the device (so we want to use a bounce buffer with it) > 2. DMA address is also a physical address of a buffer > > then we should use DMA API with virtio. > > > sev_active() above is one way to put (1). I can't say I love it but > it's tolerable. > > > But we also want promise from DMA API about 2. > > > Without promise 2 we simply can't use DMA API with a legacy device. > > > Otherwise, on a SEV system with an IOMMU which isn't 1:1 > and with a virtio device without ACCESS_PLATFORM, we are trying > to pass a virtual address, and devices without ACCESS_PLATFORM > can only access CPU physical addresses. > > So something like: > > dma_addr_is_phys_addr? On our Secure pseries platform, dma address is physical address and this proposal will help us, use DMA API. On our normal pseries platform, dma address is physical address too. But we do not necessarily need to use the DMA API. We can use the DMA API, but our handlers will do the same thing, the generic virtio handlers would do. If there is an opt-out option; even when dma addr is same as physical addr, than there will be less code duplication. Would something like this be better. (dma_addr_is_phys_addr && arch_want_to_use_dma_api()) ? RP > -- > MST -- Ram Pai ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
