Re: [PATCH] hw/dma/xlnx_dpdma: Read descriptor into buffer, not into pointer-to-buffer

[Edgar E. Iglesias]

On Fri, May 31, 2024 at 2:46 PM Peter Maydell 
wrote:

> In fdf029762f501 we factored out the handling of reading and writing
> DMA descriptors from guest memory.  Unfortunately we accidentally
> made the descriptor-read read the descriptor into the address of the
> buffer rather than into the buffer, because we didn't notice we
> needed to update the arguments to the dma_memory_read() call. Before
> the refactoring, "" is the address of a local struct DPDMADescriptor
> variable in xlnx_dpdma_start_operation(), which is the correct target
> for the guest-memory-read. But after the refactoring 'desc' is the
> "DPDMADescriptor *desc" argument to the new function, and so it is
> already an address.
>
> This bug is an overrun of a stack variable, since a pointer is at
> most 8 bytes long and we try to read 64 bytes, as well as being
> incorrect behaviour.
>
> Pass 'desc' rather than '' as the dma_memory_read() argument
> to fix this.
>
> (The same bug is not present in xlnx_dpdma_write_descriptor(),
> because there we are writing the descriptor from a local struct
> variable "DPDMADescriptor tmp_desc" and so passing _desc to
> dma_memory_write() is correct.)
>
> Spotted by Coverity: CID 1546649
>
>
+ CC Fred.

Reviewed-by: Edgar E. Iglesias 




> Fixes: fdf029762f50101 ("xlnx_dpdma: fix descriptor endianness bug")
> Signed-off-by: Peter Maydell 
> ---
>  hw/dma/xlnx_dpdma.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/hw/dma/xlnx_dpdma.c b/hw/dma/xlnx_dpdma.c
> index dde4aeca401..a685bd28bb8 100644
> --- a/hw/dma/xlnx_dpdma.c
> +++ b/hw/dma/xlnx_dpdma.c
> @@ -619,7 +619,7 @@ static MemTxResult
> xlnx_dpdma_read_descriptor(XlnxDPDMAState *s,
>DPDMADescriptor *desc)
>  {
>  MemTxResult res = dma_memory_read(_space_memory, desc_addr,
> -  , sizeof(DPDMADescriptor),
> +  desc, sizeof(DPDMADescriptor),
>MEMTXATTRS_UNSPECIFIED);
>  if (res) {
>  return res;
> --
> 2.34.1
>
>

[PATCH v8 5/8] softmmu: Replace check for RAMBlock offset 0 with xen_mr_is_memory

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

For xen, when checking for the first RAM (xen_memory), use
xen_mr_is_memory() rather than checking for a RAMBlock with
offset 0.

All Xen machines create xen_memory first so this has no
functional change for existing machines.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Reviewed-by: David Hildenbrand 
---
 system/physmem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/system/physmem.c b/system/physmem.c
index 5e6257ef65..b7847db1a2 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2229,7 +2229,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * because we don't want to map the entire memory in QEMU.
  * In that case just map the requested area.
  */
-if (block->offset == 0) {
+if (xen_mr_is_memory(block->mr)) {
 return xen_map_cache(block->mr, block->offset + addr,
  len, lock, lock,
  is_write);
-- 
2.40.1

[PATCH v8 4/8] softmmu: xen: Always pass offset + addr to xen_map_cache

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Always pass address with offset to xen_map_cache().
This is in preparation for support for grant mappings.

Since this is within a block that checks for offset == 0,
this has no functional changes.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Reviewed-by: David Hildenbrand 
---
 system/physmem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/system/physmem.c b/system/physmem.c
index 342b7a8fd4..5e6257ef65 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2230,7 +2230,8 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * In that case just map the requested area.
  */
 if (block->offset == 0) {
-return xen_map_cache(block->mr, addr, len, lock, lock,
+return xen_map_cache(block->mr, block->offset + addr,
+ len, lock, lock,
  is_write);
 }
 
-- 
2.40.1

[PATCH v8 0/8] xen: Support grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Hi,

Grant mappings are a mechanism in Xen for guests to grant each other
permissions to map and share pages. These grants can be temporary
so both map and unmaps must be respected. See here for more info:
https://github.com/xen-project/xen/blob/master/docs/misc/grant-tables.txt

Currently, the primary use-case for grants in QEMU, is with VirtIO backends.
Grant mappings will only work with models that use the address_space_map/unmap
interfaces, any other access will fail with appropriate error messages.

In response to feedback we got on v3, later version switch approach
from adding new MemoryRegion types and map/unmap hooks to instead reusing
the existing xen_map_cache() hooks (with extensions). Almost all of the
changes are now contained to the Xen modules.

This approach also refactors the mapcache to support multiple instances
(one for existing foreign mappings and another for grant mappings).

I've only enabled grants for the ARM PVH machine since that is what
I can currently test on.

Cheers,
Edgar

ChangeLog:

v7 -> v8:
* Remove xen_mr_is_* stubs.

v6 -> v7:
* Use g_autofree in xen_remap_bucket().
* Flatten nested if-statements in xen_map_cache().
* Fix typo in error message in xen_map_cache().

v5 -> v6:
* Correct passing of ram_addr_offset in xen_replace_cache_entry_unlocked.

v4 -> v5:
* Compute grant_ref from address_index to xen_remap_bucket().
* Rename grant_is_write to is_write.
* Remove unnecessary + mc->bucket_size - 1 in
  xen_invalidate_map_cache_entry_unlocked().
* Remove use of global mapcache in refactor of
  xen_replace_cache_entry_unlocked().
* Add error checking for xengnttab_unmap().
* Add assert in xen_replace_cache_entry_unlocked() against grant mappings.
* Fix memory leak when freeing first entry in mapcache buckets.
* Assert that bucket_shift is >= XC_PAGE_SHIFT when creating mapcache.
* Add missing use of xen_mr_is_memory() in hw/xen/xen-hvm-common.c.
* Rebase with master.

v3 -> v4:
* Reuse existing xen_map_cache hooks.
* Reuse existing map-cache for both foreign and grant mappings.
* Only enable grants for the ARM PVH machine (removed i386).

v2 -> v3:
* Drop patch 1/7. This was done because device unplug is an x86-only case.
* Add missing qemu_mutex_unlock() before return.

v1 -> v2:
* Split patch 2/7 to keep phymem.c changes in a separate.
* In patch "xen: add map and unmap callbacks for grant" add check for total
  allowed grant < XEN_MAX_VIRTIO_GRANTS.
* Fix formatting issues and re-based with master latest.

Edgar E. Iglesias (8):
  xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable
  xen: mapcache: Unmap first entries in buckets
  xen: Add xen_mr_is_memory()
  softmmu: xen: Always pass offset + addr to xen_map_cache
  softmmu: Replace check for RAMBlock offset 0 with xen_mr_is_memory
  xen: mapcache: Pass the ram_addr offset to xen_map_cache()
  xen: mapcache: Add support for grant mappings
  hw/arm: xen: Enable use of grant mappings

 hw/arm/xen_arm.c|   5 +
 hw/xen/xen-hvm-common.c |  18 ++-
 hw/xen/xen-mapcache.c   | 234 
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen-mapcache.h   |   2 +
 include/sysemu/xen.h|   2 +
 system/physmem.c|  12 +-
 7 files changed, 211 insertions(+), 65 deletions(-)


base-commit: 79d7475f39f1b0f05fcb159f5cdcbf162340dc7e
-- 
2.40.1

[PATCH v8 3/8] xen: Add xen_mr_is_memory()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add xen_mr_is_memory() to abstract away tests for the
xen_memory MR.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Acked-by: David Hildenbrand 
Reviewed-by: Philippe Mathieu-Daudé 
---
 hw/xen/xen-hvm-common.c | 10 --
 include/sysemu/xen.h|  1 +
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 2d1b032121..a0a0252da0 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -12,6 +12,12 @@
 
 MemoryRegion xen_memory;
 
+/* Check for xen memory.  */
+bool xen_mr_is_memory(MemoryRegion *mr)
+{
+return mr == _memory;
+}
+
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
Error **errp)
 {
@@ -28,7 +34,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, 
MemoryRegion *mr,
 return;
 }
 
-if (mr == _memory) {
+if (xen_mr_is_memory(mr)) {
 return;
 }
 
@@ -55,7 +61,7 @@ static void xen_set_memory(struct MemoryListener *listener,
 {
 XenIOState *state = container_of(listener, XenIOState, memory_listener);
 
-if (section->mr == _memory) {
+if (xen_mr_is_memory(section->mr)) {
 return;
 } else {
 if (add) {
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index 754ec2e6cb..3445888e39 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -49,4 +49,5 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, 
ram_addr_t size,
 
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
+bool xen_mr_is_memory(MemoryRegion *mr);
 #endif
-- 
2.40.1

[PATCH v8 6/8] xen: mapcache: Pass the ram_addr offset to xen_map_cache()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Pass the ram_addr offset to xen_map_cache.
This is in preparation for adding grant mappings that need
to compute the address within the RAMBlock.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: David Hildenbrand 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Philippe Mathieu-Daudé 
---
 hw/xen/xen-mapcache.c | 16 +++-
 include/sysemu/xen-mapcache.h |  2 ++
 system/physmem.c  |  9 +
 3 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index ec95445696..a07c47b0b1 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
  void *vaddr,
  hwaddr size,
  hwaddr address_index,
- bool dummy)
+ bool dummy,
+ ram_addr_t ram_offset)
 {
 uint8_t *vaddr_base;
 xen_pfn_t *pfns;
@@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_offset,
uint8_t lock, bool dma, bool is_write)
 {
 MapCacheEntry *entry, *pentry = NULL,
@@ -337,14 +339,16 @@ tryagain:
 if (!entry) {
 entry = g_new0(MapCacheEntry, 1);
 pentry->next = entry;
-xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+ ram_offset);
 } else if (!entry->lock) {
 if (!entry->vaddr_base || entry->paddr_index != address_index ||
 entry->size != cache_size ||
 !test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
-xen_remap_bucket(mc, entry, NULL, cache_size, address_index, 
dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+ ram_offset);
 }
 }
 
@@ -391,13 +395,15 @@ tryagain:
 
 uint8_t *xen_map_cache(MemoryRegion *mr,
hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_addr_offset,
uint8_t lock, bool dma,
bool is_write)
 {
 uint8_t *p;
 
 mapcache_lock(mapcache);
-p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
+p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
+   lock, dma, is_write);
 mapcache_unlock(mapcache);
 return p;
 }
@@ -632,7 +638,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache 
*mc,
 trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
 xen_remap_bucket(mc, entry, entry->vaddr_base,
- cache_size, address_index, false);
+ cache_size, address_index, false, old_phys_addr);
 if (!test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index 1ec9e66752..b5e3ea1bc0 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
 void xen_map_cache_init(phys_offset_to_gaddr_t f,
 void *opaque);
 uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_addr_offset,
uint8_t lock, bool dma,
bool is_write);
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
@@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t 
f,
 static inline uint8_t *xen_map_cache(MemoryRegion *mr,
  hwaddr phys_addr,
  hwaddr size,
+ ram_addr_t ram_addr_offset,
  uint8_t lock,
  bool dma,
  bool is_write)
diff --git a/system/physmem.c b/system/physmem.c
index b7847db1a2..33d09f7571 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2231,13 +2231,14 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  */
 if (xen_mr_is_memory(block->mr)) {
 return xen_map_cache(block->mr, block->offset + addr,
- len, lock, lock,
- is_write);
+  

[PATCH v8 2/8] xen: mapcache: Unmap first entries in buckets

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

When invalidating memory ranges, if we happen to hit the first
entry in a bucket we were never unmapping it. This was harmless
for foreign mappings but now that we're looking to reuse the
mapcache for transient grant mappings, we must unmap entries
when invalidated.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-mapcache.c | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index bc860f4373..ec95445696 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -491,18 +491,23 @@ static void 
xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
 return;
 }
 entry->lock--;
-if (entry->lock > 0 || pentry == NULL) {
+if (entry->lock > 0) {
 return;
 }
 
-pentry->next = entry->next;
 ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
 if (munmap(entry->vaddr_base, entry->size) != 0) {
 perror("unmap fails");
 exit(-1);
 }
+
 g_free(entry->valid_mapping);
-g_free(entry);
+if (pentry) {
+pentry->next = entry->next;
+g_free(entry);
+} else {
+memset(entry, 0, sizeof *entry);
+}
 }
 
 typedef struct XenMapCacheData {
-- 
2.40.1

[PATCH v8 1/8] xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Make MCACHE_BUCKET_SHIFT runtime configurable per cache instance.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-mapcache.c | 54 ++-
 1 file changed, 33 insertions(+), 21 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index fa6813b1ad..bc860f4373 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -23,13 +23,10 @@
 
 
 #if HOST_LONG_BITS == 32
-#  define MCACHE_BUCKET_SHIFT 16
 #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
 #else
-#  define MCACHE_BUCKET_SHIFT 20
 #  define MCACHE_MAX_SIZE (1UL<<35) /* 32GB Cap */
 #endif
-#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
 
 /* This is the size of the virtual address space reserve to QEMU that will not
  * be use by MapCache.
@@ -65,7 +62,8 @@ typedef struct MapCache {
 /* For most cases (>99.9%), the page address is the same. */
 MapCacheEntry *last_entry;
 unsigned long max_mcache_size;
-unsigned int mcache_bucket_shift;
+unsigned int bucket_shift;
+unsigned long bucket_size;
 
 phys_offset_to_gaddr_t phys_offset_to_gaddr;
 QemuMutex lock;
@@ -95,11 +93,14 @@ static inline int test_bits(int nr, int size, const 
unsigned long *addr)
 
 static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
void *opaque,
+   unsigned int bucket_shift,
unsigned long max_size)
 {
 unsigned long size;
 MapCache *mc;
 
+assert(bucket_shift >= XC_PAGE_SHIFT);
+
 mc = g_new0(MapCache, 1);
 
 mc->phys_offset_to_gaddr = f;
@@ -108,12 +109,14 @@ static MapCache 
*xen_map_cache_init_single(phys_offset_to_gaddr_t f,
 
 QTAILQ_INIT(>locked_entries);
 
+mc->bucket_shift = bucket_shift;
+mc->bucket_size = 1UL << bucket_shift;
 mc->max_mcache_size = max_size;
 
 mc->nr_buckets =
 (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
-  (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
- (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
+  (1UL << (bucket_shift - XC_PAGE_SHIFT)) - 1) >>
+ (bucket_shift - XC_PAGE_SHIFT));
 
 size = mc->nr_buckets * sizeof(MapCacheEntry);
 size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
@@ -126,6 +129,13 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 {
 struct rlimit rlimit_as;
 unsigned long max_mcache_size;
+unsigned int bucket_shift;
+
+if (HOST_LONG_BITS == 32) {
+bucket_shift = 16;
+} else {
+bucket_shift = 20;
+}
 
 if (geteuid() == 0) {
 rlimit_as.rlim_cur = RLIM_INFINITY;
@@ -146,7 +156,9 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 }
 }
 
-mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
+mapcache = xen_map_cache_init_single(f, opaque,
+ bucket_shift,
+ max_mcache_size);
 setrlimit(RLIMIT_AS, _as);
 }
 
@@ -195,7 +207,7 @@ static void xen_remap_bucket(MapCache *mc,
 entry->valid_mapping = NULL;
 
 for (i = 0; i < nb_pfn; i++) {
-pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
+pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
 }
 
 /*
@@ -266,8 +278,8 @@ static uint8_t *xen_map_cache_unlocked(MapCache *mc,
 bool dummy = false;
 
 tryagain:
-address_index  = phys_addr >> MCACHE_BUCKET_SHIFT;
-address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
+address_index  = phys_addr >> mc->bucket_shift;
+address_offset = phys_addr & (mc->bucket_size - 1);
 
 trace_xen_map_cache(phys_addr);
 
@@ -294,14 +306,14 @@ tryagain:
 return mc->last_entry->vaddr_base + address_offset;
 }
 
-/* size is always a multiple of MCACHE_BUCKET_SIZE */
+/* size is always a multiple of mc->bucket_size */
 if (size) {
 cache_size = size + address_offset;
-if (cache_size % MCACHE_BUCKET_SIZE) {
-cache_size += MCACHE_BUCKET_SIZE - (cache_size % 
MCACHE_BUCKET_SIZE);
+if (cache_size % mc->bucket_size) {
+cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
 }
 } else {
-cache_size = MCACHE_BUCKET_SIZE;
+cache_size = mc->bucket_size;
 }
 
 entry = >entry[address_index % mc->nr_buckets];
@@ -422,7 +434,7 @@ static ram_addr_t 
xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
 raddr = RAM_ADDR_INVALID;
 } else {
-raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
+   

[PATCH v8 8/8] hw/arm: xen: Enable use of grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Manos Pitsidianakis 
Reviewed-by: Philippe Mathieu-Daudé 
---
 hw/arm/xen_arm.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/hw/arm/xen_arm.c b/hw/arm/xen_arm.c
index 15fa7dfa84..6fad829ede 100644
--- a/hw/arm/xen_arm.c
+++ b/hw/arm/xen_arm.c
@@ -125,6 +125,11 @@ static void xen_init_ram(MachineState *machine)
  GUEST_RAM1_BASE, ram_size[1]);
 memory_region_add_subregion(sysmem, GUEST_RAM1_BASE, _hi);
 }
+
+/* Setup support for grants.  */
+memory_region_init_ram(_grants, NULL, "xen.grants", block_len,
+   _fatal);
+memory_region_add_subregion(sysmem, XEN_GRANT_ADDR_OFF, _grants);
 }
 
 void arch_handle_ioreq(XenIOState *state, ioreq_t *req)
-- 
2.40.1

[PATCH v8 7/8] xen: mapcache: Add support for grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add a second mapcache for grant mappings. The mapcache for
grants needs to work with XC_PAGE_SIZE granularity since
we can't map larger ranges than what has been granted to us.

Like with foreign mappings (xen_memory), machines using grants
are expected to initialize the xen_grants MR and map it
into their address-map accordingly.

CC: Manos Pitsidianakis 
Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-hvm-common.c |  12 ++-
 hw/xen/xen-mapcache.c   | 165 +---
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen.h|   1 +
 4 files changed, 144 insertions(+), 37 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index a0a0252da0..b8ace1c368 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -10,12 +10,18 @@
 #include "hw/boards.h"
 #include "hw/xen/arch_hvm.h"
 
-MemoryRegion xen_memory;
+MemoryRegion xen_memory, xen_grants;
 
-/* Check for xen memory.  */
+/* Check for any kind of xen memory, foreign mappings or grants.  */
 bool xen_mr_is_memory(MemoryRegion *mr)
 {
-return mr == _memory;
+return mr == _memory || mr == _grants;
+}
+
+/* Check specifically for grants.  */
+bool xen_mr_is_grants(MemoryRegion *mr)
+{
+return mr == _grants;
 }
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index a07c47b0b1..5f23b0adbe 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -14,6 +14,7 @@
 
 #include 
 
+#include "hw/xen/xen-hvm-common.h"
 #include "hw/xen/xen_native.h"
 #include "qemu/bitmap.h"
 
@@ -21,6 +22,8 @@
 #include "sysemu/xen-mapcache.h"
 #include "trace.h"
 
+#include 
+#include 
 
 #if HOST_LONG_BITS == 32
 #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
 unsigned long *valid_mapping;
 uint32_t lock;
 #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
 uint8_t flags;
 hwaddr size;
 struct MapCacheEntry *next;
@@ -71,6 +75,8 @@ typedef struct MapCache {
 } MapCache;
 
 static MapCache *mapcache;
+static MapCache *mapcache_grants;
+static xengnttab_handle *xen_region_gnttabdev;
 
 static inline void mapcache_lock(MapCache *mc)
 {
@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 unsigned long max_mcache_size;
 unsigned int bucket_shift;
 
+xen_region_gnttabdev = xengnttab_open(NULL, 0);
+if (xen_region_gnttabdev == NULL) {
+error_report("mapcache: Failed to open gnttab device");
+exit(EXIT_FAILURE);
+}
+
 if (HOST_LONG_BITS == 32) {
 bucket_shift = 16;
 } else {
@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 mapcache = xen_map_cache_init_single(f, opaque,
  bucket_shift,
  max_mcache_size);
+
+/*
+ * Grant mappings must use XC_PAGE_SIZE granularity since we can't
+ * map anything beyond the number of pages granted to us.
+ */
+mapcache_grants = xen_map_cache_init_single(f, opaque,
+XC_PAGE_SHIFT,
+max_mcache_size);
+
 setrlimit(RLIMIT_AS, _as);
 }
 
@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
  hwaddr size,
  hwaddr address_index,
  bool dummy,
+ bool grant,
+ bool is_write,
  ram_addr_t ram_offset)
 {
 uint8_t *vaddr_base;
-xen_pfn_t *pfns;
-int *err;
+g_autofree uint32_t *refs = NULL;
+g_autofree xen_pfn_t *pfns = NULL;
+g_autofree int *err;
 unsigned int i;
 hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
 
 trace_xen_remap_bucket(address_index);
 
-pfns = g_new0(xen_pfn_t, nb_pfn);
+if (grant) {
+refs = g_new0(uint32_t, nb_pfn);
+} else {
+pfns = g_new0(xen_pfn_t, nb_pfn);
+}
 err = g_new0(int, nb_pfn);
 
 if (entry->vaddr_base != NULL) {
@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
 g_free(entry->valid_mapping);
 entry->valid_mapping = NULL;
 
-for (i = 0; i < nb_pfn; i++) {
-pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+if (grant) {
+hwaddr grant_base = address_index - (ram_offset >> XC_PAGE_SHIFT);
+
+for (i = 0; i < nb_pfn; i++) {
+refs[i] = grant_base + i;
+}
+} else {
+for (i = 0; i < nb_pfn; i++) {
+pfns[i] = (address_index << (mc->bucket_shift 

Re: [PATCH v7 3/8] xen: Add xen_mr_is_memory()

[Edgar E. Iglesias]

On Mon, May 27, 2024 at 6:25 PM Philippe Mathieu-Daudé 
wrote:

> Hi Edgar,
>
> On 24/5/24 12:51, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" 
> >
> > Add xen_mr_is_memory() to abstract away tests for the
> > xen_memory MR.
> >
> > No functional changes.
> >
> > Signed-off-by: Edgar E. Iglesias 
> > Reviewed-by: Stefano Stabellini 
> > Acked-by: David Hildenbrand 
> > ---
> >   hw/xen/xen-hvm-common.c | 10 --
> >   include/sysemu/xen.h|  8 
> >   2 files changed, 16 insertions(+), 2 deletions(-)
>
> To consolidate we could add:
>
>static MemoryRegion xen_memory;
>
>MemoryRegion *xen_mr_memory_init(uint64_t block_len)
>{
>   assert(!xen_memory.size);
>   memory_region_init_ram(_memory, NULL, "xen.ram", block_len,
> _fatal);
>   return _memory;
>}
>
> and remove the extern declaration.
>

Thanks,

We have a future patch series in the workings that will add a PVH machine
for x86, I'll keep this in mind for then!


>
> > diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
> > index 754ec2e6cb..dc72f83bcb 100644
> > --- a/include/sysemu/xen.h
> > +++ b/include/sysemu/xen.h
> > @@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start,
> ram_addr_t length);
> >   void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
> >  struct MemoryRegion *mr, Error **errp);
> >
> > +bool xen_mr_is_memory(MemoryRegion *mr);
> > +
> >   #else /* !CONFIG_XEN_IS_POSSIBLE */
> >
> >   #define xen_enabled() 0
> > @@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr,
> ram_addr_t size,
> >   g_assert_not_reached();
> >   }
> >
> > +static inline bool xen_mr_is_memory(MemoryRegion *mr)
> > +{
> > +g_assert_not_reached();
> > +return false;
>
> No need for the stub, just always declare xen_mr_is_memory() ...
> > +}
> > +
> >   #endif /* CONFIG_XEN_IS_POSSIBLE */
>
> ... here.
>
> >   #endif
>
> Removing the stub:
> Reviewed-by: Philippe Mathieu-Daudé 
>
>
Thanks Philippe, will remove the stubs in v8.

Cheers,
Edgar

[PATCH v7 2/8] xen: mapcache: Unmap first entries in buckets

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

When invalidating memory ranges, if we happen to hit the first
entry in a bucket we were never unmapping it. This was harmless
for foreign mappings but now that we're looking to reuse the
mapcache for transient grant mappings, we must unmap entries
when invalidated.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-mapcache.c | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index bc860f4373..ec95445696 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -491,18 +491,23 @@ static void 
xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
 return;
 }
 entry->lock--;
-if (entry->lock > 0 || pentry == NULL) {
+if (entry->lock > 0) {
 return;
 }
 
-pentry->next = entry->next;
 ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
 if (munmap(entry->vaddr_base, entry->size) != 0) {
 perror("unmap fails");
 exit(-1);
 }
+
 g_free(entry->valid_mapping);
-g_free(entry);
+if (pentry) {
+pentry->next = entry->next;
+g_free(entry);
+} else {
+memset(entry, 0, sizeof *entry);
+}
 }
 
 typedef struct XenMapCacheData {
-- 
2.40.1

[PATCH v7 7/8] xen: mapcache: Add support for grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add a second mapcache for grant mappings. The mapcache for
grants needs to work with XC_PAGE_SIZE granularity since
we can't map larger ranges than what has been granted to us.

Like with foreign mappings (xen_memory), machines using grants
are expected to initialize the xen_grants MR and map it
into their address-map accordingly.

CC: Manos Pitsidianakis 
Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-hvm-common.c |  12 ++-
 hw/xen/xen-mapcache.c   | 165 +---
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen.h|   7 ++
 4 files changed, 150 insertions(+), 37 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index a0a0252da0..b8ace1c368 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -10,12 +10,18 @@
 #include "hw/boards.h"
 #include "hw/xen/arch_hvm.h"
 
-MemoryRegion xen_memory;
+MemoryRegion xen_memory, xen_grants;
 
-/* Check for xen memory.  */
+/* Check for any kind of xen memory, foreign mappings or grants.  */
 bool xen_mr_is_memory(MemoryRegion *mr)
 {
-return mr == _memory;
+return mr == _memory || mr == _grants;
+}
+
+/* Check specifically for grants.  */
+bool xen_mr_is_grants(MemoryRegion *mr)
+{
+return mr == _grants;
 }
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index a07c47b0b1..5f23b0adbe 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -14,6 +14,7 @@
 
 #include 
 
+#include "hw/xen/xen-hvm-common.h"
 #include "hw/xen/xen_native.h"
 #include "qemu/bitmap.h"
 
@@ -21,6 +22,8 @@
 #include "sysemu/xen-mapcache.h"
 #include "trace.h"
 
+#include 
+#include 
 
 #if HOST_LONG_BITS == 32
 #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
 unsigned long *valid_mapping;
 uint32_t lock;
 #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
 uint8_t flags;
 hwaddr size;
 struct MapCacheEntry *next;
@@ -71,6 +75,8 @@ typedef struct MapCache {
 } MapCache;
 
 static MapCache *mapcache;
+static MapCache *mapcache_grants;
+static xengnttab_handle *xen_region_gnttabdev;
 
 static inline void mapcache_lock(MapCache *mc)
 {
@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 unsigned long max_mcache_size;
 unsigned int bucket_shift;
 
+xen_region_gnttabdev = xengnttab_open(NULL, 0);
+if (xen_region_gnttabdev == NULL) {
+error_report("mapcache: Failed to open gnttab device");
+exit(EXIT_FAILURE);
+}
+
 if (HOST_LONG_BITS == 32) {
 bucket_shift = 16;
 } else {
@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 mapcache = xen_map_cache_init_single(f, opaque,
  bucket_shift,
  max_mcache_size);
+
+/*
+ * Grant mappings must use XC_PAGE_SIZE granularity since we can't
+ * map anything beyond the number of pages granted to us.
+ */
+mapcache_grants = xen_map_cache_init_single(f, opaque,
+XC_PAGE_SHIFT,
+max_mcache_size);
+
 setrlimit(RLIMIT_AS, _as);
 }
 
@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
  hwaddr size,
  hwaddr address_index,
  bool dummy,
+ bool grant,
+ bool is_write,
  ram_addr_t ram_offset)
 {
 uint8_t *vaddr_base;
-xen_pfn_t *pfns;
-int *err;
+g_autofree uint32_t *refs = NULL;
+g_autofree xen_pfn_t *pfns = NULL;
+g_autofree int *err;
 unsigned int i;
 hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
 
 trace_xen_remap_bucket(address_index);
 
-pfns = g_new0(xen_pfn_t, nb_pfn);
+if (grant) {
+refs = g_new0(uint32_t, nb_pfn);
+} else {
+pfns = g_new0(xen_pfn_t, nb_pfn);
+}
 err = g_new0(int, nb_pfn);
 
 if (entry->vaddr_base != NULL) {
@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
 g_free(entry->valid_mapping);
 entry->valid_mapping = NULL;
 
-for (i = 0; i < nb_pfn; i++) {
-pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+if (grant) {
+hwaddr grant_base = address_index - (ram_offset >> XC_PAGE_SHIFT);
+
+for (i = 0; i < nb_pfn; i++) {
+refs[i] = grant_base + i;
+}
+} else {
+for (i = 0; i < nb_pfn; i++) {
+pfns[i] = (address_index << (mc->bucket_shift 

[PATCH v7 6/8] xen: mapcache: Pass the ram_addr offset to xen_map_cache()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Pass the ram_addr offset to xen_map_cache.
This is in preparation for adding grant mappings that need
to compute the address within the RAMBlock.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: David Hildenbrand 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-mapcache.c | 16 +++-
 include/sysemu/xen-mapcache.h |  2 ++
 system/physmem.c  |  9 +
 3 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index ec95445696..a07c47b0b1 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
  void *vaddr,
  hwaddr size,
  hwaddr address_index,
- bool dummy)
+ bool dummy,
+ ram_addr_t ram_offset)
 {
 uint8_t *vaddr_base;
 xen_pfn_t *pfns;
@@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_offset,
uint8_t lock, bool dma, bool is_write)
 {
 MapCacheEntry *entry, *pentry = NULL,
@@ -337,14 +339,16 @@ tryagain:
 if (!entry) {
 entry = g_new0(MapCacheEntry, 1);
 pentry->next = entry;
-xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+ ram_offset);
 } else if (!entry->lock) {
 if (!entry->vaddr_base || entry->paddr_index != address_index ||
 entry->size != cache_size ||
 !test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
-xen_remap_bucket(mc, entry, NULL, cache_size, address_index, 
dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+ ram_offset);
 }
 }
 
@@ -391,13 +395,15 @@ tryagain:
 
 uint8_t *xen_map_cache(MemoryRegion *mr,
hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_addr_offset,
uint8_t lock, bool dma,
bool is_write)
 {
 uint8_t *p;
 
 mapcache_lock(mapcache);
-p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
+p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
+   lock, dma, is_write);
 mapcache_unlock(mapcache);
 return p;
 }
@@ -632,7 +638,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache 
*mc,
 trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
 xen_remap_bucket(mc, entry, entry->vaddr_base,
- cache_size, address_index, false);
+ cache_size, address_index, false, old_phys_addr);
 if (!test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index 1ec9e66752..b5e3ea1bc0 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
 void xen_map_cache_init(phys_offset_to_gaddr_t f,
 void *opaque);
 uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_addr_offset,
uint8_t lock, bool dma,
bool is_write);
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
@@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t 
f,
 static inline uint8_t *xen_map_cache(MemoryRegion *mr,
  hwaddr phys_addr,
  hwaddr size,
+ ram_addr_t ram_addr_offset,
  uint8_t lock,
  bool dma,
  bool is_write)
diff --git a/system/physmem.c b/system/physmem.c
index b7847db1a2..33d09f7571 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2231,13 +2231,14 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  */
 if (xen_mr_is_memory(block->mr)) {
 return xen_map_cache(block->mr, block->offset + addr,
- len, lock, lock,
- is_write);
+ len, block->offset,
+ 

[PATCH v7 8/8] hw/arm: xen: Enable use of grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Manos Pitsidianakis 
---
 hw/arm/xen_arm.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/hw/arm/xen_arm.c b/hw/arm/xen_arm.c
index 15fa7dfa84..6fad829ede 100644
--- a/hw/arm/xen_arm.c
+++ b/hw/arm/xen_arm.c
@@ -125,6 +125,11 @@ static void xen_init_ram(MachineState *machine)
  GUEST_RAM1_BASE, ram_size[1]);
 memory_region_add_subregion(sysmem, GUEST_RAM1_BASE, _hi);
 }
+
+/* Setup support for grants.  */
+memory_region_init_ram(_grants, NULL, "xen.grants", block_len,
+   _fatal);
+memory_region_add_subregion(sysmem, XEN_GRANT_ADDR_OFF, _grants);
 }
 
 void arch_handle_ioreq(XenIOState *state, ioreq_t *req)
-- 
2.40.1

[PATCH v7 0/8] xen: Support grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Hi,

Grant mappings are a mechanism in Xen for guests to grant each other
permissions to map and share pages. These grants can be temporary
so both map and unmaps must be respected. See here for more info:
https://github.com/xen-project/xen/blob/master/docs/misc/grant-tables.txt

Currently, the primary use-case for grants in QEMU, is with VirtIO backends.
Grant mappings will only work with models that use the address_space_map/unmap
interfaces, any other access will fail with appropriate error messages.

In response to feedback we got on v3, later version switch approach
from adding new MemoryRegion types and map/unmap hooks to instead reusing
the existing xen_map_cache() hooks (with extensions). Almost all of the
changes are now contained to the Xen modules.

This approach also refactors the mapcache to support multiple instances
(one for existing foreign mappings and another for grant mappings).

I've only enabled grants for the ARM PVH machine since that is what
I can currently test on.

Cheers,
Edgar

ChangeLog:

v6 -> v7:
* Use g_autofree in xen_remap_bucket().
* Flatten nested if-statements in xen_map_cache().
* Fix typo in error message in xen_map_cache().

v5 -> v6:
* Correct passing of ram_addr_offset in xen_replace_cache_entry_unlocked.

v4 -> v5:
* Compute grant_ref from address_index to xen_remap_bucket().
* Rename grant_is_write to is_write.
* Remove unnecessary + mc->bucket_size - 1 in
  xen_invalidate_map_cache_entry_unlocked().
* Remove use of global mapcache in refactor of
  xen_replace_cache_entry_unlocked().
* Add error checking for xengnttab_unmap().
* Add assert in xen_replace_cache_entry_unlocked() against grant mappings.
* Fix memory leak when freeing first entry in mapcache buckets.
* Assert that bucket_shift is >= XC_PAGE_SHIFT when creating mapcache.
* Add missing use of xen_mr_is_memory() in hw/xen/xen-hvm-common.c.
* Rebase with master.

v3 -> v4:
* Reuse existing xen_map_cache hooks.
* Reuse existing map-cache for both foreign and grant mappings.
* Only enable grants for the ARM PVH machine (removed i386).

v2 -> v3:
* Drop patch 1/7. This was done because device unplug is an x86-only case.
* Add missing qemu_mutex_unlock() before return.

v1 -> v2:
* Split patch 2/7 to keep phymem.c changes in a separate.
* In patch "xen: add map and unmap callbacks for grant" add check for total
  allowed grant < XEN_MAX_VIRTIO_GRANTS.
* Fix formatting issues and re-based with master latest.


Edgar E. Iglesias (8):
  xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable
  xen: mapcache: Unmap first entries in buckets
  xen: Add xen_mr_is_memory()
  softmmu: xen: Always pass offset + addr to xen_map_cache
  softmmu: Replace check for RAMBlock offset 0 with xen_mr_is_memory
  xen: mapcache: Pass the ram_addr offset to xen_map_cache()
  xen: mapcache: Add support for grant mappings
  hw/arm: xen: Enable use of grant mappings

 hw/arm/xen_arm.c|   5 +
 hw/xen/xen-hvm-common.c |  18 ++-
 hw/xen/xen-mapcache.c   | 234 
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen-mapcache.h   |   2 +
 include/sysemu/xen.h|  15 ++
 system/physmem.c|  12 +-
 7 files changed, 224 insertions(+), 65 deletions(-)


base-commit: 70581940cabcc51b329652becddfbc6a261b1b83
-- 
2.40.1

[PATCH v7 3/8] xen: Add xen_mr_is_memory()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add xen_mr_is_memory() to abstract away tests for the
xen_memory MR.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Acked-by: David Hildenbrand 
---
 hw/xen/xen-hvm-common.c | 10 --
 include/sysemu/xen.h|  8 
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 2d1b032121..a0a0252da0 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -12,6 +12,12 @@
 
 MemoryRegion xen_memory;
 
+/* Check for xen memory.  */
+bool xen_mr_is_memory(MemoryRegion *mr)
+{
+return mr == _memory;
+}
+
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
Error **errp)
 {
@@ -28,7 +34,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, 
MemoryRegion *mr,
 return;
 }
 
-if (mr == _memory) {
+if (xen_mr_is_memory(mr)) {
 return;
 }
 
@@ -55,7 +61,7 @@ static void xen_set_memory(struct MemoryListener *listener,
 {
 XenIOState *state = container_of(listener, XenIOState, memory_listener);
 
-if (section->mr == _memory) {
+if (xen_mr_is_memory(section->mr)) {
 return;
 } else {
 if (add) {
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index 754ec2e6cb..dc72f83bcb 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t 
length);
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
struct MemoryRegion *mr, Error **errp);
 
+bool xen_mr_is_memory(MemoryRegion *mr);
+
 #else /* !CONFIG_XEN_IS_POSSIBLE */
 
 #define xen_enabled() 0
@@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, 
ram_addr_t size,
 g_assert_not_reached();
 }
 
+static inline bool xen_mr_is_memory(MemoryRegion *mr)
+{
+g_assert_not_reached();
+return false;
+}
+
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 #endif
-- 
2.40.1

[PATCH v7 4/8] softmmu: xen: Always pass offset + addr to xen_map_cache

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Always pass address with offset to xen_map_cache().
This is in preparation for support for grant mappings.

Since this is within a block that checks for offset == 0,
this has no functional changes.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Reviewed-by: David Hildenbrand 
---
 system/physmem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/system/physmem.c b/system/physmem.c
index 342b7a8fd4..5e6257ef65 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2230,7 +2230,8 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * In that case just map the requested area.
  */
 if (block->offset == 0) {
-return xen_map_cache(block->mr, addr, len, lock, lock,
+return xen_map_cache(block->mr, block->offset + addr,
+ len, lock, lock,
  is_write);
 }
 
-- 
2.40.1

[PATCH v7 1/8] xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Make MCACHE_BUCKET_SHIFT runtime configurable per cache instance.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-mapcache.c | 54 ++-
 1 file changed, 33 insertions(+), 21 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index fa6813b1ad..bc860f4373 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -23,13 +23,10 @@
 
 
 #if HOST_LONG_BITS == 32
-#  define MCACHE_BUCKET_SHIFT 16
 #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
 #else
-#  define MCACHE_BUCKET_SHIFT 20
 #  define MCACHE_MAX_SIZE (1UL<<35) /* 32GB Cap */
 #endif
-#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
 
 /* This is the size of the virtual address space reserve to QEMU that will not
  * be use by MapCache.
@@ -65,7 +62,8 @@ typedef struct MapCache {
 /* For most cases (>99.9%), the page address is the same. */
 MapCacheEntry *last_entry;
 unsigned long max_mcache_size;
-unsigned int mcache_bucket_shift;
+unsigned int bucket_shift;
+unsigned long bucket_size;
 
 phys_offset_to_gaddr_t phys_offset_to_gaddr;
 QemuMutex lock;
@@ -95,11 +93,14 @@ static inline int test_bits(int nr, int size, const 
unsigned long *addr)
 
 static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
void *opaque,
+   unsigned int bucket_shift,
unsigned long max_size)
 {
 unsigned long size;
 MapCache *mc;
 
+assert(bucket_shift >= XC_PAGE_SHIFT);
+
 mc = g_new0(MapCache, 1);
 
 mc->phys_offset_to_gaddr = f;
@@ -108,12 +109,14 @@ static MapCache 
*xen_map_cache_init_single(phys_offset_to_gaddr_t f,
 
 QTAILQ_INIT(>locked_entries);
 
+mc->bucket_shift = bucket_shift;
+mc->bucket_size = 1UL << bucket_shift;
 mc->max_mcache_size = max_size;
 
 mc->nr_buckets =
 (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
-  (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
- (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
+  (1UL << (bucket_shift - XC_PAGE_SHIFT)) - 1) >>
+ (bucket_shift - XC_PAGE_SHIFT));
 
 size = mc->nr_buckets * sizeof(MapCacheEntry);
 size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
@@ -126,6 +129,13 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 {
 struct rlimit rlimit_as;
 unsigned long max_mcache_size;
+unsigned int bucket_shift;
+
+if (HOST_LONG_BITS == 32) {
+bucket_shift = 16;
+} else {
+bucket_shift = 20;
+}
 
 if (geteuid() == 0) {
 rlimit_as.rlim_cur = RLIM_INFINITY;
@@ -146,7 +156,9 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 }
 }
 
-mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
+mapcache = xen_map_cache_init_single(f, opaque,
+ bucket_shift,
+ max_mcache_size);
 setrlimit(RLIMIT_AS, _as);
 }
 
@@ -195,7 +207,7 @@ static void xen_remap_bucket(MapCache *mc,
 entry->valid_mapping = NULL;
 
 for (i = 0; i < nb_pfn; i++) {
-pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
+pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
 }
 
 /*
@@ -266,8 +278,8 @@ static uint8_t *xen_map_cache_unlocked(MapCache *mc,
 bool dummy = false;
 
 tryagain:
-address_index  = phys_addr >> MCACHE_BUCKET_SHIFT;
-address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
+address_index  = phys_addr >> mc->bucket_shift;
+address_offset = phys_addr & (mc->bucket_size - 1);
 
 trace_xen_map_cache(phys_addr);
 
@@ -294,14 +306,14 @@ tryagain:
 return mc->last_entry->vaddr_base + address_offset;
 }
 
-/* size is always a multiple of MCACHE_BUCKET_SIZE */
+/* size is always a multiple of mc->bucket_size */
 if (size) {
 cache_size = size + address_offset;
-if (cache_size % MCACHE_BUCKET_SIZE) {
-cache_size += MCACHE_BUCKET_SIZE - (cache_size % 
MCACHE_BUCKET_SIZE);
+if (cache_size % mc->bucket_size) {
+cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
 }
 } else {
-cache_size = MCACHE_BUCKET_SIZE;
+cache_size = mc->bucket_size;
 }
 
 entry = >entry[address_index % mc->nr_buckets];
@@ -422,7 +434,7 @@ static ram_addr_t 
xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
 raddr = RAM_ADDR_INVALID;
 } else {
-raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
+   

[PATCH v7 5/8] softmmu: Replace check for RAMBlock offset 0 with xen_mr_is_memory

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

For xen, when checking for the first RAM (xen_memory), use
xen_mr_is_memory() rather than checking for a RAMBlock with
offset 0.

All Xen machines create xen_memory first so this has no
functional change for existing machines.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Reviewed-by: David Hildenbrand 
---
 system/physmem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/system/physmem.c b/system/physmem.c
index 5e6257ef65..b7847db1a2 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2229,7 +2229,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * because we don't want to map the entire memory in QEMU.
  * In that case just map the requested area.
  */
-if (block->offset == 0) {
+if (xen_mr_is_memory(block->mr)) {
 return xen_map_cache(block->mr, block->offset + addr,
  len, lock, lock,
  is_write);
-- 
2.40.1

Re: [PATCH v6 7/8] xen: mapcache: Add support for grant mappings

[Edgar E. Iglesias]

On Thu, May 23, 2024 at 9:47 AM Manos Pitsidianakis <
manos.pitsidiana...@linaro.org> wrote:

> On Thu, 16 May 2024 18:48, "Edgar E. Iglesias" 
> wrote:
> >From: "Edgar E. Iglesias" 
> >
> >Add a second mapcache for grant mappings. The mapcache for
> >grants needs to work with XC_PAGE_SIZE granularity since
> >we can't map larger ranges than what has been granted to us.
> >
> >Like with foreign mappings (xen_memory), machines using grants
> >are expected to initialize the xen_grants MR and map it
> >into their address-map accordingly.
> >
> >Signed-off-by: Edgar E. Iglesias 
> >Reviewed-by: Stefano Stabellini 
> >---
> > hw/xen/xen-hvm-common.c |  12 ++-
> > hw/xen/xen-mapcache.c   | 163 ++--
> > include/hw/xen/xen-hvm-common.h |   3 +
> > include/sysemu/xen.h|   7 ++
> > 4 files changed, 152 insertions(+), 33 deletions(-)
> >
> >diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
> >index a0a0252da0..b8ace1c368 100644
> >--- a/hw/xen/xen-hvm-common.c
> >+++ b/hw/xen/xen-hvm-common.c
> >@@ -10,12 +10,18 @@
> > #include "hw/boards.h"
> > #include "hw/xen/arch_hvm.h"
> >
> >-MemoryRegion xen_memory;
> >+MemoryRegion xen_memory, xen_grants;
> >
> >-/* Check for xen memory.  */
> >+/* Check for any kind of xen memory, foreign mappings or grants.  */
> > bool xen_mr_is_memory(MemoryRegion *mr)
> > {
> >-return mr == _memory;
> >+return mr == _memory || mr == _grants;
> >+}
> >+
> >+/* Check specifically for grants.  */
> >+bool xen_mr_is_grants(MemoryRegion *mr)
> >+{
> >+return mr == _grants;
> > }
> >
> > void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion
> *mr,
> >diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> >index a07c47b0b1..1cbc2aeaa9 100644
> >--- a/hw/xen/xen-mapcache.c
> >+++ b/hw/xen/xen-mapcache.c
> >@@ -14,6 +14,7 @@
> >
> > #include 
> >
> >+#include "hw/xen/xen-hvm-common.h"
> > #include "hw/xen/xen_native.h"
> > #include "qemu/bitmap.h"
> >
> >@@ -21,6 +22,8 @@
> > #include "sysemu/xen-mapcache.h"
> > #include "trace.h"
> >
> >+#include 
> >+#include 
> >
> > #if HOST_LONG_BITS == 32
> > #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
> >@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
> > unsigned long *valid_mapping;
> > uint32_t lock;
> > #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> >+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
>
> Might we get more entry kinds in the future? (for example foreign maps).
> Maybe this could be an enum.
>
>
Perhaps. Foreign mappings are already supported, this flag separates
ordinary foreign mappings from grant foreign mappings.
IMO, since this is not an external interface it's probably better to change
it once we have a concrete use-case at hand.



> > uint8_t flags;
> > hwaddr size;
> > struct MapCacheEntry *next;
> >@@ -71,6 +75,8 @@ typedef struct MapCache {
> > } MapCache;
> >
> > static MapCache *mapcache;
> >+static MapCache *mapcache_grants;
> >+static xengnttab_handle *xen_region_gnttabdev;
> >
> > static inline void mapcache_lock(MapCache *mc)
> > {
> >@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f,
> void *opaque)
> > unsigned long max_mcache_size;
> > unsigned int bucket_shift;
> >
> >+xen_region_gnttabdev = xengnttab_open(NULL, 0);
> >+if (xen_region_gnttabdev == NULL) {
> >+error_report("mapcache: Failed to open gnttab device");
> >+exit(EXIT_FAILURE);
> >+}
> >+
> > if (HOST_LONG_BITS == 32) {
> > bucket_shift = 16;
> > } else {
> >@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f,
> void *opaque)
> > mapcache = xen_map_cache_init_single(f, opaque,
> >  bucket_shift,
> >  max_mcache_size);
> >+
> >+/*
> >+ * Grant mappings must use XC_PAGE_SIZE granularity since we can't
> >+ * map anything beyond the number of pages granted to us.
> >+ */
> >+mapcache_grants = xen_map_cache_init_single(f, opaque,
> >+XC_PAGE_SHIFT,
> >+max_mcache_siz

[PATCH v6 5/8] softmmu: Replace check for RAMBlock offset 0 with xen_mr_is_memory

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

For xen, when checking for the first RAM (xen_memory), use
xen_mr_is_memory() rather than checking for a RAMBlock with
offset 0.

All Xen machines create xen_memory first so this has no
functional change for existing machines.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 system/physmem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/system/physmem.c b/system/physmem.c
index 5e6257ef65..b7847db1a2 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2229,7 +2229,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * because we don't want to map the entire memory in QEMU.
  * In that case just map the requested area.
  */
-if (block->offset == 0) {
+if (xen_mr_is_memory(block->mr)) {
 return xen_map_cache(block->mr, block->offset + addr,
  len, lock, lock,
  is_write);
-- 
2.40.1

[PATCH v6 7/8] xen: mapcache: Add support for grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add a second mapcache for grant mappings. The mapcache for
grants needs to work with XC_PAGE_SIZE granularity since
we can't map larger ranges than what has been granted to us.

Like with foreign mappings (xen_memory), machines using grants
are expected to initialize the xen_grants MR and map it
into their address-map accordingly.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-hvm-common.c |  12 ++-
 hw/xen/xen-mapcache.c   | 163 ++--
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen.h|   7 ++
 4 files changed, 152 insertions(+), 33 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index a0a0252da0..b8ace1c368 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -10,12 +10,18 @@
 #include "hw/boards.h"
 #include "hw/xen/arch_hvm.h"
 
-MemoryRegion xen_memory;
+MemoryRegion xen_memory, xen_grants;
 
-/* Check for xen memory.  */
+/* Check for any kind of xen memory, foreign mappings or grants.  */
 bool xen_mr_is_memory(MemoryRegion *mr)
 {
-return mr == _memory;
+return mr == _memory || mr == _grants;
+}
+
+/* Check specifically for grants.  */
+bool xen_mr_is_grants(MemoryRegion *mr)
+{
+return mr == _grants;
 }
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index a07c47b0b1..1cbc2aeaa9 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -14,6 +14,7 @@
 
 #include 
 
+#include "hw/xen/xen-hvm-common.h"
 #include "hw/xen/xen_native.h"
 #include "qemu/bitmap.h"
 
@@ -21,6 +22,8 @@
 #include "sysemu/xen-mapcache.h"
 #include "trace.h"
 
+#include 
+#include 
 
 #if HOST_LONG_BITS == 32
 #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
 unsigned long *valid_mapping;
 uint32_t lock;
 #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
 uint8_t flags;
 hwaddr size;
 struct MapCacheEntry *next;
@@ -71,6 +75,8 @@ typedef struct MapCache {
 } MapCache;
 
 static MapCache *mapcache;
+static MapCache *mapcache_grants;
+static xengnttab_handle *xen_region_gnttabdev;
 
 static inline void mapcache_lock(MapCache *mc)
 {
@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 unsigned long max_mcache_size;
 unsigned int bucket_shift;
 
+xen_region_gnttabdev = xengnttab_open(NULL, 0);
+if (xen_region_gnttabdev == NULL) {
+error_report("mapcache: Failed to open gnttab device");
+exit(EXIT_FAILURE);
+}
+
 if (HOST_LONG_BITS == 32) {
 bucket_shift = 16;
 } else {
@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 mapcache = xen_map_cache_init_single(f, opaque,
  bucket_shift,
  max_mcache_size);
+
+/*
+ * Grant mappings must use XC_PAGE_SIZE granularity since we can't
+ * map anything beyond the number of pages granted to us.
+ */
+mapcache_grants = xen_map_cache_init_single(f, opaque,
+XC_PAGE_SHIFT,
+max_mcache_size);
+
 setrlimit(RLIMIT_AS, _as);
 }
 
@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
  hwaddr size,
  hwaddr address_index,
  bool dummy,
+ bool grant,
+ bool is_write,
  ram_addr_t ram_offset)
 {
 uint8_t *vaddr_base;
-xen_pfn_t *pfns;
+uint32_t *refs = NULL;
+xen_pfn_t *pfns = NULL;
 int *err;
 unsigned int i;
 hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
 
 trace_xen_remap_bucket(address_index);
 
-pfns = g_new0(xen_pfn_t, nb_pfn);
+if (grant) {
+refs = g_new0(uint32_t, nb_pfn);
+} else {
+pfns = g_new0(xen_pfn_t, nb_pfn);
+}
 err = g_new0(int, nb_pfn);
 
 if (entry->vaddr_base != NULL) {
@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
 g_free(entry->valid_mapping);
 entry->valid_mapping = NULL;
 
-for (i = 0; i < nb_pfn; i++) {
-pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+if (grant) {
+hwaddr grant_base = address_index - (ram_offset >> XC_PAGE_SHIFT);
+
+for (i = 0; i < nb_pfn; i++) {
+refs[i] = grant_base + i;
+}
+} else {
+for (i = 0; i < nb_pfn; i++) {
+pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + 
i;
+}
 }
 
-/*
- * If the cal

[PATCH v6 1/8] xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Make MCACHE_BUCKET_SHIFT runtime configurable per cache instance.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-mapcache.c | 54 ++-
 1 file changed, 33 insertions(+), 21 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index fa6813b1ad..bc860f4373 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -23,13 +23,10 @@
 
 
 #if HOST_LONG_BITS == 32
-#  define MCACHE_BUCKET_SHIFT 16
 #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
 #else
-#  define MCACHE_BUCKET_SHIFT 20
 #  define MCACHE_MAX_SIZE (1UL<<35) /* 32GB Cap */
 #endif
-#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
 
 /* This is the size of the virtual address space reserve to QEMU that will not
  * be use by MapCache.
@@ -65,7 +62,8 @@ typedef struct MapCache {
 /* For most cases (>99.9%), the page address is the same. */
 MapCacheEntry *last_entry;
 unsigned long max_mcache_size;
-unsigned int mcache_bucket_shift;
+unsigned int bucket_shift;
+unsigned long bucket_size;
 
 phys_offset_to_gaddr_t phys_offset_to_gaddr;
 QemuMutex lock;
@@ -95,11 +93,14 @@ static inline int test_bits(int nr, int size, const 
unsigned long *addr)
 
 static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
void *opaque,
+   unsigned int bucket_shift,
unsigned long max_size)
 {
 unsigned long size;
 MapCache *mc;
 
+assert(bucket_shift >= XC_PAGE_SHIFT);
+
 mc = g_new0(MapCache, 1);
 
 mc->phys_offset_to_gaddr = f;
@@ -108,12 +109,14 @@ static MapCache 
*xen_map_cache_init_single(phys_offset_to_gaddr_t f,
 
 QTAILQ_INIT(>locked_entries);
 
+mc->bucket_shift = bucket_shift;
+mc->bucket_size = 1UL << bucket_shift;
 mc->max_mcache_size = max_size;
 
 mc->nr_buckets =
 (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
-  (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
- (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
+  (1UL << (bucket_shift - XC_PAGE_SHIFT)) - 1) >>
+ (bucket_shift - XC_PAGE_SHIFT));
 
 size = mc->nr_buckets * sizeof(MapCacheEntry);
 size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
@@ -126,6 +129,13 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 {
 struct rlimit rlimit_as;
 unsigned long max_mcache_size;
+unsigned int bucket_shift;
+
+if (HOST_LONG_BITS == 32) {
+bucket_shift = 16;
+} else {
+bucket_shift = 20;
+}
 
 if (geteuid() == 0) {
 rlimit_as.rlim_cur = RLIM_INFINITY;
@@ -146,7 +156,9 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 }
 }
 
-mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
+mapcache = xen_map_cache_init_single(f, opaque,
+ bucket_shift,
+ max_mcache_size);
 setrlimit(RLIMIT_AS, _as);
 }
 
@@ -195,7 +207,7 @@ static void xen_remap_bucket(MapCache *mc,
 entry->valid_mapping = NULL;
 
 for (i = 0; i < nb_pfn; i++) {
-pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
+pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
 }
 
 /*
@@ -266,8 +278,8 @@ static uint8_t *xen_map_cache_unlocked(MapCache *mc,
 bool dummy = false;
 
 tryagain:
-address_index  = phys_addr >> MCACHE_BUCKET_SHIFT;
-address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
+address_index  = phys_addr >> mc->bucket_shift;
+address_offset = phys_addr & (mc->bucket_size - 1);
 
 trace_xen_map_cache(phys_addr);
 
@@ -294,14 +306,14 @@ tryagain:
 return mc->last_entry->vaddr_base + address_offset;
 }
 
-/* size is always a multiple of MCACHE_BUCKET_SIZE */
+/* size is always a multiple of mc->bucket_size */
 if (size) {
 cache_size = size + address_offset;
-if (cache_size % MCACHE_BUCKET_SIZE) {
-cache_size += MCACHE_BUCKET_SIZE - (cache_size % 
MCACHE_BUCKET_SIZE);
+if (cache_size % mc->bucket_size) {
+cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
 }
 } else {
-cache_size = MCACHE_BUCKET_SIZE;
+cache_size = mc->bucket_size;
 }
 
 entry = >entry[address_index % mc->nr_buckets];
@@ -422,7 +434,7 @@ static ram_addr_t 
xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
 raddr = RAM_ADDR_INVALID;
 } else {
-raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
+   

[PATCH v6 0/8] xen: Support grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Hi,

Grant mappings are a mechanism in Xen for guests to grant each other
permissions to map and share pages. These grants can be temporary
so both map and unmaps must be respected. See here for more info:
https://github.com/xen-project/xen/blob/master/docs/misc/grant-tables.txt

Currently, the primary use-case for grants in QEMU, is with VirtIO backends.
Grant mappings will only work with models that use the address_space_map/unmap
interfaces, any other access will fail with appropriate error messages.

In response to feedback we got on v3, later version switch approach
from adding new MemoryRegion types and map/unmap hooks to instead reusing
the existing xen_map_cache() hooks (with extensions). Almost all of the
changes are now contained to the Xen modules.

This approach also refactors the mapcache to support multiple instances
(one for existing foreign mappings and another for grant mappings).

I've only enabled grants for the ARM PVH machine since that is what
I can currently test on.

Cheers,
Edgar

ChangeLog:

v5 -> v6:
* Correct passing of ram_addr_offset in xen_replace_cache_entry_unlocked.

v4 -> v5:
* Compute grant_ref from address_index to xen_remap_bucket().
* Rename grant_is_write to is_write.
* Remove unnecessary + mc->bucket_size - 1 in
  xen_invalidate_map_cache_entry_unlocked().
* Remove use of global mapcache in refactor of
  xen_replace_cache_entry_unlocked().
* Add error checking for xengnttab_unmap().
* Add assert in xen_replace_cache_entry_unlocked() against grant mappings.
* Fix memory leak when freeing first entry in mapcache buckets.
* Assert that bucket_shift is >= XC_PAGE_SHIFT when creating mapcache.
* Add missing use of xen_mr_is_memory() in hw/xen/xen-hvm-common.c.
* Rebase with master.

v3 -> v4:
* Reuse existing xen_map_cache hooks.
* Reuse existing map-cache for both foreign and grant mappings.
* Only enable grants for the ARM PVH machine (removed i386).

v2 -> v3:
* Drop patch 1/7. This was done because device unplug is an x86-only case.
* Add missing qemu_mutex_unlock() before return.

v1 -> v2:
* Split patch 2/7 to keep phymem.c changes in a separate.
* In patch "xen: add map and unmap callbacks for grant" add check for total
  allowed grant < XEN_MAX_VIRTIO_GRANTS.
* Fix formatting issues and re-based with master latest.


Edgar E. Iglesias (8):
  xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable
  xen: mapcache: Unmap first entries in buckets
  xen: Add xen_mr_is_memory()
  softmmu: xen: Always pass offset + addr to xen_map_cache
  softmmu: Replace check for RAMBlock offset 0 with xen_mr_is_memory
  xen: mapcache: Pass the ram_addr offset to xen_map_cache()
  xen: mapcache: Add support for grant mappings
  hw/arm: xen: Enable use of grant mappings

 hw/arm/xen_arm.c|   5 +
 hw/xen/xen-hvm-common.c |  18 ++-
 hw/xen/xen-mapcache.c   | 232 
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen-mapcache.h   |   2 +
 include/sysemu/xen.h|  15 +++
 system/physmem.c|  12 +-
 7 files changed, 226 insertions(+), 61 deletions(-)

-- 
2.40.1

[PATCH v6 6/8] xen: mapcache: Pass the ram_addr offset to xen_map_cache()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Pass the ram_addr offset to xen_map_cache.
This is in preparation for adding grant mappings that need
to compute the address within the RAMBlock.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 16 +++-
 include/sysemu/xen-mapcache.h |  2 ++
 system/physmem.c  |  9 +
 3 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index ec95445696..a07c47b0b1 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
  void *vaddr,
  hwaddr size,
  hwaddr address_index,
- bool dummy)
+ bool dummy,
+ ram_addr_t ram_offset)
 {
 uint8_t *vaddr_base;
 xen_pfn_t *pfns;
@@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_offset,
uint8_t lock, bool dma, bool is_write)
 {
 MapCacheEntry *entry, *pentry = NULL,
@@ -337,14 +339,16 @@ tryagain:
 if (!entry) {
 entry = g_new0(MapCacheEntry, 1);
 pentry->next = entry;
-xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+ ram_offset);
 } else if (!entry->lock) {
 if (!entry->vaddr_base || entry->paddr_index != address_index ||
 entry->size != cache_size ||
 !test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
-xen_remap_bucket(mc, entry, NULL, cache_size, address_index, 
dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+ ram_offset);
 }
 }
 
@@ -391,13 +395,15 @@ tryagain:
 
 uint8_t *xen_map_cache(MemoryRegion *mr,
hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_addr_offset,
uint8_t lock, bool dma,
bool is_write)
 {
 uint8_t *p;
 
 mapcache_lock(mapcache);
-p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
+p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
+   lock, dma, is_write);
 mapcache_unlock(mapcache);
 return p;
 }
@@ -632,7 +638,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache 
*mc,
 trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
 xen_remap_bucket(mc, entry, entry->vaddr_base,
- cache_size, address_index, false);
+ cache_size, address_index, false, old_phys_addr);
 if (!test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index 1ec9e66752..b5e3ea1bc0 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
 void xen_map_cache_init(phys_offset_to_gaddr_t f,
 void *opaque);
 uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_addr_offset,
uint8_t lock, bool dma,
bool is_write);
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
@@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t 
f,
 static inline uint8_t *xen_map_cache(MemoryRegion *mr,
  hwaddr phys_addr,
  hwaddr size,
+ ram_addr_t ram_addr_offset,
  uint8_t lock,
  bool dma,
  bool is_write)
diff --git a/system/physmem.c b/system/physmem.c
index b7847db1a2..33d09f7571 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2231,13 +2231,14 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  */
 if (xen_mr_is_memory(block->mr)) {
 return xen_map_cache(block->mr, block->offset + addr,
- len, lock, lock,
- is_write);
+ len, block->offset,
+ lock, lock, is_write);

[PATCH v6 4/8] softmmu: xen: Always pass offset + addr to xen_map_cache

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Always pass address with offset to xen_map_cache().
This is in preparation for support for grant mappings.

Since this is within a block that checks for offset == 0,
this has no functional changes.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 system/physmem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/system/physmem.c b/system/physmem.c
index 342b7a8fd4..5e6257ef65 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2230,7 +2230,8 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * In that case just map the requested area.
  */
 if (block->offset == 0) {
-return xen_map_cache(block->mr, addr, len, lock, lock,
+return xen_map_cache(block->mr, block->offset + addr,
+ len, lock, lock,
  is_write);
 }
 
-- 
2.40.1

[PATCH v6 2/8] xen: mapcache: Unmap first entries in buckets

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

When invalidating memory ranges, if we happen to hit the first
entry in a bucket we were never unmapping it. This was harmless
for foreign mappings but now that we're looking to reuse the
mapcache for transient grant mappings, we must unmap entries
when invalidated.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-mapcache.c | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index bc860f4373..ec95445696 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -491,18 +491,23 @@ static void 
xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
 return;
 }
 entry->lock--;
-if (entry->lock > 0 || pentry == NULL) {
+if (entry->lock > 0) {
 return;
 }
 
-pentry->next = entry->next;
 ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
 if (munmap(entry->vaddr_base, entry->size) != 0) {
 perror("unmap fails");
 exit(-1);
 }
+
 g_free(entry->valid_mapping);
-g_free(entry);
+if (pentry) {
+pentry->next = entry->next;
+g_free(entry);
+} else {
+memset(entry, 0, sizeof *entry);
+}
 }
 
 typedef struct XenMapCacheData {
-- 
2.40.1

[PATCH v6 8/8] hw/arm: xen: Enable use of grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/arm/xen_arm.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/hw/arm/xen_arm.c b/hw/arm/xen_arm.c
index 15fa7dfa84..6fad829ede 100644
--- a/hw/arm/xen_arm.c
+++ b/hw/arm/xen_arm.c
@@ -125,6 +125,11 @@ static void xen_init_ram(MachineState *machine)
  GUEST_RAM1_BASE, ram_size[1]);
 memory_region_add_subregion(sysmem, GUEST_RAM1_BASE, _hi);
 }
+
+/* Setup support for grants.  */
+memory_region_init_ram(_grants, NULL, "xen.grants", block_len,
+   _fatal);
+memory_region_add_subregion(sysmem, XEN_GRANT_ADDR_OFF, _grants);
 }
 
 void arch_handle_ioreq(XenIOState *state, ioreq_t *req)
-- 
2.40.1

[PATCH v6 3/8] xen: Add xen_mr_is_memory()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add xen_mr_is_memory() to abstract away tests for the
xen_memory MR.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Acked-by: David Hildenbrand 
---
 hw/xen/xen-hvm-common.c | 10 --
 include/sysemu/xen.h|  8 
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 2d1b032121..a0a0252da0 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -12,6 +12,12 @@
 
 MemoryRegion xen_memory;
 
+/* Check for xen memory.  */
+bool xen_mr_is_memory(MemoryRegion *mr)
+{
+return mr == _memory;
+}
+
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
Error **errp)
 {
@@ -28,7 +34,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, 
MemoryRegion *mr,
 return;
 }
 
-if (mr == _memory) {
+if (xen_mr_is_memory(mr)) {
 return;
 }
 
@@ -55,7 +61,7 @@ static void xen_set_memory(struct MemoryListener *listener,
 {
 XenIOState *state = container_of(listener, XenIOState, memory_listener);
 
-if (section->mr == _memory) {
+if (xen_mr_is_memory(section->mr)) {
 return;
 } else {
 if (add) {
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index 754ec2e6cb..dc72f83bcb 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t 
length);
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
struct MemoryRegion *mr, Error **errp);
 
+bool xen_mr_is_memory(MemoryRegion *mr);
+
 #else /* !CONFIG_XEN_IS_POSSIBLE */
 
 #define xen_enabled() 0
@@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, 
ram_addr_t size,
 g_assert_not_reached();
 }
 
+static inline bool xen_mr_is_memory(MemoryRegion *mr)
+{
+g_assert_not_reached();
+return false;
+}
+
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 #endif
-- 
2.40.1

Re: [PATCH v5 6/8] xen: mapcache: Pass the ram_addr offset to xen_map_cache()

[Edgar E. Iglesias]

On Thu, May 16, 2024 at 1:08 AM Stefano Stabellini
 wrote:
>
> On Fri, 3 May 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" 
> >
> > Pass the ram_addr offset to xen_map_cache.
> > This is in preparation for adding grant mappings that need
> > to compute the address within the RAMBlock.
> >
> > No functional changes.
> >
> > Signed-off-by: Edgar E. Iglesias 
> > ---
> >  hw/xen/xen-mapcache.c | 16 +++-
> >  include/sysemu/xen-mapcache.h |  2 ++
> >  system/physmem.c  |  9 +
> >  3 files changed, 18 insertions(+), 9 deletions(-)
> >
> > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > index ec95445696..26bc38a9e3 100644
> > --- a/hw/xen/xen-mapcache.c
> > +++ b/hw/xen/xen-mapcache.c
> > @@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
> >   void *vaddr,
> >   hwaddr size,
> >   hwaddr address_index,
> > - bool dummy)
> > + bool dummy,
> > + ram_addr_t ram_offset)
> >  {
> >  uint8_t *vaddr_base;
> >  xen_pfn_t *pfns;
> > @@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
> >
> >  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> > hwaddr phys_addr, hwaddr size,
> > +   ram_addr_t ram_offset,
> > uint8_t lock, bool dma, bool 
> > is_write)
> >  {
> >  MapCacheEntry *entry, *pentry = NULL,
> > @@ -337,14 +339,16 @@ tryagain:
> >  if (!entry) {
> >  entry = g_new0(MapCacheEntry, 1);
> >  pentry->next = entry;
> > -xen_remap_bucket(mc, entry, NULL, cache_size, address_index, 
> > dummy);
> > +xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
> > + ram_offset);
> >  } else if (!entry->lock) {
> >  if (!entry->vaddr_base || entry->paddr_index != address_index ||
> >  entry->size != cache_size ||
> >  !test_bits(address_offset >> XC_PAGE_SHIFT,
> >  test_bit_size >> XC_PAGE_SHIFT,
> >  entry->valid_mapping)) {
> > -xen_remap_bucket(mc, entry, NULL, cache_size, address_index, 
> > dummy);
> > +xen_remap_bucket(mc, entry, NULL, cache_size, address_index, 
> > dummy,
> > + ram_offset);
> >  }
> >  }
> >
> > @@ -391,13 +395,15 @@ tryagain:
> >
> >  uint8_t *xen_map_cache(MemoryRegion *mr,
> > hwaddr phys_addr, hwaddr size,
> > +   ram_addr_t ram_addr_offset,
> > uint8_t lock, bool dma,
> > bool is_write)
> >  {
> >  uint8_t *p;
> >
> >  mapcache_lock(mapcache);
> > -p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, 
> > is_write);
> > +p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
> > +   lock, dma, is_write);
> >  mapcache_unlock(mapcache);
> >  return p;
> >  }
> > @@ -632,7 +638,7 @@ static uint8_t 
> > *xen_replace_cache_entry_unlocked(MapCache *mc,
> >  trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
> >
> >  xen_remap_bucket(mc, entry, entry->vaddr_base,
> > - cache_size, address_index, false);
> > + cache_size, address_index, false, new_phys_addr);
>
> Everything else makes sense, but I don't understand how can it be that
> new_phys_addr is the block->offset here?
>

Agreed, this should be old_phys_addr propagated via phys_offset from
xen_add_to_physmap().
The reason this didn't have an effect here is that we currently only
use the ram_addr offset for grants,
and xen_replace_cache_entry is never called for grants. Anyway, I'll
fix this in the next version!




>
> >  if (!test_bits(address_offset >> XC_PAGE_SHIFT,
> >  test_bit_size >> XC_PAGE_SHIFT,
> >  entry->valid_mapping)) {
> > diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
> > index 1ec9e66752..b5e3ea1bc0 100644
> > --- a/include/sysemu/xen-mapcache.h
> > +++ b/include/sysemu/x

[PATCH v5 5/8] softmmu: Replace check for RAMBlock offset 0 with xen_mr_is_memory

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

For xen, when checking for the first RAM (xen_memory), use
xen_mr_is_memory() rather than checking for a RAMBlock with
offset 0.

All Xen machines create xen_memory first so this has no
functional change for existing machines.

Signed-off-by: Edgar E. Iglesias 
---
 system/physmem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/system/physmem.c b/system/physmem.c
index 5e6257ef65..b7847db1a2 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2229,7 +2229,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * because we don't want to map the entire memory in QEMU.
  * In that case just map the requested area.
  */
-if (block->offset == 0) {
+if (xen_mr_is_memory(block->mr)) {
 return xen_map_cache(block->mr, block->offset + addr,
  len, lock, lock,
  is_write);
-- 
2.40.1

[PATCH v5 6/8] xen: mapcache: Pass the ram_addr offset to xen_map_cache()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Pass the ram_addr offset to xen_map_cache.
This is in preparation for adding grant mappings that need
to compute the address within the RAMBlock.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 16 +++-
 include/sysemu/xen-mapcache.h |  2 ++
 system/physmem.c  |  9 +
 3 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index ec95445696..26bc38a9e3 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -167,7 +167,8 @@ static void xen_remap_bucket(MapCache *mc,
  void *vaddr,
  hwaddr size,
  hwaddr address_index,
- bool dummy)
+ bool dummy,
+ ram_addr_t ram_offset)
 {
 uint8_t *vaddr_base;
 xen_pfn_t *pfns;
@@ -266,6 +267,7 @@ static void xen_remap_bucket(MapCache *mc,
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_offset,
uint8_t lock, bool dma, bool is_write)
 {
 MapCacheEntry *entry, *pentry = NULL,
@@ -337,14 +339,16 @@ tryagain:
 if (!entry) {
 entry = g_new0(MapCacheEntry, 1);
 pentry->next = entry;
-xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+ ram_offset);
 } else if (!entry->lock) {
 if (!entry->vaddr_base || entry->paddr_index != address_index ||
 entry->size != cache_size ||
 !test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
-xen_remap_bucket(mc, entry, NULL, cache_size, address_index, 
dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+ ram_offset);
 }
 }
 
@@ -391,13 +395,15 @@ tryagain:
 
 uint8_t *xen_map_cache(MemoryRegion *mr,
hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_addr_offset,
uint8_t lock, bool dma,
bool is_write)
 {
 uint8_t *p;
 
 mapcache_lock(mapcache);
-p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
+p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
+   lock, dma, is_write);
 mapcache_unlock(mapcache);
 return p;
 }
@@ -632,7 +638,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache 
*mc,
 trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
 xen_remap_bucket(mc, entry, entry->vaddr_base,
- cache_size, address_index, false);
+ cache_size, address_index, false, new_phys_addr);
 if (!test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index 1ec9e66752..b5e3ea1bc0 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
 void xen_map_cache_init(phys_offset_to_gaddr_t f,
 void *opaque);
 uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_addr_offset,
uint8_t lock, bool dma,
bool is_write);
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
@@ -37,6 +38,7 @@ static inline void xen_map_cache_init(phys_offset_to_gaddr_t 
f,
 static inline uint8_t *xen_map_cache(MemoryRegion *mr,
  hwaddr phys_addr,
  hwaddr size,
+ ram_addr_t ram_addr_offset,
  uint8_t lock,
  bool dma,
  bool is_write)
diff --git a/system/physmem.c b/system/physmem.c
index b7847db1a2..33d09f7571 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2231,13 +2231,14 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  */
 if (xen_mr_is_memory(block->mr)) {
 return xen_map_cache(block->mr, block->offset + addr,
- len, lock, lock,
- is_write);
+ len, block->offset,
+ lock, lock, is_write);

[PATCH v5 2/8] xen: mapcache: Unmap first entries in buckets

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

When invalidating memory ranges, if we happen to hit the first
entry in a bucket we were never unmapping it. This was harmless
for foreign mappings but now that we're looking to reuse the
mapcache for transient grant mappings, we must unmap entries
when invalidated.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-mapcache.c | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index bc860f4373..ec95445696 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -491,18 +491,23 @@ static void 
xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
 return;
 }
 entry->lock--;
-if (entry->lock > 0 || pentry == NULL) {
+if (entry->lock > 0) {
 return;
 }
 
-pentry->next = entry->next;
 ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
 if (munmap(entry->vaddr_base, entry->size) != 0) {
 perror("unmap fails");
 exit(-1);
 }
+
 g_free(entry->valid_mapping);
-g_free(entry);
+if (pentry) {
+pentry->next = entry->next;
+g_free(entry);
+} else {
+memset(entry, 0, sizeof *entry);
+}
 }
 
 typedef struct XenMapCacheData {
-- 
2.40.1

[PATCH v5 7/8] xen: mapcache: Add support for grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add a second mapcache for grant mappings. The mapcache for
grants needs to work with XC_PAGE_SIZE granularity since
we can't map larger ranges than what has been granted to us.

Like with foreign mappings (xen_memory), machines using grants
are expected to initialize the xen_grants MR and map it
into their address-map accordingly.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-hvm-common.c |  12 ++-
 hw/xen/xen-mapcache.c   | 163 ++--
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen.h|   7 ++
 4 files changed, 152 insertions(+), 33 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index c94f1990c5..7a1e2ce4b3 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -10,12 +10,18 @@
 #include "hw/boards.h"
 #include "hw/xen/arch_hvm.h"
 
-MemoryRegion xen_memory;
+MemoryRegion xen_memory, xen_grants;
 
-/* Check for xen memory.  */
+/* Check for any kind of xen memory, foreign mappings or grants.  */
 bool xen_mr_is_memory(MemoryRegion *mr)
 {
-return mr == _memory;
+return mr == _memory || mr == _grants;
+}
+
+/* Check specifically for grants.  */
+bool xen_mr_is_grants(MemoryRegion *mr)
+{
+return mr == _grants;
 }
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 26bc38a9e3..25041ab02d 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -14,6 +14,7 @@
 
 #include 
 
+#include "hw/xen/xen-hvm-common.h"
 #include "hw/xen/xen_native.h"
 #include "qemu/bitmap.h"
 
@@ -21,6 +22,8 @@
 #include "sysemu/xen-mapcache.h"
 #include "trace.h"
 
+#include 
+#include 
 
 #if HOST_LONG_BITS == 32
 #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
 unsigned long *valid_mapping;
 uint32_t lock;
 #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
 uint8_t flags;
 hwaddr size;
 struct MapCacheEntry *next;
@@ -71,6 +75,8 @@ typedef struct MapCache {
 } MapCache;
 
 static MapCache *mapcache;
+static MapCache *mapcache_grants;
+static xengnttab_handle *xen_region_gnttabdev;
 
 static inline void mapcache_lock(MapCache *mc)
 {
@@ -131,6 +137,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 unsigned long max_mcache_size;
 unsigned int bucket_shift;
 
+xen_region_gnttabdev = xengnttab_open(NULL, 0);
+if (xen_region_gnttabdev == NULL) {
+error_report("mapcache: Failed to open gnttab device");
+exit(EXIT_FAILURE);
+}
+
 if (HOST_LONG_BITS == 32) {
 bucket_shift = 16;
 } else {
@@ -159,6 +171,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 mapcache = xen_map_cache_init_single(f, opaque,
  bucket_shift,
  max_mcache_size);
+
+/*
+ * Grant mappings must use XC_PAGE_SIZE granularity since we can't
+ * map anything beyond the number of pages granted to us.
+ */
+mapcache_grants = xen_map_cache_init_single(f, opaque,
+XC_PAGE_SHIFT,
+max_mcache_size);
+
 setrlimit(RLIMIT_AS, _as);
 }
 
@@ -168,17 +189,24 @@ static void xen_remap_bucket(MapCache *mc,
  hwaddr size,
  hwaddr address_index,
  bool dummy,
+ bool grant,
+ bool is_write,
  ram_addr_t ram_offset)
 {
 uint8_t *vaddr_base;
-xen_pfn_t *pfns;
+uint32_t *refs = NULL;
+xen_pfn_t *pfns = NULL;
 int *err;
 unsigned int i;
 hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
 
 trace_xen_remap_bucket(address_index);
 
-pfns = g_new0(xen_pfn_t, nb_pfn);
+if (grant) {
+refs = g_new0(uint32_t, nb_pfn);
+} else {
+pfns = g_new0(xen_pfn_t, nb_pfn);
+}
 err = g_new0(int, nb_pfn);
 
 if (entry->vaddr_base != NULL) {
@@ -207,21 +235,51 @@ static void xen_remap_bucket(MapCache *mc,
 g_free(entry->valid_mapping);
 entry->valid_mapping = NULL;
 
-for (i = 0; i < nb_pfn; i++) {
-pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+if (grant) {
+hwaddr grant_base = address_index - (ram_offset >> XC_PAGE_SHIFT);
+
+for (i = 0; i < nb_pfn; i++) {
+refs[i] = grant_base + i;
+}
+} else {
+for (i = 0; i < nb_pfn; i++) {
+pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + 
i;
+}
 }
 
-/*
- * If the caller has requested t

[PATCH v5 3/8] xen: Add xen_mr_is_memory()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add xen_mr_is_memory() to abstract away tests for the
xen_memory MR.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Acked-by: David Hildenbrand 
---
 hw/xen/xen-hvm-common.c | 10 --
 include/sysemu/xen.h|  8 
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 1627da7398..c94f1990c5 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -12,6 +12,12 @@
 
 MemoryRegion xen_memory;
 
+/* Check for xen memory.  */
+bool xen_mr_is_memory(MemoryRegion *mr)
+{
+return mr == _memory;
+}
+
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
Error **errp)
 {
@@ -28,7 +34,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, 
MemoryRegion *mr,
 return;
 }
 
-if (mr == _memory) {
+if (xen_mr_is_memory(mr)) {
 return;
 }
 
@@ -55,7 +61,7 @@ static void xen_set_memory(struct MemoryListener *listener,
 {
 XenIOState *state = container_of(listener, XenIOState, memory_listener);
 
-if (section->mr == _memory) {
+if (xen_mr_is_memory(section->mr)) {
 return;
 } else {
 if (add) {
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index 754ec2e6cb..dc72f83bcb 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t 
length);
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
struct MemoryRegion *mr, Error **errp);
 
+bool xen_mr_is_memory(MemoryRegion *mr);
+
 #else /* !CONFIG_XEN_IS_POSSIBLE */
 
 #define xen_enabled() 0
@@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, 
ram_addr_t size,
 g_assert_not_reached();
 }
 
+static inline bool xen_mr_is_memory(MemoryRegion *mr)
+{
+g_assert_not_reached();
+return false;
+}
+
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 #endif
-- 
2.40.1

[PATCH v5 0/8] xen: Support grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Hi,

Grant mappings are a mechanism in Xen for guests to grant each other
permissions to map and share pages. These grants can be temporary
so both map and unmaps must be respected. See here for more info:
https://github.com/xen-project/xen/blob/master/docs/misc/grant-tables.txt

Currently, the primary use-case for grants in QEMU, is with VirtIO backends.
Grant mappings will only work with models that use the address_space_map/unmap
interfaces, any other access will fail with appropriate error messages.

In response to feedback we got on v3, later version switch approach
from adding new MemoryRegion types and map/unmap hooks to instead reusing
the existing xen_map_cache() hooks (with extensions). Almost all of the
changes are now contained to the Xen modules.

This approach also refactors the mapcache to support multiple instances
(one for existing foreign mappings and another for grant mappings).

I've only enabled grants for the ARM PVH machine since that is what
I can currently test on.

Cheers,
Edgar

ChangeLog:

v4 -> v5:
* Compute grant_ref from address_index to xen_remap_bucket().
* Rename grant_is_write to is_write and comment on why foreign
  mappings don't yet use it.
* Remove unnecessary + mc->bucket_size - 1 in
  xen_invalidate_map_cache_entry_unlocked().
* Remove use of global mapcache in refactor of
  xen_replace_cache_entry_unlocked().
* Add error checking for xengnttab_unmap().
* Add assert in xen_replace_cache_entry_unlocked() against grant mappings.
* Fix memory leak when freeing first entry in mapcache buckets.
* Assert that bucket_shift is >= XC_PAGE_SHIFT when creating mapcache.
* Add missing use of xen_mr_is_memory() in hw/xen/xen-hvm-common.c.
* Rebase with master.

v3 -> v4:
* Reuse existing xen_map_cache hooks.
* Reuse existing map-cache for both foreign and grant mappings.
* Only enable grants for the ARM PVH machine (removed i386).

v2 -> v3:
* Drop patch 1/7. This was done because device unplug is an x86-only case.
* Add missing qemu_mutex_unlock() before return.

v1 -> v2:
* Split patch 2/7 to keep phymem.c changes in a separate.
* In patch "xen: add map and unmap callbacks for grant" add check for total
  allowed grant < XEN_MAX_VIRTIO_GRANTS.
* Fix formatting issues and re-based with master latest.


Edgar E. Iglesias (8):
  xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable
  xen: mapcache: Unmap first entries in buckets
  xen: Add xen_mr_is_memory()
  softmmu: xen: Always pass offset + addr to xen_map_cache
  softmmu: Replace check for RAMBlock offset 0 with xen_mr_is_memory
  xen: mapcache: Pass the ram_addr offset to xen_map_cache()
  xen: mapcache: Add support for grant mappings
  hw/arm: xen: Enable use of grant mappings

 hw/arm/xen_arm.c|   5 +
 hw/xen/xen-hvm-common.c |  18 ++-
 hw/xen/xen-mapcache.c   | 232 
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen-mapcache.h   |   2 +
 include/sysemu/xen.h|  15 +++
 system/physmem.c|  12 +-
 7 files changed, 226 insertions(+), 61 deletions(-)

-- 
2.40.1

[PATCH v5 4/8] softmmu: xen: Always pass offset + addr to xen_map_cache

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Always pass address with offset to xen_map_cache().
This is in preparation for support for grant mappings.

Since this is within a block that checks for offset == 0,
this has no functional changes.

Signed-off-by: Edgar E. Iglesias 
---
 system/physmem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/system/physmem.c b/system/physmem.c
index 342b7a8fd4..5e6257ef65 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2230,7 +2230,8 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * In that case just map the requested area.
  */
 if (block->offset == 0) {
-return xen_map_cache(block->mr, addr, len, lock, lock,
+return xen_map_cache(block->mr, block->offset + addr,
+ len, lock, lock,
  is_write);
 }
 
-- 
2.40.1

[PATCH v5 1/8] xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Make MCACHE_BUCKET_SHIFT runtime configurable per cache instance.

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/xen/xen-mapcache.c | 54 ++-
 1 file changed, 33 insertions(+), 21 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index fa6813b1ad..bc860f4373 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -23,13 +23,10 @@
 
 
 #if HOST_LONG_BITS == 32
-#  define MCACHE_BUCKET_SHIFT 16
 #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
 #else
-#  define MCACHE_BUCKET_SHIFT 20
 #  define MCACHE_MAX_SIZE (1UL<<35) /* 32GB Cap */
 #endif
-#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
 
 /* This is the size of the virtual address space reserve to QEMU that will not
  * be use by MapCache.
@@ -65,7 +62,8 @@ typedef struct MapCache {
 /* For most cases (>99.9%), the page address is the same. */
 MapCacheEntry *last_entry;
 unsigned long max_mcache_size;
-unsigned int mcache_bucket_shift;
+unsigned int bucket_shift;
+unsigned long bucket_size;
 
 phys_offset_to_gaddr_t phys_offset_to_gaddr;
 QemuMutex lock;
@@ -95,11 +93,14 @@ static inline int test_bits(int nr, int size, const 
unsigned long *addr)
 
 static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
void *opaque,
+   unsigned int bucket_shift,
unsigned long max_size)
 {
 unsigned long size;
 MapCache *mc;
 
+assert(bucket_shift >= XC_PAGE_SHIFT);
+
 mc = g_new0(MapCache, 1);
 
 mc->phys_offset_to_gaddr = f;
@@ -108,12 +109,14 @@ static MapCache 
*xen_map_cache_init_single(phys_offset_to_gaddr_t f,
 
 QTAILQ_INIT(>locked_entries);
 
+mc->bucket_shift = bucket_shift;
+mc->bucket_size = 1UL << bucket_shift;
 mc->max_mcache_size = max_size;
 
 mc->nr_buckets =
 (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
-  (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
- (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
+  (1UL << (bucket_shift - XC_PAGE_SHIFT)) - 1) >>
+ (bucket_shift - XC_PAGE_SHIFT));
 
 size = mc->nr_buckets * sizeof(MapCacheEntry);
 size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
@@ -126,6 +129,13 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 {
 struct rlimit rlimit_as;
 unsigned long max_mcache_size;
+unsigned int bucket_shift;
+
+if (HOST_LONG_BITS == 32) {
+bucket_shift = 16;
+} else {
+bucket_shift = 20;
+}
 
 if (geteuid() == 0) {
 rlimit_as.rlim_cur = RLIM_INFINITY;
@@ -146,7 +156,9 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 }
 }
 
-mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
+mapcache = xen_map_cache_init_single(f, opaque,
+ bucket_shift,
+ max_mcache_size);
 setrlimit(RLIMIT_AS, _as);
 }
 
@@ -195,7 +207,7 @@ static void xen_remap_bucket(MapCache *mc,
 entry->valid_mapping = NULL;
 
 for (i = 0; i < nb_pfn; i++) {
-pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
+pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
 }
 
 /*
@@ -266,8 +278,8 @@ static uint8_t *xen_map_cache_unlocked(MapCache *mc,
 bool dummy = false;
 
 tryagain:
-address_index  = phys_addr >> MCACHE_BUCKET_SHIFT;
-address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
+address_index  = phys_addr >> mc->bucket_shift;
+address_offset = phys_addr & (mc->bucket_size - 1);
 
 trace_xen_map_cache(phys_addr);
 
@@ -294,14 +306,14 @@ tryagain:
 return mc->last_entry->vaddr_base + address_offset;
 }
 
-/* size is always a multiple of MCACHE_BUCKET_SIZE */
+/* size is always a multiple of mc->bucket_size */
 if (size) {
 cache_size = size + address_offset;
-if (cache_size % MCACHE_BUCKET_SIZE) {
-cache_size += MCACHE_BUCKET_SIZE - (cache_size % 
MCACHE_BUCKET_SIZE);
+if (cache_size % mc->bucket_size) {
+cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
 }
 } else {
-cache_size = MCACHE_BUCKET_SIZE;
+cache_size = mc->bucket_size;
 }
 
 entry = >entry[address_index % mc->nr_buckets];
@@ -422,7 +434,7 @@ static ram_addr_t 
xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
 raddr = RAM_ADDR_INVALID;
 } else {
-raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
+   

[PATCH v5 8/8] hw/arm: xen: Enable use of grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
---
 hw/arm/xen_arm.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/hw/arm/xen_arm.c b/hw/arm/xen_arm.c
index 15fa7dfa84..6fad829ede 100644
--- a/hw/arm/xen_arm.c
+++ b/hw/arm/xen_arm.c
@@ -125,6 +125,11 @@ static void xen_init_ram(MachineState *machine)
  GUEST_RAM1_BASE, ram_size[1]);
 memory_region_add_subregion(sysmem, GUEST_RAM1_BASE, _hi);
 }
+
+/* Setup support for grants.  */
+memory_region_init_ram(_grants, NULL, "xen.grants", block_len,
+   _fatal);
+memory_region_add_subregion(sysmem, XEN_GRANT_ADDR_OFF, _grants);
 }
 
 void arch_handle_ioreq(XenIOState *state, ioreq_t *req)
-- 
2.40.1

Re: [PATCH v4 15/17] xen: mapcache: Remove assumption of RAMBlock with 0 offset

[Edgar E. Iglesias]

On Thu, May 2, 2024 at 10:02 PM Stefano Stabellini
 wrote:
>
> On Thu, 2 May 2024, Edgar E. Iglesias wrote:
> > On Thu, May 2, 2024 at 8:53 PM Stefano Stabellini
> >  wrote:
> > >
> > > +Xenia
> > >
> > > On Thu, 2 May 2024, Edgar E. Iglesias wrote:
> > > > On Wed, May 1, 2024 at 11:24 PM Stefano Stabellini
> > > >  wrote:
> > > > >
> > > > > On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > > > > > From: "Edgar E. Iglesias" 
> > > > > >
> > > > > > The current mapcache assumes that all memory is mapped
> > > > > > in a single RAM MR (the first one with offset 0). Remove
> > > > > > this assumption and propagate the offset to the mapcache
> > > > > > so it can do reverse mappings (from hostptr -> ram_addr).
> > > > > >
> > > > > > This is in preparation for adding grant mappings.
> > > > > >
> > > > > > Signed-off-by: Edgar E. Iglesias 
> > > > >
> > > > >
> > > > > Looking at xen_remap_bucket, it is only using address_index (without
> > > > > adding ram_offset) to map foreign memory. From xen_remap_bucket, I 
> > > > > would
> > > > > understand that address_index already includes the ram_offset.
> > > > >
> > > > > Meaning that if we want to map foreign mapping at address 0x5000, then
> > > > > address_index would be 0x5000, even if ram_offset is 0x1000.
> > > > >
> > > > > But then looking xen_ram_addr_from_mapcache_single ram_offset is added
> > > > > to paddr_index to calculate the physical address. So in that case we
> > > > > would want address_index to be 0x4000 and ram_offset to be 0x1000. But
> > > > > xen_remap_bucket would have to sum address_index and ram_offset to map
> > > > > foreign memory.
> > > > >
> > > > > So I am a bit confused, did I get it wrong? One more comment below.
> > > > >
> > > >
> > > > Thanks Stefano,
> > > >
> > > > I think the confusion is that this ram_addr_offset is not related to
> > > > guest address-space.
> > > > It's a QEMU internal thing and it shouldn't be included in the address
> > > > used to map foreign memory.
> > > > The mapcache can treat this ram_addr offset like a cookie that we keep
> > > > around to be able to do
> > > > reverse mappings from host pointers into ram_addr space
> > > > (xen_ram_addr_from_mapcache).
> > > >
> > > > The current mapcache implementation works because we've really only
> > > > been using foreign mappings
> > > > on RAMBlocks with offset 0. We're also creating RAM's such that the
> > > > offset into the RAM is also
> > > > the guest physical address, for x86 this is natural since RAM starts
> > > > at zero (for lowmem) but for
> > > > ARM we're creating larger than needed RAM's (GUEST_RAM0_BASE + 
> > > > ram-size) to
> > > > make this assumption true. Anyway, In this series I'm not addressing
> > > > this second assumption.
> > >
> > > Let's see if I understand correctly.
> > >
> > > The ram_addr space is an internal QEMU address space which is different
> > > from the guest physical address space and thus cannot and should not be
> > > used to do foreign mappings (foreign mapping hypercalls take a guest
> > > physical or a real physical address to map). Is that correct?
> > >
> > > If so, then I understand.
> > >
> >
> > Yes, that matches my understanding.
> >
> > >
> > >
> > > > There's a second call in physmem.c to xen_map_cache using the
> > > > block->offset as an address.
> > > > I was considering removing that second call since I can't see how it 
> > > > can work
> > > > (except perhaps in some specific use-case by luck?). Anyway, for now
> > > > I've left it unmodified.
> > >
> > > Yes, that code was written with the assumption that block->offset is an
> > > offset in the guest physical address space and could be used as a guest
> > > physical address. Actually, you might have spotted a real bug.
> > >
> > > The intent was for smaller regions (not the bit RAM region, things like
> > > a ROM region for instance) we could map 

Re: [PATCH v4 14/17] xen: Add xen_mr_is_memory()

[Edgar E. Iglesias]

On Mon, May 6, 2024 at 11:59 AM Philippe Mathieu-Daudé
 wrote:
>
> On 2/5/24 09:26, David Hildenbrand wrote:
> > On 30.04.24 18:49, Edgar E. Iglesias wrote:
> >> From: "Edgar E. Iglesias" 
> >>
> >> Add xen_mr_is_memory() to abstract away tests for the
> >> xen_memory MR.
> >>
> >> Signed-off-by: Edgar E. Iglesias 
> >> ---
> >
> > [...]
> >
> >>   #endif
> >> diff --git a/system/physmem.c b/system/physmem.c
> >> index ad7a8c7d95..1a5ffcba2a 100644
> >> --- a/system/physmem.c
> >> +++ b/system/physmem.c
> >> @@ -2227,7 +2227,7 @@ static void *qemu_ram_ptr_length(RAMBlock
> >> *block, ram_addr_t addr,
> >>* because we don't want to map the entire memory in QEMU.
> >>* In that case just map the requested area.
> >>*/
> >> -if (block->offset == 0) {
> >> +if (xen_mr_is_memory(block->mr)) {
> >>   return xen_map_cache(block->mr, addr, len, lock, lock,
> >>is_write);
> >>   }
> >
> > I'd have moved that into a separate patch, because this is not a simple
> > abstraction here.
>
> Yes please, maybe using Stefano review comment in the description.
>

Thanks, for v5 I've split out this particular change into a separate patch:

softmmu: Replace check for RAMBlock offset 0 with xen_mr_is_memory

    For xen, when checking for the first RAM (xen_memory), use
xen_mr_is_memory() rather than checking for a RAMBlock with
offset 0.

All Xen machines create xen_memory first so this has no
functional change for existing machines.

Signed-off-by: Edgar E. Iglesias 

diff --git a/system/physmem.c b/system/physmem.c
index ad7a8c7d95..1a5ffcba2a 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2227,7 +2227,7 @@ static void *qemu_ram_ptr_length(RAMBlock
*block, ram_addr_t addr,
  * because we don't want to map the entire memory in QEMU.
  * In that case just map the requested area.
  */
-if (block->offset == 0) {
+if (xen_mr_is_memory(block->mr)) {
 return xen_map_cache(block->mr, addr, len, lock, lock,
  is_write);
 }



> >
> > Acked-by: David Hildenbrand 
> >
>

Re: [PATCH v4 16/17] xen: mapcache: Add support for grant mappings

[Edgar E. Iglesias]

On Thu, May 2, 2024 at 9:18 PM Stefano Stabellini
 wrote:
>
> On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" 
> >
> > Add a second mapcache for grant mappings. The mapcache for
> > grants needs to work with XC_PAGE_SIZE granularity since
> > we can't map larger ranges than what has been granted to us.
> >
> > Like with foreign mappings (xen_memory), machines using grants
> > are expected to initialize the xen_grants MR and map it
> > into their address-map accordingly.
> >
> > Signed-off-by: Edgar E. Iglesias 
> > ---
> >  hw/xen/xen-hvm-common.c |  12 ++-
> >  hw/xen/xen-mapcache.c   | 158 +---
> >  include/hw/xen/xen-hvm-common.h |   3 +
> >  include/sysemu/xen.h|   7 ++
> >  4 files changed, 145 insertions(+), 35 deletions(-)
> >
> > diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
> > index 0267b88d26..fdec400491 100644
> > --- a/hw/xen/xen-hvm-common.c
> > +++ b/hw/xen/xen-hvm-common.c
> > @@ -10,12 +10,18 @@
> >  #include "hw/boards.h"
> >  #include "hw/xen/arch_hvm.h"
> >
> > -MemoryRegion xen_memory;
> > +MemoryRegion xen_memory, xen_grants;
> >
> > -/* Check for xen memory.  */
> > +/* Check for any kind of xen memory, foreign mappings or grants.  */
> >  bool xen_mr_is_memory(MemoryRegion *mr)
> >  {
> > -return mr == _memory;
> > +return mr == _memory || mr == _grants;
> > +}
> > +
> > +/* Check specifically for grants.  */
> > +bool xen_mr_is_grants(MemoryRegion *mr)
> > +{
> > +return mr == _grants;
> >  }
> >
> >  void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
> > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > index 1b32d0c003..96cd68e28d 100644
> > --- a/hw/xen/xen-mapcache.c
> > +++ b/hw/xen/xen-mapcache.c
> > @@ -14,6 +14,7 @@
> >
> >  #include 
> >
> > +#include "hw/xen/xen-hvm-common.h"
> >  #include "hw/xen/xen_native.h"
> >  #include "qemu/bitmap.h"
> >
> > @@ -21,6 +22,8 @@
> >  #include "sysemu/xen-mapcache.h"
> >  #include "trace.h"
> >
> > +#include 
> > +#include 
> >
> >  #if HOST_LONG_BITS == 32
> >  #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
> > @@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
> >  unsigned long *valid_mapping;
> >  uint32_t lock;
> >  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> > +#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
> >  uint8_t flags;
> >  hwaddr size;
> >
> > @@ -74,6 +78,8 @@ typedef struct MapCache {
> >  } MapCache;
> >
> >  static MapCache *mapcache;
> > +static MapCache *mapcache_grants;
> > +static xengnttab_handle *xen_region_gnttabdev;
> >
> >  static inline void mapcache_lock(MapCache *mc)
> >  {
> > @@ -132,6 +138,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
> > *opaque)
> >  unsigned long max_mcache_size;
> >  unsigned int bucket_shift;
> >
> > +xen_region_gnttabdev = xengnttab_open(NULL, 0);
> > +if (xen_region_gnttabdev == NULL) {
> > +error_report("mapcache: Failed to open gnttab device");
> > +exit(EXIT_FAILURE);
> > +}
> > +
> >  if (HOST_LONG_BITS == 32) {
> >  bucket_shift = 16;
> >  } else {
> > @@ -160,6 +172,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
> > *opaque)
> >  mapcache = xen_map_cache_init_single(f, opaque,
> >   bucket_shift,
> >   max_mcache_size);
> > +
> > +/*
> > + * Grant mappings must use XC_PAGE_SIZE granularity since we can't
> > + * map anything beyond the number of pages granted to us.
> > + */
> > +mapcache_grants = xen_map_cache_init_single(f, opaque,
> > +XC_PAGE_SHIFT,
> > +max_mcache_size);
> > +
> >  setrlimit(RLIMIT_AS, _as);
> >  }
> >
> > @@ -169,17 +190,25 @@ static void xen_remap_bucket(MapCache *mc,
> >   hwaddr size,
> >   hwaddr address_index,
> >   bool dummy,
> > + bool grant,
> > +   

Re: [PATCH v4 15/17] xen: mapcache: Remove assumption of RAMBlock with 0 offset

[Edgar E. Iglesias]

On Thu, May 2, 2024 at 8:53 PM Stefano Stabellini
 wrote:
>
> +Xenia
>
> On Thu, 2 May 2024, Edgar E. Iglesias wrote:
> > On Wed, May 1, 2024 at 11:24 PM Stefano Stabellini
> >  wrote:
> > >
> > > On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > > > From: "Edgar E. Iglesias" 
> > > >
> > > > The current mapcache assumes that all memory is mapped
> > > > in a single RAM MR (the first one with offset 0). Remove
> > > > this assumption and propagate the offset to the mapcache
> > > > so it can do reverse mappings (from hostptr -> ram_addr).
> > > >
> > > > This is in preparation for adding grant mappings.
> > > >
> > > > Signed-off-by: Edgar E. Iglesias 
> > >
> > >
> > > Looking at xen_remap_bucket, it is only using address_index (without
> > > adding ram_offset) to map foreign memory. From xen_remap_bucket, I would
> > > understand that address_index already includes the ram_offset.
> > >
> > > Meaning that if we want to map foreign mapping at address 0x5000, then
> > > address_index would be 0x5000, even if ram_offset is 0x1000.
> > >
> > > But then looking xen_ram_addr_from_mapcache_single ram_offset is added
> > > to paddr_index to calculate the physical address. So in that case we
> > > would want address_index to be 0x4000 and ram_offset to be 0x1000. But
> > > xen_remap_bucket would have to sum address_index and ram_offset to map
> > > foreign memory.
> > >
> > > So I am a bit confused, did I get it wrong? One more comment below.
> > >
> >
> > Thanks Stefano,
> >
> > I think the confusion is that this ram_addr_offset is not related to
> > guest address-space.
> > It's a QEMU internal thing and it shouldn't be included in the address
> > used to map foreign memory.
> > The mapcache can treat this ram_addr offset like a cookie that we keep
> > around to be able to do
> > reverse mappings from host pointers into ram_addr space
> > (xen_ram_addr_from_mapcache).
> >
> > The current mapcache implementation works because we've really only
> > been using foreign mappings
> > on RAMBlocks with offset 0. We're also creating RAM's such that the
> > offset into the RAM is also
> > the guest physical address, for x86 this is natural since RAM starts
> > at zero (for lowmem) but for
> > ARM we're creating larger than needed RAM's (GUEST_RAM0_BASE + ram-size) to
> > make this assumption true. Anyway, In this series I'm not addressing
> > this second assumption.
>
> Let's see if I understand correctly.
>
> The ram_addr space is an internal QEMU address space which is different
> from the guest physical address space and thus cannot and should not be
> used to do foreign mappings (foreign mapping hypercalls take a guest
> physical or a real physical address to map). Is that correct?
>
> If so, then I understand.
>

Yes, that matches my understanding.

>
>
> > There's a second call in physmem.c to xen_map_cache using the
> > block->offset as an address.
> > I was considering removing that second call since I can't see how it can 
> > work
> > (except perhaps in some specific use-case by luck?). Anyway, for now
> > I've left it unmodified.
>
> Yes, that code was written with the assumption that block->offset is an
> offset in the guest physical address space and could be used as a guest
> physical address. Actually, you might have spotted a real bug.
>
> The intent was for smaller regions (not the bit RAM region, things like
> a ROM region for instance) we could map them in full. So here we were
> trying to map the whole thing from start to finish using block->offset
> as start.
>
>
> > > > ---
> > > >  hw/xen/xen-mapcache.c | 25 ++---
> > > >  include/sysemu/xen-mapcache.h |  2 ++
> > > >  system/physmem.c  |  8 
> > > >  3 files changed, 24 insertions(+), 11 deletions(-)
> > > >
> > > > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > > > index 09b5f36d9c..1b32d0c003 100644
> > > > --- a/hw/xen/xen-mapcache.c
> > > > +++ b/hw/xen/xen-mapcache.c
> > > > @@ -43,6 +43,9 @@ typedef struct MapCacheEntry {
> > > >  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> > > >  uint8_t flags;
> > > >  hwaddr size;
> > > > +
> > > > +/* Keep ram_addr offset for reverse mappings (hostptr -

Re: [PATCH v4 12/17] xen: mapcache: Unmap first entries in buckets

[Edgar E. Iglesias]

On Tue, Apr 30, 2024 at 6:50 PM Edgar E. Iglesias
 wrote:
>
> From: "Edgar E. Iglesias" 
>
> When invalidating memory ranges, if we happen to hit the first
> entry in a bucket we were never unmapping it. This was harmless
> for foreign mappings but now that we're looking to reuse the
> mapcache for transient grant mappings, we must unmap entries
> when invalidated.
>
> Signed-off-by: Edgar E. Iglesias 
> ---
>  hw/xen/xen-mapcache.c | 12 
>  1 file changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 4f98d284dd..0365311788 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -486,18 +486,22 @@ static void 
> xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
>  return;
>  }
>  entry->lock--;
> -if (entry->lock > 0 || pentry == NULL) {
> +if (entry->lock > 0) {
>  return;
>  }
>
> -pentry->next = entry->next;
>  ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
>  if (munmap(entry->vaddr_base, entry->size) != 0) {
>  perror("unmap fails");
>  exit(-1);
>  }
> -g_free(entry->valid_mapping);
> -g_free(entry);
> +if (pentry) {
> +pentry->next = entry->next;
> +g_free(entry->valid_mapping);
> +g_free(entry);
> +} else {
> +memset(entry, 0, sizeof *entry);

I noticed that we're leaking entry->valid_mapping here. I'll fix this for v5.

Cheers,
Edgar


> +}
>  }
>
>  typedef struct XenMapCacheData {
> --
> 2.40.1
>

Re: [PATCH v4 13/17] softmmu: Pass RAM MemoryRegion and is_write xen_map_cache()

[Edgar E. Iglesias]

On Thu, May 2, 2024 at 9:24 AM David Hildenbrand  wrote:
>
> On 30.04.24 18:49, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" 
> >
> > Propagate MR and is_write to xen_map_cache().
>
> I'm pretty sure the patch subject is missing a "to" :)

Thanks David! I'll fix it in v5!

Cheers,
Edgar


>
> > This is in preparation for adding support for grant mappings.
> >
> > No functional change.
> >
>
> Reviewed-by: David Hildenbrand 
> --
> Cheers,
>
> David / dhildenb
>

Re: [PATCH v4 15/17] xen: mapcache: Remove assumption of RAMBlock with 0 offset

[Edgar E. Iglesias]

On Wed, May 1, 2024 at 11:24 PM Stefano Stabellini
 wrote:
>
> On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" 
> >
> > The current mapcache assumes that all memory is mapped
> > in a single RAM MR (the first one with offset 0). Remove
> > this assumption and propagate the offset to the mapcache
> > so it can do reverse mappings (from hostptr -> ram_addr).
> >
> > This is in preparation for adding grant mappings.
> >
> > Signed-off-by: Edgar E. Iglesias 
>
>
> Looking at xen_remap_bucket, it is only using address_index (without
> adding ram_offset) to map foreign memory. From xen_remap_bucket, I would
> understand that address_index already includes the ram_offset.
>
> Meaning that if we want to map foreign mapping at address 0x5000, then
> address_index would be 0x5000, even if ram_offset is 0x1000.
>
> But then looking xen_ram_addr_from_mapcache_single ram_offset is added
> to paddr_index to calculate the physical address. So in that case we
> would want address_index to be 0x4000 and ram_offset to be 0x1000. But
> xen_remap_bucket would have to sum address_index and ram_offset to map
> foreign memory.
>
> So I am a bit confused, did I get it wrong? One more comment below.
>

Thanks Stefano,

I think the confusion is that this ram_addr_offset is not related to
guest address-space.
It's a QEMU internal thing and it shouldn't be included in the address
used to map foreign memory.
The mapcache can treat this ram_addr offset like a cookie that we keep
around to be able to do
reverse mappings from host pointers into ram_addr space
(xen_ram_addr_from_mapcache).

The current mapcache implementation works because we've really only
been using foreign mappings
on RAMBlocks with offset 0. We're also creating RAM's such that the
offset into the RAM is also
the guest physical address, for x86 this is natural since RAM starts
at zero (for lowmem) but for
ARM we're creating larger than needed RAM's (GUEST_RAM0_BASE + ram-size) to
make this assumption true. Anyway, In this series I'm not addressing
this second assumption.

There's a second call in physmem.c to xen_map_cache using the
block->offset as an address.
I was considering removing that second call since I can't see how it can work
(except perhaps in some specific use-case by luck?). Anyway, for now
I've left it unmodified.


>
> > ---
> >  hw/xen/xen-mapcache.c | 25 ++---
> >  include/sysemu/xen-mapcache.h |  2 ++
> >  system/physmem.c  |  8 
> >  3 files changed, 24 insertions(+), 11 deletions(-)
> >
> > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > index 09b5f36d9c..1b32d0c003 100644
> > --- a/hw/xen/xen-mapcache.c
> > +++ b/hw/xen/xen-mapcache.c
> > @@ -43,6 +43,9 @@ typedef struct MapCacheEntry {
> >  #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
> >  uint8_t flags;
> >  hwaddr size;
> > +
> > +/* Keep ram_addr offset for reverse mappings (hostptr -> ram_addr).  */
> > +ram_addr_t ram_offset;
> >  struct MapCacheEntry *next;
> >  } MapCacheEntry;
> >
> > @@ -165,7 +168,8 @@ static void xen_remap_bucket(MapCache *mc,
> >   void *vaddr,
> >   hwaddr size,
> >   hwaddr address_index,
> > - bool dummy)
> > + bool dummy,
> > + ram_addr_t ram_offset)
> >  {
> >  uint8_t *vaddr_base;
> >  xen_pfn_t *pfns;
> > @@ -244,6 +248,7 @@ static void xen_remap_bucket(MapCache *mc,
> >  entry->size = size;
> >  entry->valid_mapping = g_new0(unsigned long,
> >BITS_TO_LONGS(size >> XC_PAGE_SHIFT));
> > +entry->ram_offset = ram_offset;
> >
> >  if (dummy) {
> >  entry->flags |= XEN_MAPCACHE_ENTRY_DUMMY;
> > @@ -264,6 +269,7 @@ static void xen_remap_bucket(MapCache *mc,
> >
> >  static uint8_t *xen_map_cache_unlocked(MapCache *mc,
> > hwaddr phys_addr, hwaddr size,
> > +   ram_addr_t ram_offset,
> > uint8_t lock, bool dma, bool 
> > is_write)
> >  {
> >  MapCacheEntry *entry, *pentry = NULL,
> > @@ -335,14 +341,16 @@ tryagain:
> >  if (!entry) {
> >  entry = g_new0(MapCacheEntry, 1);
> >  pentry->next = entry;
> > -xen_remap_bucket(mc, entry, NULL, cache_size, address_index, 
> > dummy);
> > +xen_re

Re: [PATCH v4 07/17] xen: mapcache: Refactor xen_replace_cache_entry_unlocked

[Edgar E. Iglesias]

On Wed, May 1, 2024 at 10:46 PM Stefano Stabellini
 wrote:
>
> On Tue, 30 Apr 2024, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" 
> >
> > Add MapCache argument to xen_replace_cache_entry_unlocked in
> > preparation for supporting multiple map caches.
> >
> > No functional change.
> >
> > Signed-off-by: Edgar E. Iglesias 
> > ---
> >  hw/xen/xen-mapcache.c | 8 +---
> >  1 file changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > index d2deff70c8..6e758eff94 100644
> > --- a/hw/xen/xen-mapcache.c
> > +++ b/hw/xen/xen-mapcache.c
> > @@ -556,7 +556,8 @@ void xen_invalidate_map_cache(void)
> >  mapcache_unlock(mapcache);
> >  }
> >
> > -static uint8_t *xen_replace_cache_entry_unlocked(hwaddr old_phys_addr,
> > +static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
> > + hwaddr old_phys_addr,
> >   hwaddr new_phys_addr,
> >   hwaddr size)
> >  {
> > @@ -578,7 +579,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(hwaddr 
> > old_phys_addr,
> >  cache_size += MCACHE_BUCKET_SIZE - (cache_size % 
> > MCACHE_BUCKET_SIZE);
> >  }
> >
> > -entry = >entry[address_index % mapcache->nr_buckets];
> > +entry = >entry[address_index % mc->nr_buckets];
> >  while (entry && !(entry->paddr_index == address_index &&
> >entry->size == cache_size)) {
> >  entry = entry->next;
>
> There is still a global mapcache pointer in use in this function:
>
>   xen_remap_bucket(mapcache, entry, entry->vaddr_base,
>


Thanks! I had accidentally put the change to use mc in future patches.
Will fix in v5.

Cheers,
Edgar


>
> > @@ -614,7 +615,8 @@ uint8_t *xen_replace_cache_entry(hwaddr old_phys_addr,
> >  uint8_t *p;
> >
> >  mapcache_lock(mapcache);
> > -p = xen_replace_cache_entry_unlocked(old_phys_addr, new_phys_addr, 
> > size);
> > +p = xen_replace_cache_entry_unlocked(mapcache, old_phys_addr,
> > + new_phys_addr, size);
> >  mapcache_unlock(mapcache);
> >  return p;
> >  }
> > --
> > 2.40.1
> >

[PATCH v4 14/17] xen: Add xen_mr_is_memory()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add xen_mr_is_memory() to abstract away tests for the
xen_memory MR.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-hvm-common.c | 8 +++-
 include/sysemu/xen.h| 8 
 system/physmem.c| 2 +-
 3 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 1627da7398..0267b88d26 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -12,6 +12,12 @@
 
 MemoryRegion xen_memory;
 
+/* Check for xen memory.  */
+bool xen_mr_is_memory(MemoryRegion *mr)
+{
+return mr == _memory;
+}
+
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
Error **errp)
 {
@@ -28,7 +34,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, 
MemoryRegion *mr,
 return;
 }
 
-if (mr == _memory) {
+if (xen_mr_is_memory(mr)) {
 return;
 }
 
diff --git a/include/sysemu/xen.h b/include/sysemu/xen.h
index 754ec2e6cb..dc72f83bcb 100644
--- a/include/sysemu/xen.h
+++ b/include/sysemu/xen.h
@@ -34,6 +34,8 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t 
length);
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
struct MemoryRegion *mr, Error **errp);
 
+bool xen_mr_is_memory(MemoryRegion *mr);
+
 #else /* !CONFIG_XEN_IS_POSSIBLE */
 
 #define xen_enabled() 0
@@ -47,6 +49,12 @@ static inline void xen_ram_alloc(ram_addr_t ram_addr, 
ram_addr_t size,
 g_assert_not_reached();
 }
 
+static inline bool xen_mr_is_memory(MemoryRegion *mr)
+{
+g_assert_not_reached();
+return false;
+}
+
 #endif /* CONFIG_XEN_IS_POSSIBLE */
 
 #endif
diff --git a/system/physmem.c b/system/physmem.c
index ad7a8c7d95..1a5ffcba2a 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2227,7 +2227,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * because we don't want to map the entire memory in QEMU.
  * In that case just map the requested area.
  */
-if (block->offset == 0) {
+if (xen_mr_is_memory(block->mr)) {
 return xen_map_cache(block->mr, addr, len, lock, lock,
  is_write);
 }
-- 
2.40.1

[PATCH v4 13/17] softmmu: Pass RAM MemoryRegion and is_write xen_map_cache()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Propagate MR and is_write to xen_map_cache().
This is in preparation for adding support for grant mappings.

No functional change.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 10 ++
 include/sysemu/xen-mapcache.h | 11 +++
 system/physmem.c  | 31 +++
 3 files changed, 36 insertions(+), 16 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 0365311788..09b5f36d9c 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -264,7 +264,7 @@ static void xen_remap_bucket(MapCache *mc,
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
hwaddr phys_addr, hwaddr size,
-   uint8_t lock, bool dma)
+   uint8_t lock, bool dma, bool is_write)
 {
 MapCacheEntry *entry, *pentry = NULL,
   *free_entry = NULL, *free_pentry = NULL;
@@ -387,13 +387,15 @@ tryagain:
 return mc->last_entry->vaddr_base + address_offset;
 }
 
-uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
-   uint8_t lock, bool dma)
+uint8_t *xen_map_cache(MemoryRegion *mr,
+   hwaddr phys_addr, hwaddr size,
+   uint8_t lock, bool dma,
+   bool is_write)
 {
 uint8_t *p;
 
 mapcache_lock(mapcache);
-p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma);
+p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
 mapcache_unlock(mapcache);
 return p;
 }
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index 10c2e3082a..1ec9e66752 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -18,8 +18,9 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
 
 void xen_map_cache_init(phys_offset_to_gaddr_t f,
 void *opaque);
-uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
-   uint8_t lock, bool dma);
+uint8_t *xen_map_cache(MemoryRegion *mr, hwaddr phys_addr, hwaddr size,
+   uint8_t lock, bool dma,
+   bool is_write);
 ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
 void xen_invalidate_map_cache_entry(uint8_t *buffer);
 void xen_invalidate_map_cache(void);
@@ -33,10 +34,12 @@ static inline void 
xen_map_cache_init(phys_offset_to_gaddr_t f,
 {
 }
 
-static inline uint8_t *xen_map_cache(hwaddr phys_addr,
+static inline uint8_t *xen_map_cache(MemoryRegion *mr,
+ hwaddr phys_addr,
  hwaddr size,
  uint8_t lock,
- bool dma)
+ bool dma,
+ bool is_write)
 {
 abort();
 }
diff --git a/system/physmem.c b/system/physmem.c
index f114b972a5..ad7a8c7d95 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2190,11 +2190,22 @@ void qemu_ram_remap(ram_addr_t addr, ram_addr_t length)
 
 /*
  * Return a host pointer to guest's ram.
+ * For Xen, foreign mappings get created if they don't already exist.
+ *
+ * @block: block for the RAM to lookup (optional and may be NULL).
+ * @addr: address within the memory region.
+ * @size: pointer to requested size (optional and may be NULL).
+ *size may get modified and return a value smaller than
+ *what was requested.
+ * @lock: wether to lock the mapping in xen-mapcache until invalidated.
+ * @is_write: hint wether to map RW or RO in the xen-mapcache.
+ *(optional and may always be set to true).
  *
  * Called within RCU critical section.
  */
 static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
- hwaddr *size, bool lock)
+ hwaddr *size, bool lock,
+ bool is_write)
 {
 hwaddr len = 0;
 
@@ -2217,10 +2228,13 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * In that case just map the requested area.
  */
 if (block->offset == 0) {
-return xen_map_cache(addr, len, lock, lock);
+return xen_map_cache(block->mr, addr, len, lock, lock,
+ is_write);
 }
 
-block->host = xen_map_cache(block->offset, block->max_length, 1, lock);
+block->host = xen_map_cache(block->mr, block->offset,
+block->max_length, 1,
+lock, is_write);
 }
 
 return ramblock_ptr(block, addr);
@@ -2236,7 +2250,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  */
 void *qemu_map_ram_ptr(RAMBlock *ram_block, ram_addr_t addr)
 {
-return qemu_ram_ptr_length(ram_block, addr, NULL, false)

[PATCH v4 04/17] xen: mapcache: Refactor xen_map_cache for multi-instance

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Make xen_map_cache take a MapCache as argument. This is in
prepaparation to support multiple map caches.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 35 ++-
 1 file changed, 18 insertions(+), 17 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 3f11562075..896021d86f 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -240,7 +240,8 @@ static void xen_remap_bucket(MapCacheEntry *entry,
 g_free(err);
 }
 
-static uint8_t *xen_map_cache_unlocked(hwaddr phys_addr, hwaddr size,
+static uint8_t *xen_map_cache_unlocked(MapCache *mc,
+   hwaddr phys_addr, hwaddr size,
uint8_t lock, bool dma)
 {
 MapCacheEntry *entry, *pentry = NULL,
@@ -269,16 +270,16 @@ tryagain:
 test_bit_size = XC_PAGE_SIZE;
 }
 
-if (mapcache->last_entry != NULL &&
-mapcache->last_entry->paddr_index == address_index &&
+if (mc->last_entry != NULL &&
+mc->last_entry->paddr_index == address_index &&
 !lock && !size &&
 test_bits(address_offset >> XC_PAGE_SHIFT,
   test_bit_size >> XC_PAGE_SHIFT,
-  mapcache->last_entry->valid_mapping)) {
+  mc->last_entry->valid_mapping)) {
 trace_xen_map_cache_return(
-mapcache->last_entry->vaddr_base + address_offset
+mc->last_entry->vaddr_base + address_offset
 );
-return mapcache->last_entry->vaddr_base + address_offset;
+return mc->last_entry->vaddr_base + address_offset;
 }
 
 /* size is always a multiple of MCACHE_BUCKET_SIZE */
@@ -291,7 +292,7 @@ tryagain:
 cache_size = MCACHE_BUCKET_SIZE;
 }
 
-entry = >entry[address_index % mapcache->nr_buckets];
+entry = >entry[address_index % mc->nr_buckets];
 
 while (entry && (lock || entry->lock) && entry->vaddr_base &&
 (entry->paddr_index != address_index || entry->size != cache_size 
||
@@ -326,10 +327,10 @@ tryagain:
 if(!test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
-mapcache->last_entry = NULL;
+mc->last_entry = NULL;
 #ifdef XEN_COMPAT_PHYSMAP
-if (!translated && mapcache->phys_offset_to_gaddr) {
-phys_addr = mapcache->phys_offset_to_gaddr(phys_addr, size);
+if (!translated && mc->phys_offset_to_gaddr) {
+phys_addr = mc->phys_offset_to_gaddr(phys_addr, size);
 translated = true;
 goto tryagain;
 }
@@ -342,7 +343,7 @@ tryagain:
 return NULL;
 }
 
-mapcache->last_entry = entry;
+mc->last_entry = entry;
 if (lock) {
 MapCacheRev *reventry = g_new0(MapCacheRev, 1);
 entry->lock++;
@@ -352,16 +353,16 @@ tryagain:
 abort();
 }
 reventry->dma = dma;
-reventry->vaddr_req = mapcache->last_entry->vaddr_base + 
address_offset;
-reventry->paddr_index = mapcache->last_entry->paddr_index;
+reventry->vaddr_req = mc->last_entry->vaddr_base + address_offset;
+reventry->paddr_index = mc->last_entry->paddr_index;
 reventry->size = entry->size;
-QTAILQ_INSERT_HEAD(>locked_entries, reventry, next);
+QTAILQ_INSERT_HEAD(>locked_entries, reventry, next);
 }
 
 trace_xen_map_cache_return(
-mapcache->last_entry->vaddr_base + address_offset
+mc->last_entry->vaddr_base + address_offset
 );
-return mapcache->last_entry->vaddr_base + address_offset;
+return mc->last_entry->vaddr_base + address_offset;
 }
 
 uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
@@ -370,7 +371,7 @@ uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
 uint8_t *p;
 
 mapcache_lock(mapcache);
-p = xen_map_cache_unlocked(phys_addr, size, lock, dma);
+p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma);
 mapcache_unlock(mapcache);
 return p;
 }
-- 
2.40.1

[PATCH v4 02/17] xen: let xen_ram_addr_from_mapcache() return -1 in case of not found entry

[Edgar E. Iglesias]

From: Juergen Gross 

Today xen_ram_addr_from_mapcache() will either abort() or return 0 in
case it can't find a matching entry for a pointer value. Both cases
are bad, so change that to return an invalid address instead.

Signed-off-by: Juergen Gross 
Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Alex Bennée 
Reviewed-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 11 +++
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 7f59080ba7..b7cefb78f7 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -394,13 +394,8 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 }
 }
 if (!found) {
-trace_xen_ram_addr_from_mapcache_not_found(ptr);
-QTAILQ_FOREACH(reventry, >locked_entries, next) {
-trace_xen_ram_addr_from_mapcache_found(reventry->paddr_index,
-   reventry->vaddr_req);
-}
-abort();
-return 0;
+mapcache_unlock();
+return RAM_ADDR_INVALID;
 }
 
 entry = >entry[paddr_index % mapcache->nr_buckets];
@@ -409,7 +404,7 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 }
 if (!entry) {
 trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
-raddr = 0;
+raddr = RAM_ADDR_INVALID;
 } else {
 raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
  ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
-- 
2.40.1

[PATCH v4 11/17] xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Make MCACHE_BUCKET_SHIFT runtime configurable per cache instance.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 52 ++-
 1 file changed, 31 insertions(+), 21 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 72a7e25e3e..4f98d284dd 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -23,13 +23,10 @@
 
 
 #if HOST_LONG_BITS == 32
-#  define MCACHE_BUCKET_SHIFT 16
 #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
 #else
-#  define MCACHE_BUCKET_SHIFT 20
 #  define MCACHE_MAX_SIZE (1UL<<35) /* 32GB Cap */
 #endif
-#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
 
 /* This is the size of the virtual address space reserve to QEMU that will not
  * be use by MapCache.
@@ -65,7 +62,8 @@ typedef struct MapCache {
 /* For most cases (>99.9%), the page address is the same. */
 MapCacheEntry *last_entry;
 unsigned long max_mcache_size;
-unsigned int mcache_bucket_shift;
+unsigned int bucket_shift;
+unsigned long bucket_size;
 
 phys_offset_to_gaddr_t phys_offset_to_gaddr;
 QemuMutex lock;
@@ -95,6 +93,7 @@ static inline int test_bits(int nr, int size, const unsigned 
long *addr)
 
 static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
void *opaque,
+   unsigned int bucket_shift,
unsigned long max_size)
 {
 unsigned long size;
@@ -108,12 +107,14 @@ static MapCache 
*xen_map_cache_init_single(phys_offset_to_gaddr_t f,
 
 QTAILQ_INIT(>locked_entries);
 
+mc->bucket_shift = bucket_shift;
+mc->bucket_size = 1UL << bucket_shift;
 mc->max_mcache_size = max_size;
 
 mc->nr_buckets =
 (((mc->max_mcache_size >> XC_PAGE_SHIFT) +
-  (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
- (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
+  (1UL << (bucket_shift - XC_PAGE_SHIFT)) - 1) >>
+ (bucket_shift - XC_PAGE_SHIFT));
 
 size = mc->nr_buckets * sizeof(MapCacheEntry);
 size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
@@ -126,6 +127,13 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 {
 struct rlimit rlimit_as;
 unsigned long max_mcache_size;
+unsigned int bucket_shift;
+
+if (HOST_LONG_BITS == 32) {
+bucket_shift = 16;
+} else {
+bucket_shift = 20;
+}
 
 if (geteuid() == 0) {
 rlimit_as.rlim_cur = RLIM_INFINITY;
@@ -146,7 +154,9 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 }
 }
 
-mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
+mapcache = xen_map_cache_init_single(f, opaque,
+ bucket_shift,
+ max_mcache_size);
 setrlimit(RLIMIT_AS, _as);
 }
 
@@ -195,7 +205,7 @@ static void xen_remap_bucket(MapCache *mc,
 entry->valid_mapping = NULL;
 
 for (i = 0; i < nb_pfn; i++) {
-pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
+pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
 }
 
 /*
@@ -266,8 +276,8 @@ static uint8_t *xen_map_cache_unlocked(MapCache *mc,
 bool dummy = false;
 
 tryagain:
-address_index  = phys_addr >> MCACHE_BUCKET_SHIFT;
-address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
+address_index  = phys_addr >> mc->bucket_shift;
+address_offset = phys_addr & (mc->bucket_size - 1);
 
 trace_xen_map_cache(phys_addr);
 
@@ -294,14 +304,14 @@ tryagain:
 return mc->last_entry->vaddr_base + address_offset;
 }
 
-/* size is always a multiple of MCACHE_BUCKET_SIZE */
+/* size is always a multiple of mc->bucket_size */
 if (size) {
 cache_size = size + address_offset;
-if (cache_size % MCACHE_BUCKET_SIZE) {
-cache_size += MCACHE_BUCKET_SIZE - (cache_size % 
MCACHE_BUCKET_SIZE);
+if (cache_size % mc->bucket_size) {
+cache_size += mc->bucket_size - (cache_size % mc->bucket_size);
 }
 } else {
-cache_size = MCACHE_BUCKET_SIZE;
+cache_size = mc->bucket_size;
 }
 
 entry = >entry[address_index % mc->nr_buckets];
@@ -419,7 +429,7 @@ static ram_addr_t 
xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
 raddr = RAM_ADDR_INVALID;
 } else {
-raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
+raddr = (reventry->paddr_index << mc->bucket_shift) +
  ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
 }
 mapcache_unlock(mc);
@@ 

[PATCH v4 01/17] softmmu: let qemu_map_ram_ptr() use qemu_ram_ptr_length()

[Edgar E. Iglesias]

From: Juergen Gross 

qemu_map_ram_ptr() and qemu_ram_ptr_length() share quite some code, so
modify qemu_ram_ptr_length() a little bit and use it for
qemu_map_ram_ptr(), too.

Signed-off-by: Juergen Gross 
Signed-off-by: Vikram Garhwal 
Signed-off-by: Edgar E. Iglesias 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Alex Bennée 
Reviewed-by: Edgar E. Iglesias 
---
 system/physmem.c | 56 
 1 file changed, 23 insertions(+), 33 deletions(-)

diff --git a/system/physmem.c b/system/physmem.c
index 1a81c226ba..f114b972a5 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2188,43 +2188,17 @@ void qemu_ram_remap(ram_addr_t addr, ram_addr_t length)
 }
 #endif /* !_WIN32 */
 
-/* Return a host pointer to ram allocated with qemu_ram_alloc.
- * This should not be used for general purpose DMA.  Use address_space_map
- * or address_space_rw instead. For local memory (e.g. video ram) that the
- * device owns, use memory_region_get_ram_ptr.
- *
- * Called within RCU critical section.
- */
-void *qemu_map_ram_ptr(RAMBlock *block, ram_addr_t addr)
-{
-if (block == NULL) {
-block = qemu_get_ram_block(addr);
-addr -= block->offset;
-}
-
-if (xen_enabled() && block->host == NULL) {
-/* We need to check if the requested address is in the RAM
- * because we don't want to map the entire memory in QEMU.
- * In that case just map until the end of the page.
- */
-if (block->offset == 0) {
-return xen_map_cache(addr, 0, 0, false);
-}
-
-block->host = xen_map_cache(block->offset, block->max_length, 1, 
false);
-}
-return ramblock_ptr(block, addr);
-}
-
-/* Return a host pointer to guest's ram. Similar to qemu_map_ram_ptr
- * but takes a size argument.
+/*
+ * Return a host pointer to guest's ram.
  *
  * Called within RCU critical section.
  */
 static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
  hwaddr *size, bool lock)
 {
-if (*size == 0) {
+hwaddr len = 0;
+
+if (size && *size == 0) {
 return NULL;
 }
 
@@ -2232,7 +2206,10 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
 block = qemu_get_ram_block(addr);
 addr -= block->offset;
 }
-*size = MIN(*size, block->max_length - addr);
+if (size) {
+*size = MIN(*size, block->max_length - addr);
+len = *size;
+}
 
 if (xen_enabled() && block->host == NULL) {
 /* We need to check if the requested address is in the RAM
@@ -2240,7 +2217,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
  * In that case just map the requested area.
  */
 if (block->offset == 0) {
-return xen_map_cache(addr, *size, lock, lock);
+return xen_map_cache(addr, len, lock, lock);
 }
 
 block->host = xen_map_cache(block->offset, block->max_length, 1, lock);
@@ -2249,6 +2226,19 @@ static void *qemu_ram_ptr_length(RAMBlock *block, 
ram_addr_t addr,
 return ramblock_ptr(block, addr);
 }
 
+/*
+ * Return a host pointer to ram allocated with qemu_ram_alloc.
+ * This should not be used for general purpose DMA.  Use address_space_map
+ * or address_space_rw instead. For local memory (e.g. video ram) that the
+ * device owns, use memory_region_get_ram_ptr.
+ *
+ * Called within RCU critical section.
+ */
+void *qemu_map_ram_ptr(RAMBlock *ram_block, ram_addr_t addr)
+{
+return qemu_ram_ptr_length(ram_block, addr, NULL, false);
+}
+
 /* Return the offset of a hostpointer within a ramblock */
 ram_addr_t qemu_ram_block_host_offset(RAMBlock *rb, void *host)
 {
-- 
2.40.1

[PATCH v4 12/17] xen: mapcache: Unmap first entries in buckets

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

When invalidating memory ranges, if we happen to hit the first
entry in a bucket we were never unmapping it. This was harmless
for foreign mappings but now that we're looking to reuse the
mapcache for transient grant mappings, we must unmap entries
when invalidated.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 4f98d284dd..0365311788 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -486,18 +486,22 @@ static void 
xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
 return;
 }
 entry->lock--;
-if (entry->lock > 0 || pentry == NULL) {
+if (entry->lock > 0) {
 return;
 }
 
-pentry->next = entry->next;
 ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size);
 if (munmap(entry->vaddr_base, entry->size) != 0) {
 perror("unmap fails");
 exit(-1);
 }
-g_free(entry->valid_mapping);
-g_free(entry);
+if (pentry) {
+pentry->next = entry->next;
+g_free(entry->valid_mapping);
+g_free(entry);
+} else {
+memset(entry, 0, sizeof *entry);
+}
 }
 
 typedef struct XenMapCacheData {
-- 
2.40.1

[PATCH v4 03/17] xen: mapcache: Refactor lock functions for multi-instance

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Make the lock functions take MapCache * as argument. This is
in preparation for supporting multiple caches.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 34 +-
 1 file changed, 17 insertions(+), 17 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index b7cefb78f7..3f11562075 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -74,14 +74,14 @@ typedef struct MapCache {
 
 static MapCache *mapcache;
 
-static inline void mapcache_lock(void)
+static inline void mapcache_lock(MapCache *mc)
 {
-qemu_mutex_lock(>lock);
+qemu_mutex_lock(>lock);
 }
 
-static inline void mapcache_unlock(void)
+static inline void mapcache_unlock(MapCache *mc)
 {
-qemu_mutex_unlock(>lock);
+qemu_mutex_unlock(>lock);
 }
 
 static inline int test_bits(int nr, int size, const unsigned long *addr)
@@ -369,9 +369,9 @@ uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
 {
 uint8_t *p;
 
-mapcache_lock();
+mapcache_lock(mapcache);
 p = xen_map_cache_unlocked(phys_addr, size, lock, dma);
-mapcache_unlock();
+mapcache_unlock(mapcache);
 return p;
 }
 
@@ -384,7 +384,7 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 ram_addr_t raddr;
 int found = 0;
 
-mapcache_lock();
+mapcache_lock(mapcache);
 QTAILQ_FOREACH(reventry, >locked_entries, next) {
 if (reventry->vaddr_req == ptr) {
 paddr_index = reventry->paddr_index;
@@ -394,7 +394,7 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 }
 }
 if (!found) {
-mapcache_unlock();
+mapcache_unlock(mapcache);
 return RAM_ADDR_INVALID;
 }
 
@@ -409,7 +409,7 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
  ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
 }
-mapcache_unlock();
+mapcache_unlock(mapcache);
 return raddr;
 }
 
@@ -480,9 +480,9 @@ static void xen_invalidate_map_cache_entry_bh(void *opaque)
 {
 XenMapCacheData *data = opaque;
 
-mapcache_lock();
+mapcache_lock(mapcache);
 xen_invalidate_map_cache_entry_unlocked(data->buffer);
-mapcache_unlock();
+mapcache_unlock(mapcache);
 
 aio_co_wake(data->co);
 }
@@ -498,9 +498,9 @@ void coroutine_mixed_fn 
xen_invalidate_map_cache_entry(uint8_t *buffer)
 xen_invalidate_map_cache_entry_bh, );
 qemu_coroutine_yield();
 } else {
-mapcache_lock();
+mapcache_lock(mapcache);
 xen_invalidate_map_cache_entry_unlocked(buffer);
-mapcache_unlock();
+mapcache_unlock(mapcache);
 }
 }
 
@@ -512,7 +512,7 @@ void xen_invalidate_map_cache(void)
 /* Flush pending AIO before destroying the mapcache */
 bdrv_drain_all();
 
-mapcache_lock();
+mapcache_lock(mapcache);
 
 QTAILQ_FOREACH(reventry, >locked_entries, next) {
 if (!reventry->dma) {
@@ -546,7 +546,7 @@ void xen_invalidate_map_cache(void)
 
 mapcache->last_entry = NULL;
 
-mapcache_unlock();
+mapcache_unlock(mapcache);
 }
 
 static uint8_t *xen_replace_cache_entry_unlocked(hwaddr old_phys_addr,
@@ -606,8 +606,8 @@ uint8_t *xen_replace_cache_entry(hwaddr old_phys_addr,
 {
 uint8_t *p;
 
-mapcache_lock();
+mapcache_lock(mapcache);
 p = xen_replace_cache_entry_unlocked(old_phys_addr, new_phys_addr, size);
-mapcache_unlock();
+mapcache_unlock(mapcache);
 return p;
 }
-- 
2.40.1

[PATCH v4 09/17] xen: mapcache: Break out xen_invalidate_map_cache_single()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Break out xen_invalidate_map_cache_single().

No functional changes.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 25 +++--
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 34454da2f6..dd08cd296b 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -512,17 +512,14 @@ void coroutine_mixed_fn 
xen_invalidate_map_cache_entry(uint8_t *buffer)
 }
 }
 
-void xen_invalidate_map_cache(void)
+static void xen_invalidate_map_cache_single(MapCache *mc)
 {
 unsigned long i;
 MapCacheRev *reventry;
 
-/* Flush pending AIO before destroying the mapcache */
-bdrv_drain_all();
-
-mapcache_lock(mapcache);
+mapcache_lock(mc);
 
-QTAILQ_FOREACH(reventry, >locked_entries, next) {
+QTAILQ_FOREACH(reventry, >locked_entries, next) {
 if (!reventry->dma) {
 continue;
 }
@@ -530,8 +527,8 @@ void xen_invalidate_map_cache(void)
reventry->vaddr_req);
 }
 
-for (i = 0; i < mapcache->nr_buckets; i++) {
-MapCacheEntry *entry = >entry[i];
+for (i = 0; i < mc->nr_buckets; i++) {
+MapCacheEntry *entry = >entry[i];
 
 if (entry->vaddr_base == NULL) {
 continue;
@@ -552,9 +549,17 @@ void xen_invalidate_map_cache(void)
 entry->valid_mapping = NULL;
 }
 
-mapcache->last_entry = NULL;
+mc->last_entry = NULL;
 
-mapcache_unlock(mapcache);
+mapcache_unlock(mc);
+}
+
+void xen_invalidate_map_cache(void)
+{
+/* Flush pending AIO before destroying the mapcache */
+bdrv_drain_all();
+
+xen_invalidate_map_cache_single(mapcache);
 }
 
 static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
-- 
2.40.1

[PATCH v4 00/17] xen: Support grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Hi,

This is a follow-up on Vikrams v3:
http://next.patchew.org/QEMU/20240227223501.28475-1-vikram.garh...@amd.com/

Grant mappings are a mechanism in Xen for guests to grant each other
permissions to map and share pages. These grants can be temporary
so both map and unmaps must be respected. See here for more info:
https://github.com/xen-project/xen/blob/master/docs/misc/grant-tables.txt

Currently, the primary use-case for this is with QEMU's VirtIO backends.
Grant mappings will only work with models that use the address_space_map/unmap
interfaces, any other access will fail with appropriate error messages.

In response to feedback we got on v3, this version switches approach
from adding new MemoryRegion types and map/unmap hooks to instead reusing
the existing xen_map_cache() hooks (with extensions). Almost all of the
changes are now contained to the Xen modules.

This approach also refactors the mapcache to support multiple instances
(one for existing foreign mappings and another for grant mappings).

Patch 1 - 10 are refactorings with minimal functional changes.
Patch 3 - 10 could possibly get squashed into one but I've left them
separate to make them easier to review.

I've only enabled grants for the ARM PVH machine since that is what
I can currently test on.

Cheers,
Edgar

ChangeLog:

v3 -> v4:
* Major changes.
* Reuse existing xen_map_cache hooks.
* Reuse existing map-cache for both foreign and grant mappings.
* Only enable grants for the ARM PVH machine (removed i386).

v2 -> v3:
* Drop patch 1/7. This was done because device unplug is an x86-only case.
* Add missing qemu_mutex_unlock() before return.

v1 -> v2:
* Split patch 2/7 to keep phymem.c changes in a separate.
* In patch "xen: add map and unmap callbacks for grant" add check for total
  allowed grant < XEN_MAX_VIRTIO_GRANTS.
* Fix formatting issues and re-based with master latest.

Edgar E. Iglesias (15):
  xen: mapcache: Refactor lock functions for multi-instance
  xen: mapcache: Refactor xen_map_cache for multi-instance
  xen: mapcache: Refactor xen_remap_bucket for multi-instance
  xen: mapcache: Break out xen_ram_addr_from_mapcache_single
  xen: mapcache: Refactor xen_replace_cache_entry_unlocked
  xen: mapcache: Refactor xen_invalidate_map_cache_entry_unlocked
  xen: mapcache: Break out xen_invalidate_map_cache_single()
  xen: mapcache: Break out xen_map_cache_init_single()
  xen: mapcache: Make MCACHE_BUCKET_SHIFT runtime configurable
  xen: mapcache: Unmap first entries in buckets
  softmmu: Pass RAM MemoryRegion and is_write xen_map_cache()
  xen: Add xen_mr_is_memory()
  xen: mapcache: Remove assumption of RAMBlock with 0 offset
  xen: mapcache: Add support for grant mappings
  hw/arm: xen: Enable use of grant mappings

Juergen Gross (2):
  softmmu: let qemu_map_ram_ptr() use qemu_ram_ptr_length()
  xen: let xen_ram_addr_from_mapcache() return -1 in case of not found
entry

 hw/arm/xen_arm.c|   5 +
 hw/xen/xen-hvm-common.c |  16 +-
 hw/xen/xen-mapcache.c   | 408 +---
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen-mapcache.h   |  13 +-
 include/sysemu/xen.h|  15 ++
 system/physmem.c|  83 ---
 7 files changed, 365 insertions(+), 178 deletions(-)

-- 
2.40.1

[PATCH v4 05/17] xen: mapcache: Refactor xen_remap_bucket for multi-instance

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add MapCache argument to xen_remap_bucket in preparation
to support multiple map caches.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 896021d86f..326a9b61ca 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -139,7 +139,8 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 mapcache->entry = g_malloc0(size);
 }
 
-static void xen_remap_bucket(MapCacheEntry *entry,
+static void xen_remap_bucket(MapCache *mc,
+ MapCacheEntry *entry,
  void *vaddr,
  hwaddr size,
  hwaddr address_index,
@@ -313,14 +314,14 @@ tryagain:
 if (!entry) {
 entry = g_new0(MapCacheEntry, 1);
 pentry->next = entry;
-xen_remap_bucket(entry, NULL, cache_size, address_index, dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
 } else if (!entry->lock) {
 if (!entry->vaddr_base || entry->paddr_index != address_index ||
 entry->size != cache_size ||
 !test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
-xen_remap_bucket(entry, NULL, cache_size, address_index, dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, 
dummy);
 }
 }
 
@@ -587,7 +588,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(hwaddr 
old_phys_addr,
 
 trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
-xen_remap_bucket(entry, entry->vaddr_base,
+xen_remap_bucket(mapcache, entry, entry->vaddr_base,
  cache_size, address_index, false);
 if (!test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
-- 
2.40.1

[PATCH v4 17/17] hw/arm: xen: Enable use of grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Signed-off-by: Edgar E. Iglesias 
---
 hw/arm/xen_arm.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/hw/arm/xen_arm.c b/hw/arm/xen_arm.c
index 15fa7dfa84..6fad829ede 100644
--- a/hw/arm/xen_arm.c
+++ b/hw/arm/xen_arm.c
@@ -125,6 +125,11 @@ static void xen_init_ram(MachineState *machine)
  GUEST_RAM1_BASE, ram_size[1]);
 memory_region_add_subregion(sysmem, GUEST_RAM1_BASE, _hi);
 }
+
+/* Setup support for grants.  */
+memory_region_init_ram(_grants, NULL, "xen.grants", block_len,
+   _fatal);
+memory_region_add_subregion(sysmem, XEN_GRANT_ADDR_OFF, _grants);
 }
 
 void arch_handle_ioreq(XenIOState *state, ioreq_t *req)
-- 
2.40.1

[PATCH v4 08/17] xen: mapcache: Refactor xen_invalidate_map_cache_entry_unlocked

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add MapCache argument to xen_invalidate_map_cache_entry_unlocked.
This is in preparation for supporting multiple map caches.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 21 +++--
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 6e758eff94..34454da2f6 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -420,7 +420,8 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 return xen_ram_addr_from_mapcache_single(mapcache, ptr);
 }
 
-static void xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
+static void xen_invalidate_map_cache_entry_unlocked(MapCache *mc,
+uint8_t *buffer)
 {
 MapCacheEntry *entry = NULL, *pentry = NULL;
 MapCacheRev *reventry;
@@ -428,7 +429,7 @@ static void xen_invalidate_map_cache_entry_unlocked(uint8_t 
*buffer)
 hwaddr size;
 int found = 0;
 
-QTAILQ_FOREACH(reventry, >locked_entries, next) {
+QTAILQ_FOREACH(reventry, >locked_entries, next) {
 if (reventry->vaddr_req == buffer) {
 paddr_index = reventry->paddr_index;
 size = reventry->size;
@@ -438,7 +439,7 @@ static void xen_invalidate_map_cache_entry_unlocked(uint8_t 
*buffer)
 }
 if (!found) {
 trace_xen_invalidate_map_cache_entry_unlocked_not_found(buffer);
-QTAILQ_FOREACH(reventry, >locked_entries, next) {
+QTAILQ_FOREACH(reventry, >locked_entries, next) {
 trace_xen_invalidate_map_cache_entry_unlocked_found(
 reventry->paddr_index,
 reventry->vaddr_req
@@ -446,15 +447,15 @@ static void 
xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
 }
 return;
 }
-QTAILQ_REMOVE(>locked_entries, reventry, next);
+QTAILQ_REMOVE(>locked_entries, reventry, next);
 g_free(reventry);
 
-if (mapcache->last_entry != NULL &&
-mapcache->last_entry->paddr_index == paddr_index) {
-mapcache->last_entry = NULL;
+if (mc->last_entry != NULL &&
+mc->last_entry->paddr_index == paddr_index) {
+mc->last_entry = NULL;
 }
 
-entry = >entry[paddr_index % mapcache->nr_buckets];
+entry = >entry[paddr_index % mc->nr_buckets];
 while (entry && (entry->paddr_index != paddr_index || entry->size != 
size)) {
 pentry = entry;
 entry = entry->next;
@@ -488,7 +489,7 @@ static void xen_invalidate_map_cache_entry_bh(void *opaque)
 XenMapCacheData *data = opaque;
 
 mapcache_lock(mapcache);
-xen_invalidate_map_cache_entry_unlocked(data->buffer);
+xen_invalidate_map_cache_entry_unlocked(mapcache, data->buffer);
 mapcache_unlock(mapcache);
 
 aio_co_wake(data->co);
@@ -506,7 +507,7 @@ void coroutine_mixed_fn 
xen_invalidate_map_cache_entry(uint8_t *buffer)
 qemu_coroutine_yield();
 } else {
 mapcache_lock(mapcache);
-xen_invalidate_map_cache_entry_unlocked(buffer);
+xen_invalidate_map_cache_entry_unlocked(mapcache, buffer);
 mapcache_unlock(mapcache);
 }
 }
-- 
2.40.1

[PATCH v4 16/17] xen: mapcache: Add support for grant mappings

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add a second mapcache for grant mappings. The mapcache for
grants needs to work with XC_PAGE_SIZE granularity since
we can't map larger ranges than what has been granted to us.

Like with foreign mappings (xen_memory), machines using grants
are expected to initialize the xen_grants MR and map it
into their address-map accordingly.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-hvm-common.c |  12 ++-
 hw/xen/xen-mapcache.c   | 158 +---
 include/hw/xen/xen-hvm-common.h |   3 +
 include/sysemu/xen.h|   7 ++
 4 files changed, 145 insertions(+), 35 deletions(-)

diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 0267b88d26..fdec400491 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -10,12 +10,18 @@
 #include "hw/boards.h"
 #include "hw/xen/arch_hvm.h"
 
-MemoryRegion xen_memory;
+MemoryRegion xen_memory, xen_grants;
 
-/* Check for xen memory.  */
+/* Check for any kind of xen memory, foreign mappings or grants.  */
 bool xen_mr_is_memory(MemoryRegion *mr)
 {
-return mr == _memory;
+return mr == _memory || mr == _grants;
+}
+
+/* Check specifically for grants.  */
+bool xen_mr_is_grants(MemoryRegion *mr)
+{
+return mr == _grants;
 }
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 1b32d0c003..96cd68e28d 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -14,6 +14,7 @@
 
 #include 
 
+#include "hw/xen/xen-hvm-common.h"
 #include "hw/xen/xen_native.h"
 #include "qemu/bitmap.h"
 
@@ -21,6 +22,8 @@
 #include "sysemu/xen-mapcache.h"
 #include "trace.h"
 
+#include 
+#include 
 
 #if HOST_LONG_BITS == 32
 #  define MCACHE_MAX_SIZE (1UL<<31) /* 2GB Cap */
@@ -41,6 +44,7 @@ typedef struct MapCacheEntry {
 unsigned long *valid_mapping;
 uint32_t lock;
 #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
+#define XEN_MAPCACHE_ENTRY_GRANT (1 << 1)
 uint8_t flags;
 hwaddr size;
 
@@ -74,6 +78,8 @@ typedef struct MapCache {
 } MapCache;
 
 static MapCache *mapcache;
+static MapCache *mapcache_grants;
+static xengnttab_handle *xen_region_gnttabdev;
 
 static inline void mapcache_lock(MapCache *mc)
 {
@@ -132,6 +138,12 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 unsigned long max_mcache_size;
 unsigned int bucket_shift;
 
+xen_region_gnttabdev = xengnttab_open(NULL, 0);
+if (xen_region_gnttabdev == NULL) {
+error_report("mapcache: Failed to open gnttab device");
+exit(EXIT_FAILURE);
+}
+
 if (HOST_LONG_BITS == 32) {
 bucket_shift = 16;
 } else {
@@ -160,6 +172,15 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 mapcache = xen_map_cache_init_single(f, opaque,
  bucket_shift,
  max_mcache_size);
+
+/*
+ * Grant mappings must use XC_PAGE_SIZE granularity since we can't
+ * map anything beyond the number of pages granted to us.
+ */
+mapcache_grants = xen_map_cache_init_single(f, opaque,
+XC_PAGE_SHIFT,
+max_mcache_size);
+
 setrlimit(RLIMIT_AS, _as);
 }
 
@@ -169,17 +190,25 @@ static void xen_remap_bucket(MapCache *mc,
  hwaddr size,
  hwaddr address_index,
  bool dummy,
+ bool grant,
+ bool grant_is_write,
+ hwaddr grant_ref,
  ram_addr_t ram_offset)
 {
 uint8_t *vaddr_base;
-xen_pfn_t *pfns;
+uint32_t *refs = NULL;
+xen_pfn_t *pfns = NULL;
 int *err;
 unsigned int i;
 hwaddr nb_pfn = size >> XC_PAGE_SHIFT;
 
 trace_xen_remap_bucket(address_index);
 
-pfns = g_new0(xen_pfn_t, nb_pfn);
+if (grant) {
+refs = g_new0(uint32_t, nb_pfn);
+} else {
+pfns = g_new0(xen_pfn_t, nb_pfn);
+}
 err = g_new0(int, nb_pfn);
 
 if (entry->vaddr_base != NULL) {
@@ -208,21 +237,45 @@ static void xen_remap_bucket(MapCache *mc,
 g_free(entry->valid_mapping);
 entry->valid_mapping = NULL;
 
-for (i = 0; i < nb_pfn; i++) {
-pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + i;
+if (grant) {
+for (i = 0; i < nb_pfn; i++) {
+refs[i] = grant_ref + i;
+}
+} else {
+for (i = 0; i < nb_pfn; i++) {
+pfns[i] = (address_index << (mc->bucket_shift - XC_PAGE_SHIFT)) + 
i;
+}
 }
 
-/*
- * If the caller has requested the mapping at a specific address use
- * MAP_FIXED to make sure it's 

[PATCH v4 07/17] xen: mapcache: Refactor xen_replace_cache_entry_unlocked

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Add MapCache argument to xen_replace_cache_entry_unlocked in
preparation for supporting multiple map caches.

No functional change.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index d2deff70c8..6e758eff94 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -556,7 +556,8 @@ void xen_invalidate_map_cache(void)
 mapcache_unlock(mapcache);
 }
 
-static uint8_t *xen_replace_cache_entry_unlocked(hwaddr old_phys_addr,
+static uint8_t *xen_replace_cache_entry_unlocked(MapCache *mc,
+ hwaddr old_phys_addr,
  hwaddr new_phys_addr,
  hwaddr size)
 {
@@ -578,7 +579,7 @@ static uint8_t *xen_replace_cache_entry_unlocked(hwaddr 
old_phys_addr,
 cache_size += MCACHE_BUCKET_SIZE - (cache_size % MCACHE_BUCKET_SIZE);
 }
 
-entry = >entry[address_index % mapcache->nr_buckets];
+entry = >entry[address_index % mc->nr_buckets];
 while (entry && !(entry->paddr_index == address_index &&
   entry->size == cache_size)) {
 entry = entry->next;
@@ -614,7 +615,8 @@ uint8_t *xen_replace_cache_entry(hwaddr old_phys_addr,
 uint8_t *p;
 
 mapcache_lock(mapcache);
-p = xen_replace_cache_entry_unlocked(old_phys_addr, new_phys_addr, size);
+p = xen_replace_cache_entry_unlocked(mapcache, old_phys_addr,
+ new_phys_addr, size);
 mapcache_unlock(mapcache);
 return p;
 }
-- 
2.40.1

[PATCH v4 15/17] xen: mapcache: Remove assumption of RAMBlock with 0 offset

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

The current mapcache assumes that all memory is mapped
in a single RAM MR (the first one with offset 0). Remove
this assumption and propagate the offset to the mapcache
so it can do reverse mappings (from hostptr -> ram_addr).

This is in preparation for adding grant mappings.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 25 ++---
 include/sysemu/xen-mapcache.h |  2 ++
 system/physmem.c  |  8 
 3 files changed, 24 insertions(+), 11 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 09b5f36d9c..1b32d0c003 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -43,6 +43,9 @@ typedef struct MapCacheEntry {
 #define XEN_MAPCACHE_ENTRY_DUMMY (1 << 0)
 uint8_t flags;
 hwaddr size;
+
+/* Keep ram_addr offset for reverse mappings (hostptr -> ram_addr).  */
+ram_addr_t ram_offset;
 struct MapCacheEntry *next;
 } MapCacheEntry;
 
@@ -165,7 +168,8 @@ static void xen_remap_bucket(MapCache *mc,
  void *vaddr,
  hwaddr size,
  hwaddr address_index,
- bool dummy)
+ bool dummy,
+ ram_addr_t ram_offset)
 {
 uint8_t *vaddr_base;
 xen_pfn_t *pfns;
@@ -244,6 +248,7 @@ static void xen_remap_bucket(MapCache *mc,
 entry->size = size;
 entry->valid_mapping = g_new0(unsigned long,
   BITS_TO_LONGS(size >> XC_PAGE_SHIFT));
+entry->ram_offset = ram_offset;
 
 if (dummy) {
 entry->flags |= XEN_MAPCACHE_ENTRY_DUMMY;
@@ -264,6 +269,7 @@ static void xen_remap_bucket(MapCache *mc,
 
 static uint8_t *xen_map_cache_unlocked(MapCache *mc,
hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_offset,
uint8_t lock, bool dma, bool is_write)
 {
 MapCacheEntry *entry, *pentry = NULL,
@@ -335,14 +341,16 @@ tryagain:
 if (!entry) {
 entry = g_new0(MapCacheEntry, 1);
 pentry->next = entry;
-xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+ ram_offset);
 } else if (!entry->lock) {
 if (!entry->vaddr_base || entry->paddr_index != address_index ||
 entry->size != cache_size ||
 !test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
-xen_remap_bucket(mc, entry, NULL, cache_size, address_index, 
dummy);
+xen_remap_bucket(mc, entry, NULL, cache_size, address_index, dummy,
+ ram_offset);
 }
 }
 
@@ -389,13 +397,15 @@ tryagain:
 
 uint8_t *xen_map_cache(MemoryRegion *mr,
hwaddr phys_addr, hwaddr size,
+   ram_addr_t ram_addr_offset,
uint8_t lock, bool dma,
bool is_write)
 {
 uint8_t *p;
 
 mapcache_lock(mapcache);
-p = xen_map_cache_unlocked(mapcache, phys_addr, size, lock, dma, is_write);
+p = xen_map_cache_unlocked(mapcache, phys_addr, size, ram_addr_offset,
+   lock, dma, is_write);
 mapcache_unlock(mapcache);
 return p;
 }
@@ -432,7 +442,8 @@ static ram_addr_t 
xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 raddr = RAM_ADDR_INVALID;
 } else {
 raddr = (reventry->paddr_index << mc->bucket_shift) +
- ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
+ ((unsigned long) ptr - (unsigned long) entry->vaddr_base) +
+ entry->ram_offset;
 }
 mapcache_unlock(mc);
 return raddr;
@@ -627,8 +638,8 @@ static uint8_t *xen_replace_cache_entry_unlocked(MapCache 
*mc,
 
 trace_xen_replace_cache_entry_dummy(old_phys_addr, new_phys_addr);
 
-xen_remap_bucket(mapcache, entry, entry->vaddr_base,
- cache_size, address_index, false);
+xen_remap_bucket(mc, entry, entry->vaddr_base,
+ cache_size, address_index, false, entry->ram_offset);
 if (!test_bits(address_offset >> XC_PAGE_SHIFT,
 test_bit_size >> XC_PAGE_SHIFT,
 entry->valid_mapping)) {
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index 1ec9e66752..b5e3ea1bc0 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -19,6 +19,7 @@ typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
 void xen_map_cache_init(phys_offset_to_gaddr_t f,
 void *opaque);
 uint8_t

[PATCH v4 10/17] xen: mapcache: Break out xen_map_cache_init_single()

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Break out xen_map_cache_init_single() in preparation for
adding multiple map caches.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 53 ++-
 1 file changed, 32 insertions(+), 21 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index dd08cd296b..72a7e25e3e 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -93,23 +93,44 @@ static inline int test_bits(int nr, int size, const 
unsigned long *addr)
 return 0;
 }
 
-void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
+static MapCache *xen_map_cache_init_single(phys_offset_to_gaddr_t f,
+   void *opaque,
+   unsigned long max_size)
 {
 unsigned long size;
-struct rlimit rlimit_as;
+MapCache *mc;
+
+mc = g_new0(MapCache, 1);
+
+mc->phys_offset_to_gaddr = f;
+mc->opaque = opaque;
+qemu_mutex_init(>lock);
+
+QTAILQ_INIT(>locked_entries);
 
-mapcache = g_new0(MapCache, 1);
+mc->max_mcache_size = max_size;
 
-mapcache->phys_offset_to_gaddr = f;
-mapcache->opaque = opaque;
-qemu_mutex_init(>lock);
+mc->nr_buckets =
+(((mc->max_mcache_size >> XC_PAGE_SHIFT) +
+  (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
+ (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
 
-QTAILQ_INIT(>locked_entries);
+size = mc->nr_buckets * sizeof(MapCacheEntry);
+size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
+trace_xen_map_cache_init(mc->nr_buckets, size);
+mc->entry = g_malloc0(size);
+return mc;
+}
+
+void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
+{
+struct rlimit rlimit_as;
+unsigned long max_mcache_size;
 
 if (geteuid() == 0) {
 rlimit_as.rlim_cur = RLIM_INFINITY;
 rlimit_as.rlim_max = RLIM_INFINITY;
-mapcache->max_mcache_size = MCACHE_MAX_SIZE;
+max_mcache_size = MCACHE_MAX_SIZE;
 } else {
 getrlimit(RLIMIT_AS, _as);
 rlimit_as.rlim_cur = rlimit_as.rlim_max;
@@ -119,24 +140,14 @@ void xen_map_cache_init(phys_offset_to_gaddr_t f, void 
*opaque)
 " memory is not infinity");
 }
 if (rlimit_as.rlim_max < MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE) {
-mapcache->max_mcache_size = rlimit_as.rlim_max -
-NON_MCACHE_MEMORY_SIZE;
+max_mcache_size = rlimit_as.rlim_max - NON_MCACHE_MEMORY_SIZE;
 } else {
-mapcache->max_mcache_size = MCACHE_MAX_SIZE;
+max_mcache_size = MCACHE_MAX_SIZE;
 }
 }
 
+mapcache = xen_map_cache_init_single(f, opaque, max_mcache_size);
 setrlimit(RLIMIT_AS, _as);
-
-mapcache->nr_buckets =
-(((mapcache->max_mcache_size >> XC_PAGE_SHIFT) +
-  (1UL << (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT)) - 1) >>
- (MCACHE_BUCKET_SHIFT - XC_PAGE_SHIFT));
-
-size = mapcache->nr_buckets * sizeof (MapCacheEntry);
-size = (size + XC_PAGE_SIZE - 1) & ~(XC_PAGE_SIZE - 1);
-trace_xen_map_cache_init(mapcache->nr_buckets, size);
-mapcache->entry = g_malloc0(size);
 }
 
 static void xen_remap_bucket(MapCache *mc,
-- 
2.40.1

[PATCH v4 06/17] xen: mapcache: Break out xen_ram_addr_from_mapcache_single

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" 

Break out xen_ram_addr_from_mapcache_single(), a multi-cache
aware version of xen_ram_addr_from_mapcache.

No functional changes.

Signed-off-by: Edgar E. Iglesias 
---
 hw/xen/xen-mapcache.c | 17 +++--
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 326a9b61ca..d2deff70c8 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -377,7 +377,7 @@ uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
 return p;
 }
 
-ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
+static ram_addr_t xen_ram_addr_from_mapcache_single(MapCache *mc, void *ptr)
 {
 MapCacheEntry *entry = NULL;
 MapCacheRev *reventry;
@@ -386,8 +386,8 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 ram_addr_t raddr;
 int found = 0;
 
-mapcache_lock(mapcache);
-QTAILQ_FOREACH(reventry, >locked_entries, next) {
+mapcache_lock(mc);
+QTAILQ_FOREACH(reventry, >locked_entries, next) {
 if (reventry->vaddr_req == ptr) {
 paddr_index = reventry->paddr_index;
 size = reventry->size;
@@ -396,11 +396,11 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 }
 }
 if (!found) {
-mapcache_unlock(mapcache);
+mapcache_unlock(mc);
 return RAM_ADDR_INVALID;
 }
 
-entry = >entry[paddr_index % mapcache->nr_buckets];
+entry = >entry[paddr_index % mc->nr_buckets];
 while (entry && (entry->paddr_index != paddr_index || entry->size != 
size)) {
 entry = entry->next;
 }
@@ -411,10 +411,15 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
 raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
  ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
 }
-mapcache_unlock(mapcache);
+mapcache_unlock(mc);
 return raddr;
 }
 
+ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
+{
+return xen_ram_addr_from_mapcache_single(mapcache, ptr);
+}
+
 static void xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
 {
 MapCacheEntry *entry = NULL, *pentry = NULL;
-- 
2.40.1

Re: [QEMU][PATCH v3 5/7] memory: add MemoryRegion map and unmap callbacks

[Edgar E. Iglesias]

On Tue, Apr 16, 2024 at 5:55 PM Peter Xu  wrote:
>
> On Tue, Apr 16, 2024 at 03:28:41PM +0200, Jürgen Groß wrote:
> > On 16.04.24 13:32, Edgar E. Iglesias wrote:
> > > On Wed, Apr 10, 2024 at 8:56 PM Peter Xu  wrote:
> > > >
> > > > On Wed, Apr 10, 2024 at 06:44:38PM +0200, Edgar E. Iglesias wrote:
> > > > > On Tue, Feb 27, 2024 at 11:37 PM Vikram Garhwal 
> > > > > 
> > > > > wrote:
> > > > >
> > > > > > From: Juergen Gross 
> > > > > >
> > > > > > In order to support mapping and unmapping guest memory dynamically 
> > > > > > to
> > > > > > and from qemu during address_space_[un]map() operations add the 
> > > > > > map()
> > > > > > and unmap() callbacks to MemoryRegionOps.
> > > > > >
> > > > > > Those will be used e.g. for Xen grant mappings when performing guest
> > > > > > I/Os.
> > > > > >
> > > > > > Signed-off-by: Juergen Gross 
> > > > > > Signed-off-by: Vikram Garhwal 
> > > > > >
> > > > >
> > > > >
> > > > > Paolo, Peter, David, Phiippe, do you guys have any concerns with this 
> > > > > patch?
> > > >
> > >
> > > Thanks for your comments Peter,
> > >
> > >
> > > > This introduces a 3rd memory type afaict, neither direct nor !direct.
> > > >
> > > > What happens if someone does address_space_write() to it?  I didn't see 
> > > > it
> > > > covered here..
> > >
> > > You're right, that won't work, the memory needs to be mapped before it
> > > can be used.
> > > At minimum there should be some graceful failure, right now this will 
> > > crash.
> > >
> > > >
> > > > OTOH, the cover letter didn't mention too much either on the big 
> > > > picture:
> > > >
> > > > https://lore.kernel.org/all/20240227223501.28475-1-vikram.garh...@amd.com/
> > > >
> > > > I want to have a quick grasp on whether it's justified worthwhile we 
> > > > should
> > > > introduce this complexity to qemu memory core.
> > > >
> > > > Could I request a better cover letter when repost?  It'll be great to
> > > > mention things like:
> > >
> > > I'll do that, but also answer inline in the meantime since we should
> > > perhaps change the approach.
> > >
> > > >
> > > >- what is grant mapping, why it needs to be used, when it can be 
> > > > used (is
> > > >  it only relevant to vIOMMU=on)?  Some more information on the high
> > > >  level design using this type or MR would be great.
> > >
> > > https://github.com/xen-project/xen/blob/master/docs/misc/grant-tables.txt
> > >
> > > Xen VM's that use QEMU's VirtIO have a QEMU instance running in a 
> > > separate VM.
> > >
> > > There's basically two mechanisms for QEMU's Virtio backends to access
> > > the guest's RAM.
> > > 1. Foreign mappings. This gives the VM running QEMU access to the
> > > entire RAM of the guest VM.
> >
> > Additionally it requires qemu to run in dom0, while in general Xen allows
> > to run backends in less privileged "driver domains", which are usually not
> > allowed to perform foreign mappings.
> >
> > > 2. Grant mappings. This allows the guest to dynamically grant and
> > > remove access to pages as needed.
> > > So the VM running QEMU, cannot map guest RAM unless it's been
> > > instructed to do so by the guest.
> > >
> > > #2 is desirable because if QEMU gets compromised it has a smaller
> > > attack surface onto the guest.
> >
> > And it allows to run the virtio backend in a less privileged VM.
> >
> > >
> > > >
> > > >- why a 3rd memory type is required?  Do we have other alternatives?
> > >
> > > Yes, there are alternatives.
> > >
> > > 1. It was suggested by Stefano to try to handle this in existing 
> > > qemu/hw/xen/*.
> > > This would be less intrusive but perhaps also less explicit.
> > > Concerns about touching the Memory API have been raised before, so
> > > perhaps we should try this.
> > > I'm a little unsure how we would deal with unmapping when the guest
> > > removes our grants and w

Re: [PATCH RFC] prevent overflow in xlnx_dpdma_desc_get_source_address()

[Edgar E. Iglesias]

+ To: Fred

On Tue, 16 Apr 2024 at 19:56, Alexandra Diupina 
wrote:

> Peter, thank you! I agree with you that
> as mentioned in the documentation
> https://docs.amd.com/r/en-US/ug1085-zynq-ultrascale-trm/ADDR_EXT-Field,
> we should take 32 bits of the address from one field
> (for example, case 1, SRC_ADDR2_EXT - in code it is desc->source_address2)
> and 16 bits (high or low) of the address from another field
> (ADDR_EXT_23 - in code it is desc->address_extension_23, we need [15:0]
> bits)
> and combine them to make a 48 bit address.
>
> Therefore, I suggest making the following changes to the code
> so that it matches the documentation:
>
> static uint64_t xlnx_dpdma_desc_get_source_address(DPDMADescriptor *desc,
>   uint8_t frag)
> {
>  uint64_t addr = 0;
>  assert(frag < 5);
>
>  switch (frag) {
>  case 0:
>  addr = (uint64_t)desc->source_address
>  + (extract64(desc->address_extension, 16, 16) << 32);
>  break;
>  case 1:
>  addr = (uint64_t)desc->source_address2
>  + (extract64(desc->address_extension_23, 0, 16) << 32);
>  break;
>  case 2:
>  addr = (uint64_t)desc->source_address3
>  + (extract64(desc->address_extension_23, 16, 16) << 32);
>  break;
>  case 3:
>  addr = (uint64_t)desc->source_address4
>  + (extract64(desc->address_extension_45, 0, 16) << 32);
>  break;
>  case 4:
>  addr = (uint64_t)desc->source_address5
>  + (extract64(desc->address_extension_45, 16, 16) << 32);
>  break;
>  default:
>  addr = 0;
>  break;
>  }
>
>  return addr;
> }
>
>
> This change adds a type cast and also uses extract64() instead of
> extract32()
> to avoid integer overflow on addition (there was a typo in the previous
> letter).
> Also in extract64() extracts a bit field with a length of 16 bits
> instead of 12,
> the shift is changed to 32 so that the extracted field fits into bits
> [47:32] of the final address.
>
> if this calculation is correct, I'm ready to create a second version of
> the patch.
>
>
>
>
> 12/04/24 13:06, Peter Maydell пишет:
> > On Fri, 12 Apr 2024 at 09:13, Alexandra Diupina 
> wrote:
> >> Overflow can occur in a situation where desc->source_address
> >> has a maximum value (pow(2, 32) - 1), so add a cast to a
> >> larger type before the assignment.
> >>
> >> Found by Linux Verification Center (linuxtesting.org) with SVACE.
> >>
> >> Fixes: d3c6369a96 ("introduce xlnx-dpdma")
> >> Signed-off-by: Alexandra Diupina 
> >> ---
> >>   hw/dma/xlnx_dpdma.c | 20 ++--
> >>   1 file changed, 10 insertions(+), 10 deletions(-)
> >>
> >> diff --git a/hw/dma/xlnx_dpdma.c b/hw/dma/xlnx_dpdma.c
> >> index 1f5cd64ed1..224259225c 100644
> >> --- a/hw/dma/xlnx_dpdma.c
> >> +++ b/hw/dma/xlnx_dpdma.c
> >> @@ -175,24 +175,24 @@ static uint64_t
> xlnx_dpdma_desc_get_source_address(DPDMADescriptor *desc,
> >>
> >>   switch (frag) {
> >>   case 0:
> >> -addr = desc->source_address
> >> -+ (extract32(desc->address_extension, 16, 12) << 20);
> >> +addr = (uint64_t)(desc->source_address
> >> ++ (extract32(desc->address_extension, 16, 12) << 20));
> > Unless I'm confused, this cast doesn't help, because we
> > will have already done a 32-bit addition of desc->source_address
> > and the value from the address_extension part, so it doesn't
> > change the result.
> >
> > If we want to do the addition at 64 bits then using extract64()
> > would be the simplest way to arrange for that.
> >
> > However, I can't figure out what this code is trying to do and
> > make that line up with the data sheet; maybe this isn't the
> > right datasheet for this device?
> >
> > https://docs.amd.com/r/en-US/ug1085-zynq-ultrascale-trm/ADDR_EXT-Field
> >
> > The datasheet suggests that we should take 32 bits of the address
> > from one field (here desc->source_address) and 16 bits of the
> > address from another (here desc->address_extension's high bits)
> > and combine them to make a 48 bit address. But this code is only
> > looking at 12 bits of the high 16 in address_extension, and it
> > doesn't shift them right enough to put them into bits [47:32]
> > of the final address.
> >
> > Xilinx folks: what hardware is this modelling, and is it
> > really the right behaviour?
> >
> > Also, this device looks like it has a host-endianness bug: the
> > DPDMADescriptor struct is read directly from guest memory in
> > dma_memory_read(), but the device never does anything to swap
> > the fields from guest memory order to host memory order. So
> > this is likely broken on big-endian hosts.
> >
> > thanks
> > -- PMM
>
>
>

Re: [QEMU][PATCH v3 5/7] memory: add MemoryRegion map and unmap callbacks

[Edgar E. Iglesias]

On Wed, Apr 10, 2024 at 8:56 PM Peter Xu  wrote:
>
> On Wed, Apr 10, 2024 at 06:44:38PM +0200, Edgar E. Iglesias wrote:
> > On Tue, Feb 27, 2024 at 11:37 PM Vikram Garhwal 
> > wrote:
> >
> > > From: Juergen Gross 
> > >
> > > In order to support mapping and unmapping guest memory dynamically to
> > > and from qemu during address_space_[un]map() operations add the map()
> > > and unmap() callbacks to MemoryRegionOps.
> > >
> > > Those will be used e.g. for Xen grant mappings when performing guest
> > > I/Os.
> > >
> > > Signed-off-by: Juergen Gross 
> > > Signed-off-by: Vikram Garhwal 
> > >
> >
> >
> > Paolo, Peter, David, Phiippe, do you guys have any concerns with this patch?
>

Thanks for your comments Peter,


> This introduces a 3rd memory type afaict, neither direct nor !direct.
>
> What happens if someone does address_space_write() to it?  I didn't see it
> covered here..

You're right, that won't work, the memory needs to be mapped before it
can be used.
At minimum there should be some graceful failure, right now this will crash.

>
> OTOH, the cover letter didn't mention too much either on the big picture:
>
> https://lore.kernel.org/all/20240227223501.28475-1-vikram.garh...@amd.com/
>
> I want to have a quick grasp on whether it's justified worthwhile we should
> introduce this complexity to qemu memory core.
>
> Could I request a better cover letter when repost?  It'll be great to
> mention things like:

I'll do that, but also answer inline in the meantime since we should
perhaps change the approach.

>
>   - what is grant mapping, why it needs to be used, when it can be used (is
> it only relevant to vIOMMU=on)?  Some more information on the high
> level design using this type or MR would be great.

https://github.com/xen-project/xen/blob/master/docs/misc/grant-tables.txt

Xen VM's that use QEMU's VirtIO have a QEMU instance running in a separate VM.

There's basically two mechanisms for QEMU's Virtio backends to access
the guest's RAM.
1. Foreign mappings. This gives the VM running QEMU access to the
entire RAM of the guest VM.
2. Grant mappings. This allows the guest to dynamically grant and
remove access to pages as needed.
So the VM running QEMU, cannot map guest RAM unless it's been
instructed to do so by the guest.

#2 is desirable because if QEMU gets compromised it has a smaller
attack surface onto the guest.

>
>   - why a 3rd memory type is required?  Do we have other alternatives?

Yes, there are alternatives.

1. It was suggested by Stefano to try to handle this in existing qemu/hw/xen/*.
This would be less intrusive but perhaps also less explicit.
Concerns about touching the Memory API have been raised before, so
perhaps we should try this.
I'm a little unsure how we would deal with unmapping when the guest
removes our grants and we're using models that don't map but use
address_space_read/write().

2. Another approach could be to change the Xen grant-iommu in the
Linux kernel to work with a grant vIOMMU in QEMU.
Linux could explicitly ask QEMU's grant vIOMMU to map/unmap granted regions.
This would have the benefit that we wouldn't need to allocate
address-bit 63 for grants.
A drawback is that it may be slower since we're going to need to
bounce between guest/qemu a bit more.


> So it's all based on my very limited understanding of reading this:
> https://xenbits.xenproject.org/docs/4.3-testing/misc/grant-tables.txt
>
> If it's about cross-vm sharing memory, does it mean that in reality
> there are RAMs allocated, but it's only about permission management?
> In that case, is there any option we implement it with direct access
> mode (however with some extra dynamic permissions applied on top using
> some mechanism)?

Yes, I think the grant vIOMMU approach would fall into this category.

>
>   - perhaps sold old links would be great too so people can read about the
> context when it's not obvious, without a need to copy-paste.

https://wiki.xenproject.org/wiki/Virtio_On_Xen
https://lwn.net/Articles/896938/

Cheers,
Edgar

Re: [PATCH 0/6] disas/cris: Use GString instead of sprintf

[Edgar E. Iglesias]

On Sat, Apr 13, 2024 at 7:23 AM Richard Henderson
 wrote:
>
> More sprintf cleanup encouraged by the Apple deprecation.
> Probably there's a more minimal patch.  On the other hand,
> there's certainly a larger cleanup possible.
>
>
> r~


On the series:
Reviewed-by: Edgar E. Iglesias 


>
>
> Richard Henderson (6):
>   disas/cris: Untabify
>   disas/cris: Remove TRACE_CASE and related code
>   disas/cris: Drop with_reg_prefix
>   disas/cris: Use GString in print_with_operands and subroutines
>   disas/cris: Remove struct cris_disasm_data
>   disas/cris: Improve output of register names
>
>  disas/cris.c | 2498 +-
>  1 file changed, 1070 insertions(+), 1428 deletions(-)
>
> --
> 2.34.1
>

Re: [PATCH 4/6] hw, target: Add ResetType argument to hold and exit phase methods

[Edgar E. Iglesias]

On Fri, Apr 12, 2024 at 6:09 PM Peter Maydell  wrote:
>
> We pass a ResetType argument to the Resettable class enter
> phase method, but we don't pass it to hold and exit, even though
> the callsites have it readily available. This means that if
> a device cared about the ResetType it would need to record it
> in the enter phase method to use later on. Pass the type to
> all three of the phase methods to avoid having to do that.
>
> Commit created with
>
>   for dir in hw target include; do \
>   spatch --macro-file scripts/cocci-macro-file.h \
>  --sp-file scripts/coccinelle/reset-type.cocci \
>  --keep-comments --smpl-spacing --in-place \
>  --include-headers --dir $dir; done
>
> and no manual edits.
>
> Signed-off-by: Peter Maydell 

Reviewed-by: Edgar E. Iglesias 


> ---
>  include/hw/resettable.h |  4 ++--
>  hw/adc/npcm7xx_adc.c|  2 +-
>  hw/arm/pxa2xx_pic.c |  2 +-
>  hw/arm/smmu-common.c|  2 +-
>  hw/arm/smmuv3.c |  4 ++--
>  hw/arm/stellaris.c  | 10 +-
>  hw/audio/asc.c  |  2 +-
>  hw/char/cadence_uart.c  |  2 +-
>  hw/char/sifive_uart.c   |  2 +-
>  hw/core/cpu-common.c|  2 +-
>  hw/core/qdev.c  |  4 ++--
>  hw/core/reset.c |  2 +-
>  hw/core/resettable.c|  4 ++--
>  hw/display/virtio-vga.c |  4 ++--
>  hw/gpio/npcm7xx_gpio.c  |  2 +-
>  hw/gpio/pl061.c |  2 +-
>  hw/gpio/stm32l4x5_gpio.c|  2 +-
>  hw/hyperv/vmbus.c   |  2 +-
>  hw/i2c/allwinner-i2c.c  |  2 +-
>  hw/i2c/npcm7xx_smbus.c  |  2 +-
>  hw/input/adb.c  |  2 +-
>  hw/input/ps2.c  | 12 ++--
>  hw/intc/arm_gic_common.c|  2 +-
>  hw/intc/arm_gic_kvm.c   |  4 ++--
>  hw/intc/arm_gicv3_common.c  |  2 +-
>  hw/intc/arm_gicv3_its.c |  4 ++--
>  hw/intc/arm_gicv3_its_common.c  |  2 +-
>  hw/intc/arm_gicv3_its_kvm.c |  4 ++--
>  hw/intc/arm_gicv3_kvm.c |  4 ++--
>  hw/intc/xics.c  |  2 +-
>  hw/m68k/q800-glue.c |  2 +-
>  hw/misc/djmemc.c|  2 +-
>  hw/misc/iosb.c  |  2 +-
>  hw/misc/mac_via.c   |  8 
>  hw/misc/macio/cuda.c|  4 ++--
>  hw/misc/macio/pmu.c |  4 ++--
>  hw/misc/mos6522.c   |  2 +-
>  hw/misc/npcm7xx_mft.c   |  2 +-
>  hw/misc/npcm7xx_pwm.c   |  2 +-
>  hw/misc/stm32l4x5_exti.c|  2 +-
>  hw/misc/stm32l4x5_rcc.c | 10 +-
>  hw/misc/stm32l4x5_syscfg.c  |  2 +-
>  hw/misc/xlnx-versal-cframe-reg.c|  2 +-
>  hw/misc/xlnx-versal-crl.c   |  2 +-
>  hw/misc/xlnx-versal-pmc-iou-slcr.c  |  2 +-
>  hw/misc/xlnx-versal-trng.c  |  2 +-
>  hw/misc/xlnx-versal-xramc.c |  2 +-
>  hw/misc/xlnx-zynqmp-apu-ctrl.c  |  2 +-
>  hw/misc/xlnx-zynqmp-crf.c   |  2 +-
>  hw/misc/zynq_slcr.c |  4 ++--
>  hw/net/can/xlnx-zynqmp-can.c|  2 +-
>  hw/net/e1000.c  |  2 +-
>  hw/net/e1000e.c |  2 +-
>  hw/net/igb.c|  2 +-
>  hw/net/igbvf.c  |  2 +-
>  hw/nvram/xlnx-bbram.c   |  2 +-
>  hw/nvram/xlnx-versal-efuse-ctrl.c   |  2 +-
>  hw/nvram/xlnx-zynqmp-efuse.c|  2 +-
>  hw/pci-bridge/cxl_root_port.c   |  4 ++--
>  hw/pci-bridge/pcie_root_port.c  |  2 +-
>  hw/pci-host/bonito.c|  2 +-
>  hw/pci-host/pnv_phb.c   |  4 ++--
>  hw/pci-host/pnv_phb3_msi.c  |  4 ++--
>  hw/pci/pci.c|  4 ++--
>  hw/rtc/mc146818rtc.c|  2 +-
>  hw/s390x/css-bridge.c   |  2 +-
>  hw/sensor/adm1266.c |  2 +-
>  hw/sensor/adm1272.c |  2 +-
>  hw/sensor/isl_pmbus_vr.c| 10 +-
>  hw/sensor/max31785.c|  2 +-
>  hw/sensor/max34451.c|  2 +-
>  hw/ssi/npcm7xx_fiu.c|  2 +-
>  hw/timer/etraxfs_timer.c|  2 +-
>  hw/timer/npcm7xx_timer.c|  2 +-
>  hw/usb/hcd-dwc2.c   |  8 
>  hw/usb/xlnx-versal-usb2-ctrl-regs.c |  2 +-
>  hw/virtio/virtio-pci.c  |  2 +-
>  target/arm/cpu.c|  4 ++--
>  target/avr/cpu.c|  4 ++--
>  target/cris/cpu.c   |  4 ++--
>  target/he

Re: [PATCH v4 02/10] hw/core: create Resettable QOM interface

[Edgar E. Iglesias]

On Fri, Apr 12, 2024 at 3:05 PM Peter Maydell  wrote:
>
> On Thu, 11 Apr 2024 at 18:23, Philippe Mathieu-Daudé  
> wrote:
> >
> > On 11/4/24 15:43, Peter Maydell wrote:
> > > On Wed, 21 Aug 2019 at 17:34, Damien Hedde  
> > > wrote:
> > >>
> > >> This commit defines an interface allowing multi-phase reset. This aims
> > >> to solve a problem of the actual single-phase reset (built in
> > >> DeviceClass and BusClass): reset behavior is dependent on the order
> > >> in which reset handlers are called. In particular doing external
> > >> side-effect (like setting an qemu_irq) is problematic because receiving
> > >> object may not be reset yet.
> > >
> > > So, I wanted to drag up this ancient patch to ask a couple
> > > of Resettable questions, because I'm working on adding a
> > > new ResetType (the equivalent of SHUTDOWN_CAUSE_SNAPSHOT_LOAD).
> > >
> > >> +/**
> > >> + * ResetType:
> > >> + * Types of reset.
> > >> + *
> > >> + * + Cold: reset resulting from a power cycle of the object.
> > >> + *
> > >> + * TODO: Support has to be added to handle more types. In particular,
> > >> + * ResetState structure needs to be expanded.
> > >> + */
> > >
> > > Does anybody remember what this TODO comment is about? What
> > > in particular would need to be in the ResetState struct
> > > to allow another type to be added?
> >
> > IIRC this comes from this discussion:
> > https://lore.kernel.org/qemu-devel/7c193b33-8188-2cda-cbf2-fb5452544...@greensocs.com/
> > Updated in this patch (see after '---' description):
> > https://lore.kernel.org/qemu-devel/20191018150630.31099-9-damien.he...@greensocs.com/
>
> Hmm, I can't see anything in there that mentions this
> TODO or what we'd need more ResetState fields to handle.
> I guess I'll go ahead with adding my new ResetType and ignore
> this TODO, because I can't see any reason why we need to
> do anything in particular for a new ResetType...
>
> > >
> > >> +typedef enum ResetType {
> > >> +RESET_TYPE_COLD,
> > >> +} ResetType;
> > >
> > >> +typedef void (*ResettableInitPhase)(Object *obj, ResetType type);
> > >> +typedef void (*ResettableHoldPhase)(Object *obj);
> > >> +typedef void (*ResettableExitPhase)(Object *obj);
> > >
> > > Was there a reason why we only pass the ResetType to the init
> > > phase method, and not also to the hold and exit phases ?
> > > Given that many devices don't need to implement init, it
> > > seems awkward to require them to do so just to stash the
> > > ResetType somewhere so they can look at it in the hold
> > > or exit phase, so I was thinking about adding the argument
> > > to the other two phase methods.
> >
> > You are right, the type should be propagated to to all phase
> > handlers.
>
> I have some patches which do this; I'll probably send them out
> in a series next week once I've figured out whether they fit
> better in with other patches that give the motivation.
>

Hi,

I don't remember the details on your first questions but I also agree
with adding the type to the other callbacks!

Cheers,
Edgar

Re: [PATCH 2/9] disas/microblaze: Replace sprintf() by snprintf()

[Edgar E. Iglesias]

On Thu, Apr 11, 2024 at 12:43 PM Philippe Mathieu-Daudé
 wrote:
>
> sprintf() is deprecated on Darwin since macOS 13.0 / XCode 14.1,
> resulting in painful developper experience. Use snprintf() instead.
>
> Signed-off-by: Philippe Mathieu-Daudé 

Reviewed-by: Edgar E. Iglesias 


> ---
>  disas/microblaze.c | 25 -
>  1 file changed, 16 insertions(+), 9 deletions(-)
>
> diff --git a/disas/microblaze.c b/disas/microblaze.c
> index 0b89b9c4fa..49a4c0fd40 100644
> --- a/disas/microblaze.c
> +++ b/disas/microblaze.c
> @@ -600,7 +600,8 @@ static char *
>  get_field (long instr, long mask, unsigned short low)
>  {
>char tmpstr[25];
> -  sprintf(tmpstr, "%s%d", register_prefix, (int)((instr & mask) >> low));
> +  snprintf(tmpstr, sizeof(tmpstr), "%s%d", register_prefix,
> +   (int)((instr & mask) >> low));
>return(strdup(tmpstr));
>  }
>
> @@ -608,7 +609,8 @@ static char *
>  get_field_imm (long instr)
>  {
>char tmpstr[25];
> -  sprintf(tmpstr, "%d", (short)((instr & IMM_MASK) >> IMM_LOW));
> +  snprintf(tmpstr, sizeof(tmpstr), "%d",
> +   (short)((instr & IMM_MASK) >> IMM_LOW));
>return(strdup(tmpstr));
>  }
>
> @@ -616,7 +618,8 @@ static char *
>  get_field_imm5 (long instr)
>  {
>char tmpstr[25];
> -  sprintf(tmpstr, "%d", (short)((instr & IMM5_MASK) >> IMM_LOW));
> +  snprintf(tmpstr, sizeof(tmpstr), "%d",
> +   (short)((instr & IMM5_MASK) >> IMM_LOW));
>return(strdup(tmpstr));
>  }
>
> @@ -624,7 +627,8 @@ static char *
>  get_field_rfsl (long instr)
>  {
>char tmpstr[25];
> -  sprintf(tmpstr, "%s%d", fsl_register_prefix, (short)((instr & RFSL_MASK) 
> >> IMM_LOW));
> +  snprintf(tmpstr, sizeof(tmpstr), "%s%d", fsl_register_prefix,
> +   (short)((instr & RFSL_MASK) >> IMM_LOW));
>return(strdup(tmpstr));
>  }
>
> @@ -632,7 +636,8 @@ static char *
>  get_field_imm15 (long instr)
>  {
>char tmpstr[25];
> -  sprintf(tmpstr, "%d", (short)((instr & IMM15_MASK) >> IMM_LOW));
> +  snprintf(tmpstr, sizeof(tmpstr), "%d",
> +   (short)((instr & IMM15_MASK) >> IMM_LOW));
>return(strdup(tmpstr));
>  }
>
> @@ -641,7 +646,8 @@ static char *
>  get_field_unsigned_imm (long instr)
>  {
>char tmpstr[25];
> -  sprintf(tmpstr, "%d", (int)((instr & IMM_MASK) >> IMM_LOW));
> +  snprintf(tmpstr, sizeof(tmpstr), "%d",
> +   (int)((instr & IMM_MASK) >> IMM_LOW));
>return(strdup(tmpstr));
>  }
>  #endif
> @@ -653,7 +659,8 @@ get_field_unsigned_imm (long instr)
>{
>char tmpstr[25];
>
> -  sprintf(tmpstr, "%s%s", register_prefix, (((instr & IMM_MASK) >> IMM_LOW) 
> & REG_MSR_MASK) == 0 ? "pc" : "msr");
> +  snprintf(tmpstr, sizeof(tmpstr), "%s%s", register_prefix,
> +  (((instr & IMM_MASK) >> IMM_LOW) & REG_MSR_MASK) == 0 ? "pc" : 
> "msr");
>
>return(strdup(tmpstr));
>}
> @@ -709,7 +716,7 @@ get_field_special(long instr, const struct op_code_struct 
> *op)
> default :
>   {
> if ( instr & IMM_MASK) >> IMM_LOW) ^ op->immval_mask) & 0xE000) 
> == REG_PVR_MASK) {
> - sprintf(tmpstr, "%s%u", pvr_register_prefix,
> +  snprintf(tmpstr, sizeof(tmpstr), "%s%u", pvr_register_prefix,
>   (unsigned short)(((instr & IMM_MASK) >> IMM_LOW) ^
>op->immval_mask) ^ REG_PVR_MASK);
>  return(strdup(tmpstr));
> @@ -720,7 +727,7 @@ get_field_special(long instr, const struct op_code_struct 
> *op)
>   break;
> }
>
> -   sprintf(tmpstr, "%s%s", register_prefix, spr);
> +   snprintf(tmpstr, sizeof(tmpstr), "%s%s", register_prefix, spr);
> return(strdup(tmpstr));
>  }
>
> --
> 2.41.0
>

Re: [QEMU][PATCH v3 5/7] memory: add MemoryRegion map and unmap callbacks

[Edgar E. Iglesias]

On Tue, Feb 27, 2024 at 11:37 PM Vikram Garhwal 
wrote:

> From: Juergen Gross 
>
> In order to support mapping and unmapping guest memory dynamically to
> and from qemu during address_space_[un]map() operations add the map()
> and unmap() callbacks to MemoryRegionOps.
>
> Those will be used e.g. for Xen grant mappings when performing guest
> I/Os.
>
> Signed-off-by: Juergen Gross 
> Signed-off-by: Vikram Garhwal 
>


Paolo, Peter, David, Phiippe, do you guys have any concerns with this patch?

Cheers,
Edgar



> ---
>  include/exec/memory.h | 21 ++
>  system/physmem.c  | 50 +--
>  2 files changed, 60 insertions(+), 11 deletions(-)
>
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index 8626a355b3..9f7dfe59c7 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -282,6 +282,27 @@ struct MemoryRegionOps {
>  unsigned size,
>  MemTxAttrs attrs);
>
> +/*
> + * Dynamically create mapping. @addr is the guest address to map;
> @plen
> + * is the pointer to the usable length of the buffer.
> + * @mr contents can be changed in case a new memory region is created
> for
> + * the mapping.
> + * Returns the buffer address for accessing the data.
> + */
> +void *(*map)(MemoryRegion **mr,
> + hwaddr addr,
> + hwaddr *plen,
> + bool is_write,
> + MemTxAttrs attrs);
> +
> +/* Unmap an area obtained via map() before. */
> +void (*unmap)(MemoryRegion *mr,
> +  void *buffer,
> +  ram_addr_t addr,
> +  hwaddr len,
> +  bool is_write,
> +  hwaddr access_len);
> +
>  enum device_endian endianness;
>  /* Guest-visible constraints: */
>  struct {
> diff --git a/system/physmem.c b/system/physmem.c
> index 949dcb20ba..d989e9fc1f 100644
> --- a/system/physmem.c
> +++ b/system/physmem.c
> @@ -3141,6 +3141,7 @@ void *address_space_map(AddressSpace *as,
>  hwaddr len = *plen;
>  hwaddr l, xlat;
>  MemoryRegion *mr;
> +void *ptr = NULL;
>  FlatView *fv;
>
>  if (len == 0) {
> @@ -3174,12 +3175,20 @@ void *address_space_map(AddressSpace *as,
>  return bounce.buffer;
>  }
>
> -
>  memory_region_ref(mr);
> +
> +if (mr->ops && mr->ops->map) {
> +ptr = mr->ops->map(, addr, plen, is_write, attrs);
> +}
> +
>  *plen = flatview_extend_translation(fv, addr, len, mr, xlat,
>  l, is_write, attrs);
>  fuzz_dma_read_cb(addr, *plen, mr);
> -return qemu_ram_ptr_length(mr->ram_block, xlat, plen, true);
> +if (ptr == NULL) {
> +ptr = qemu_ram_ptr_length(mr->ram_block, xlat, plen, true);
> +}
> +
> +return ptr;
>  }
>
>  /* Unmaps a memory region previously mapped by address_space_map().
> @@ -3195,11 +3204,16 @@ void address_space_unmap(AddressSpace *as, void
> *buffer, hwaddr len,
>
>  mr = memory_region_from_host(buffer, );
>  assert(mr != NULL);
> -if (is_write) {
> -invalidate_and_set_dirty(mr, addr1, access_len);
> -}
> -if (xen_enabled()) {
> -xen_invalidate_map_cache_entry(buffer);
> +
> +if (mr->ops && mr->ops->unmap) {
> +mr->ops->unmap(mr, buffer, addr1, len, is_write, access_len);
> +} else {
> +if (is_write) {
> +invalidate_and_set_dirty(mr, addr1, access_len);
> +}
> +if (xen_enabled()) {
> +xen_invalidate_map_cache_entry(buffer);
> +}
>  }
>  memory_region_unref(mr);
>  return;
> @@ -3272,10 +3286,18 @@ int64_t address_space_cache_init(MemoryRegionCache
> *cache,
>   * doing this if we found actual RAM, which behaves the same
>   * regardless of attributes; so UNSPECIFIED is fine.
>   */
> +if (mr->ops && mr->ops->map) {
> +cache->ptr = mr->ops->map(, addr, , is_write,
> +  MEMTXATTRS_UNSPECIFIED);
> +}
> +
>  l = flatview_extend_translation(cache->fv, addr, len, mr,
>  cache->xlat, l, is_write,
>  MEMTXATTRS_UNSPECIFIED);
> -cache->ptr = qemu_ram_ptr_length(mr->ram_block, cache->xlat, ,
> true);
> +if (!cache->ptr) {
> +cache->ptr = qemu_ram_ptr_length(mr->ram_block, cache->xlat,
> ,
> + true);
> +}
>  } else {
>  cache->ptr = NULL;
>  }
> @@ -3297,14 +3319,20 @@ void
> address_space_cache_invalidate(MemoryRegionCache *cache,
>
>  void address_space_cache_destroy(MemoryRegionCache *cache)
>  {
> -if (!cache->mrs.mr) {
> +MemoryRegion *mr = cache->mrs.mr;
> +
> +if (!mr) {
>  

Re: [QEMU][PATCH v3 0/7] Xen: support grant mappings.

[Edgar E. Iglesias]

On Wed, Feb 28, 2024 at 8:00 PM Vikram Garhwal 
wrote:

> Hi Manos,
> On Wed, Feb 28, 2024 at 03:27:12PM +0200, Manos Pitsidianakis wrote:
> > Hello Vikram,
> >
> > Series doesn't apply on master. Can you rebase and also provide a
> > base-commit with --base= when you use git-format-patch? This
> > will help git rebase if there are any conflicts found locally.
> I rebased it with latest master and it works fine. Series is based on
> following
> commit: bfe8020c814a30479a4241aaa78b63960655962b.
>
> For v4, I will send a version with base-commit id.
>
> Can you please share what is base-commit id on your side?
>
> Thanks!
> >
> > Thanks,
>

Hi all,

I'll send a v4 on behalf of Vikram.

Stefano, I saw your comments here:
https://marc.info/?l=qemu-devel=16555103080
I've manage to loose the email so I can't reply but you indicated that
you're OK with the current patch but perhaps would have preferred another
approach.
I like what Juergen did (so I RB tagged it) but if you feel strongly about
finding another approach we can have a look.

Best regards,
Edgar

Re: [QEMU][PATCH v3 5/7] memory: add MemoryRegion map and unmap callbacks

[Edgar E. Iglesias]

On Fri, Mar 1, 2024 at 12:11 AM Stefano Stabellini 
wrote:

> On Tue, 27 Feb 2024, Vikram Garhwal wrote:
> > From: Juergen Gross 
> >
> > In order to support mapping and unmapping guest memory dynamically to
> > and from qemu during address_space_[un]map() operations add the map()
> > and unmap() callbacks to MemoryRegionOps.
> >
> > Those will be used e.g. for Xen grant mappings when performing guest
> > I/Os.
> >
> > Signed-off-by: Juergen Gross 
> > Signed-off-by: Vikram Garhwal 
>
> Reviewed-by: Stefano Stabellini 
>

Reviewed-by: Edgar E. Iglesias 



> ---
> >  include/exec/memory.h | 21 ++
> >  system/physmem.c  | 50 +--
> >  2 files changed, 60 insertions(+), 11 deletions(-)
> >
> > diff --git a/include/exec/memory.h b/include/exec/memory.h
> > index 8626a355b3..9f7dfe59c7 100644
> > --- a/include/exec/memory.h
> > +++ b/include/exec/memory.h
> > @@ -282,6 +282,27 @@ struct MemoryRegionOps {
> >  unsigned size,
> >  MemTxAttrs attrs);
> >
> > +/*
> > + * Dynamically create mapping. @addr is the guest address to map;
> @plen
> > + * is the pointer to the usable length of the buffer.
> > + * @mr contents can be changed in case a new memory region is
> created for
> > + * the mapping.
> > + * Returns the buffer address for accessing the data.
> > + */
> > +void *(*map)(MemoryRegion **mr,
> > + hwaddr addr,
> > + hwaddr *plen,
> > + bool is_write,
> > + MemTxAttrs attrs);
> > +
> > +/* Unmap an area obtained via map() before. */
> > +void (*unmap)(MemoryRegion *mr,
> > +  void *buffer,
> > +  ram_addr_t addr,
> > +  hwaddr len,
> > +  bool is_write,
> > +  hwaddr access_len);
> > +
> >  enum device_endian endianness;
> >  /* Guest-visible constraints: */
> >  struct {
> > diff --git a/system/physmem.c b/system/physmem.c
> > index 949dcb20ba..d989e9fc1f 100644
> > --- a/system/physmem.c
> > +++ b/system/physmem.c
> > @@ -3141,6 +3141,7 @@ void *address_space_map(AddressSpace *as,
> >  hwaddr len = *plen;
> >  hwaddr l, xlat;
> >  MemoryRegion *mr;
> > +void *ptr = NULL;
> >  FlatView *fv;
> >
> >  if (len == 0) {
> > @@ -3174,12 +3175,20 @@ void *address_space_map(AddressSpace *as,
> >  return bounce.buffer;
> >  }
> >
> > -
> >  memory_region_ref(mr);
> > +
> > +if (mr->ops && mr->ops->map) {
> > +ptr = mr->ops->map(, addr, plen, is_write, attrs);
> > +}
> > +
> >  *plen = flatview_extend_translation(fv, addr, len, mr, xlat,
> >  l, is_write, attrs);
> >  fuzz_dma_read_cb(addr, *plen, mr);
> > -return qemu_ram_ptr_length(mr->ram_block, xlat, plen, true);
> > +if (ptr == NULL) {
> > +ptr = qemu_ram_ptr_length(mr->ram_block, xlat, plen, true);
> > +}
> > +
> > +return ptr;
> >  }
> >
> >  /* Unmaps a memory region previously mapped by address_space_map().
> > @@ -3195,11 +3204,16 @@ void address_space_unmap(AddressSpace *as, void
> *buffer, hwaddr len,
> >
> >  mr = memory_region_from_host(buffer, );
> >  assert(mr != NULL);
> > -if (is_write) {
> > -invalidate_and_set_dirty(mr, addr1, access_len);
> > -}
> > -if (xen_enabled()) {
> > -xen_invalidate_map_cache_entry(buffer);
> > +
> > +if (mr->ops && mr->ops->unmap) {
> > +mr->ops->unmap(mr, buffer, addr1, len, is_write,
> access_len);
> > +} else {
> > +if (is_write) {
> > +invalidate_and_set_dirty(mr, addr1, access_len);
> > +}
> > +if (xen_enabled()) {
> > +xen_invalidate_map_cache_entry(buffer);
> > +}
> >  }
> >  memory_region_unref(mr);
> >  return;
> > @@ -3272,10 +3286,18 @@ int64_t
> address_space_cache_init(MemoryRegionCache *cache,
> >   * doing this if we found actual RAM, which behaves the same
> >   * regardless of attributes; so UNSPECIFIED is fine.
> >   

Re: [QEMU][PATCH v3 3/7] softmmu: let qemu_map_ram_ptr() use qemu_ram_ptr_length()

[Edgar E. Iglesias]

On Tue, Feb 27, 2024 at 11:37 PM Vikram Garhwal 
wrote:

> From: Juergen Gross 
>
> qemu_map_ram_ptr() and qemu_ram_ptr_length() share quite some code, so
> modify qemu_ram_ptr_length() a little bit and use it for
> qemu_map_ram_ptr(), too.
>
> Signed-off-by: Juergen Gross 
> Signed-off-by: Vikram Garhwal 
> Reviewed-by: Stefano Stabellini 
>

Reviewed-by: Edgar E. Iglesias 




> ---
>  system/physmem.c | 56 
>  1 file changed, 23 insertions(+), 33 deletions(-)
>
> diff --git a/system/physmem.c b/system/physmem.c
> index 84f3022099..949dcb20ba 100644
> --- a/system/physmem.c
> +++ b/system/physmem.c
> @@ -2163,43 +2163,17 @@ void qemu_ram_remap(ram_addr_t addr, ram_addr_t
> length)
>  }
>  #endif /* !_WIN32 */
>
> -/* Return a host pointer to ram allocated with qemu_ram_alloc.
> - * This should not be used for general purpose DMA.  Use address_space_map
> - * or address_space_rw instead. For local memory (e.g. video ram) that the
> - * device owns, use memory_region_get_ram_ptr.
> - *
> - * Called within RCU critical section.
> - */
> -void *qemu_map_ram_ptr(RAMBlock *block, ram_addr_t addr)
> -{
> -if (block == NULL) {
> -block = qemu_get_ram_block(addr);
> -addr -= block->offset;
> -}
> -
> -if (xen_enabled() && block->host == NULL) {
> -/* We need to check if the requested address is in the RAM
> - * because we don't want to map the entire memory in QEMU.
> - * In that case just map until the end of the page.
> - */
> -if (block->offset == 0) {
> -return xen_map_cache(addr, 0, 0, false);
> -}
> -
> -block->host = xen_map_cache(block->offset, block->max_length, 1,
> false);
> -}
> -return ramblock_ptr(block, addr);
> -}
> -
> -/* Return a host pointer to guest's ram. Similar to qemu_map_ram_ptr
> - * but takes a size argument.
> +/*
> + * Return a host pointer to guest's ram.
>   *
>   * Called within RCU critical section.
>   */
>  static void *qemu_ram_ptr_length(RAMBlock *block, ram_addr_t addr,
>   hwaddr *size, bool lock)
>  {
> -if (*size == 0) {
> +hwaddr len = 0;
> +
> +if (size && *size == 0) {
>  return NULL;
>  }
>
> @@ -2207,7 +2181,10 @@ static void *qemu_ram_ptr_length(RAMBlock *block,
> ram_addr_t addr,
>  block = qemu_get_ram_block(addr);
>  addr -= block->offset;
>  }
> -*size = MIN(*size, block->max_length - addr);
> +if (size) {
> +*size = MIN(*size, block->max_length - addr);
> +len = *size;
> +}
>
>  if (xen_enabled() && block->host == NULL) {
>  /* We need to check if the requested address is in the RAM
> @@ -2215,7 +2192,7 @@ static void *qemu_ram_ptr_length(RAMBlock *block,
> ram_addr_t addr,
>   * In that case just map the requested area.
>   */
>  if (block->offset == 0) {
> -return xen_map_cache(addr, *size, lock, lock);
> +return xen_map_cache(addr, len, lock, lock);
>  }
>
>  block->host = xen_map_cache(block->offset, block->max_length, 1,
> lock);
> @@ -2224,6 +2201,19 @@ static void *qemu_ram_ptr_length(RAMBlock *block,
> ram_addr_t addr,
>  return ramblock_ptr(block, addr);
>  }
>
> +/*
> + * Return a host pointer to ram allocated with qemu_ram_alloc.
> + * This should not be used for general purpose DMA.  Use address_space_map
> + * or address_space_rw instead. For local memory (e.g. video ram) that the
> + * device owns, use memory_region_get_ram_ptr.
> + *
> + * Called within RCU critical section.
> + */
> +void *qemu_map_ram_ptr(RAMBlock *ram_block, ram_addr_t addr)
> +{
> +return qemu_ram_ptr_length(ram_block, addr, NULL, false);
> +}
> +
>  /* Return the offset of a hostpointer within a ramblock */
>  ram_addr_t qemu_ram_block_host_offset(RAMBlock *rb, void *host)
>  {
> --
> 2.17.1
>
>
>

Re: [QEMU][PATCH v3 4/7] xen: let xen_ram_addr_from_mapcache() return -1 in case of not found entry

[Edgar E. Iglesias]

On Fri, Mar 1, 2024 at 6:08 PM Alex Bennée  wrote:

> Vikram Garhwal  writes:
>
> > From: Juergen Gross 
> >
> > Today xen_ram_addr_from_mapcache() will either abort() or return 0 in
> > case it can't find a matching entry for a pointer value. Both cases
> > are bad, so change that to return an invalid address instead.
> >
> > Signed-off-by: Juergen Gross 
> > Reviewed-by: Stefano Stabellini 
> > ---
> >  hw/xen/xen-mapcache.c | 11 +++
> >  1 file changed, 3 insertions(+), 8 deletions(-)
> >
> > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> > index dfc412d138..179b7e95b2 100644
> > --- a/hw/xen/xen-mapcache.c
> > +++ b/hw/xen/xen-mapcache.c
> > @@ -396,13 +396,8 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
> >  }
> >  }
> >  if (!found) {
> > -trace_xen_ram_addr_from_mapcache_not_found(ptr);
> > -QTAILQ_FOREACH(reventry, >locked_entries, next) {
> > -
> trace_xen_ram_addr_from_mapcache_found(reventry->paddr_index,
> > -   reventry->vaddr_req);
> > -}
>
> If these tracepoints aren't useful they need removing from trace-events.
> However I suspect it would be better to keep them in as they are fairly
> cheap.
>
> Otherwise:
>
> Reviewed-by: Alex Bennée 
>
>
Reviewed-by: Edgar E. Iglesias 

Re: [QEMU][PATCH v3 2/7] xen: add pseudo RAM region for grant mappings

[Edgar E. Iglesias]

On Fri, Mar 1, 2024 at 3:06 PM Alex Bennée  wrote:

> Vikram Garhwal  writes:
>
> > From: Juergen Gross 
> >
> > Add a memory region which can be used to automatically map granted
> > memory. It is starting at 0x8000ULL in order to be able to
> > distinguish it from normal RAM.
>
> Is the Xen memory map for HVM guests documented anywhere? I couldn't
> find anything googling or on the Xen wiki. I'm guessing this is going to
> be shared across all 64 bit HVM arches in Xen?
>
> Anyway:
>
> Reviewed-by: Alex Bennée 
>
>
Reviewed-by: Edgar E. Iglesias 

Re: [QEMU][PATCH v3 6/7] xen: add map and unmap callbacks for grant region

[Edgar E. Iglesias]

On Fri, Mar 1, 2024 at 12:34 AM Stefano Stabellini 
wrote:

> On Tue, 27 Feb 2024, Vikram Garhwal wrote:
> > From: Juergen Gross 
> >
> > Add the callbacks for mapping/unmapping guest memory via grants to the
> > special grant memory region.
> >
> > Signed-off-by: Juergen Gross 
> > Signed-off-by: Vikram Garhwal 
>
> Reviewed-by: Stefano Stabellini 
>
>
Reviewed-by: Edgar E. Iglesias 

Re: [QEMU][PATCH v3 1/7] softmmu: physmem: Split ram_block_add()

[Edgar E. Iglesias]

On Fri, Mar 1, 2024 at 12:35 PM Alex Bennée  wrote:

> Vikram Garhwal  writes:
>
> > Extract ram block list update to a new function ram_block_add_list().
> This is
> > done to support grant mappings which adds a memory region for granted
> memory and
> > updates the ram_block list.
> >
> > Signed-off-by: Juergen Gross 
> > Signed-off-by: Vikram Garhwal 
> > Reviewed-by: Stefano Stabellini 
>
> Reviewed-by: Alex Bennée 
>
>

Reviewed-by: Edgar E. Iglesias 

Re: [QEMU][PATCH v3 7/7] hw: arm: Add grant mapping.

[Edgar E. Iglesias]

On Wed, Mar 6, 2024 at 9:57 PM Vikram Garhwal 
wrote:

> Hi Alex,
> On Fri, Mar 01, 2024 at 05:10:28PM +, Alex Bennée wrote:
> > Vikram Garhwal  writes:
> >
> > > Enable grant ram mapping support for Xenpvh machine on ARM.
> > >
> > > Signed-off-by: Vikram Garhwal 
> > > Reviewed-by: Stefano Stabellini 
> > > ---
> > >  hw/arm/xen_arm.c | 3 +++
> > >  1 file changed, 3 insertions(+)
> > >
> > > diff --git a/hw/arm/xen_arm.c b/hw/arm/xen_arm.c
> > > index 32776d94df..b5993ef2a6 100644
> > > --- a/hw/arm/xen_arm.c
> > > +++ b/hw/arm/xen_arm.c
> > > @@ -125,6 +125,9 @@ static void xen_init_ram(MachineState *machine)
> > >   GUEST_RAM1_BASE, ram_size[1]);
> > >  memory_region_add_subregion(sysmem, GUEST_RAM1_BASE, _hi);
> > >  }
> > > +
> > > +DPRINTF("init grant ram mapping for XEN\n");
> >
> > I don't think we need the DPRINTF here (there others where recently
> > converted to trace-points although I suspect a memory_region tracepoint
> > would be a better place to capture this).
> May be drop the print? As it's not providing much information anyways.
>

With the DPRINTF dropped:
Reviewed-by: Edgar E. Iglesias 

Re: [PATCH 24/32] target/microblaze: Use translator_ldl

[Edgar E. Iglesias]

On Fri, Apr 5, 2024 at 12:25 PM Richard Henderson <
richard.hender...@linaro.org> wrote:

> Cc: Edgar E. Iglesias 
> Signed-off-by: Richard Henderson 
>

Reviewed-by: Edgar E. Iglesias 



> ---
>  target/microblaze/translate.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/target/microblaze/translate.c b/target/microblaze/translate.c
> index be3ff76f78..11d84bc514 100644
> --- a/target/microblaze/translate.c
> +++ b/target/microblaze/translate.c
> @@ -24,7 +24,6 @@
>  #include "tcg/tcg-op.h"
>  #include "exec/helper-proto.h"
>  #include "exec/helper-gen.h"
> -#include "exec/cpu_ldst.h"
>  #include "exec/translator.h"
>  #include "qemu/qemu-print.h"
>
> @@ -1640,7 +1639,7 @@ static void mb_tr_translate_insn(DisasContextBase
> *dcb, CPUState *cs)
>
>  dc->tb_flags_to_set = 0;
>
> -ir = cpu_ldl_code(cpu_env(cs), dc->base.pc_next);
> +ir = translator_ldl(cpu_env(cs), >base, dc->base.pc_next);
>  if (!decode(dc, ir)) {
>  trap_illegal(dc, true);
>  }
> --
> 2.34.1
>
>

Re: [PATCH 28/32] target/cris: Use cris_fetch in translate_v10.c.inc

[Edgar E. Iglesias]

On Fri, Apr 5, 2024 at 12:25 PM Richard Henderson <
richard.hender...@linaro.org> wrote:

> Cc: Edgar E. Iglesias 
> Signed-off-by: Richard Henderson 
>

Reviewed-by: Edgar E. Iglesias 


> ---
>  target/cris/translate.c |  1 -
>  target/cris/translate_v10.c.inc | 30 +-
>  2 files changed, 9 insertions(+), 22 deletions(-)
>
> diff --git a/target/cris/translate.c b/target/cris/translate.c
> index bb2d6612ba..a30c67eb07 100644
> --- a/target/cris/translate.c
> +++ b/target/cris/translate.c
> @@ -29,7 +29,6 @@
>  #include "tcg/tcg-op.h"
>  #include "exec/helper-proto.h"
>  #include "mmu.h"
> -#include "exec/cpu_ldst.h"
>  #include "exec/translator.h"
>  #include "crisv32-decode.h"
>  #include "qemu/qemu-print.h"
> diff --git a/target/cris/translate_v10.c.inc
> b/target/cris/translate_v10.c.inc
> index 73fc27c15d..c15ff47505 100644
> --- a/target/cris/translate_v10.c.inc
> +++ b/target/cris/translate_v10.c.inc
> @@ -165,20 +165,7 @@ static int dec10_prep_move_m(CPUCRISState *env,
> DisasContext *dc,
>
>  /* Load [$rs] onto T1.  */
>  if (is_imm) {
> -if (memsize != 4) {
> -if (s_ext) {
> -if (memsize == 1)
> -imm = cpu_ldsb_code(env, dc->pc + 2);
> -else
> -imm = cpu_ldsw_code(env, dc->pc + 2);
> -} else {
> -if (memsize == 1)
> -imm = cpu_ldub_code(env, dc->pc + 2);
> -else
> -imm = cpu_lduw_code(env, dc->pc + 2);
> -}
> -} else
> -imm = cpu_ldl_code(env, dc->pc + 2);
> +imm = cris_fetch(env, dc, dc->pc + 2, memsize, s_ext);
>
>  tcg_gen_movi_tl(dst, imm);
>
> @@ -929,10 +916,11 @@ static int dec10_dip(CPUCRISState *env, DisasContext
> *dc)
>  LOG_DIS("dip pc=%x opcode=%d r%d r%d\n",
>dc->pc, dc->opcode, dc->src, dc->dst);
>  if (dc->src == 15) {
> -imm = cpu_ldl_code(env, dc->pc + 2);
> +imm = cris_fetch(env, dc, dc->pc + 2, 4, 0);
>  tcg_gen_movi_tl(cpu_PR[PR_PREFIX], imm);
> -if (dc->postinc)
> +if (dc->postinc) {
>  insn_len += 4;
> +}
>  tcg_gen_addi_tl(cpu_R[15], cpu_R[15], insn_len - 2);
>  } else {
>  gen_load(dc, cpu_PR[PR_PREFIX], cpu_R[dc->src], 4, 0);
> @@ -1095,10 +1083,10 @@ static unsigned int dec10_ind(CPUCRISState *env,
> DisasContext *dc)
>  if (dc->src == 15) {
>  LOG_DIS("jump.%d %d r%d r%d direct\n", size,
>   dc->opcode, dc->src, dc->dst);
> -imm = cpu_ldl_code(env, dc->pc + 2);
> -if (dc->mode == CRISV10_MODE_AUTOINC)
> +imm = cris_fetch(env, dc, dc->pc + 2, size, 0);
> +if (dc->mode == CRISV10_MODE_AUTOINC) {
>  insn_len += size;
> -
> +}
>  c = tcg_constant_tl(dc->pc + insn_len);
>  t_gen_mov_preg_TN(dc, dc->dst, c);
>  dc->jmp_pc = imm;
> @@ -1164,7 +1152,7 @@ static unsigned int dec10_ind(CPUCRISState *env,
> DisasContext *dc)
>  case CRISV10_IND_BCC_M:
>
>  cris_cc_mask(dc, 0);
> -simm = cpu_ldsw_code(env, dc->pc + 2);
> +simm = cris_fetch(env, dc, dc->pc + 2, 2, 1);
>  simm += 4;
>
>  LOG_DIS("bcc_m: b%s %x\n", cc_name(dc->cond), dc->pc + simm);
> @@ -1185,7 +1173,7 @@ static unsigned int crisv10_decoder(CPUCRISState
> *env, DisasContext *dc)
>  unsigned int insn_len = 2;
>
>  /* Load a halfword onto the instruction register.  */
> -dc->ir = cpu_lduw_code(env, dc->pc);
> +dc->ir = cris_fetch(env, dc, dc->pc, 2, 0);
>
>  /* Now decode it.  */
>  dc->opcode   = EXTRACT_FIELD(dc->ir, 6, 9);
> --
> 2.34.1
>
>

Re: [PATCH 27/32] target/cris: Use translator_ld* in cris_fetch

[Edgar E. Iglesias]

On Fri, Apr 5, 2024 at 12:25 PM Richard Henderson <
richard.hender...@linaro.org> wrote:

> Cc: Edgar E. Iglesias 
> Signed-off-by: Richard Henderson 
>


Reviewed-by: Edgar E. Iglesias 


> ---
>  target/cris/translate.c | 25 -
>  1 file changed, 8 insertions(+), 17 deletions(-)
>
> diff --git a/target/cris/translate.c b/target/cris/translate.c
> index b5410189d4..bb2d6612ba 100644
> --- a/target/cris/translate.c
> +++ b/target/cris/translate.c
> @@ -222,37 +222,28 @@ static int sign_extend(unsigned int val, unsigned
> int width)
>  }
>
>  static int cris_fetch(CPUCRISState *env, DisasContext *dc, uint32_t addr,
> -  unsigned int size, unsigned int sign)
> +  unsigned int size, bool sign)
>  {
>  int r;
>
>  switch (size) {
>  case 4:
> -{
> -r = cpu_ldl_code(env, addr);
> +r = translator_ldl(env, >base, addr);
>  break;
> -}
>  case 2:
> -{
> +r = translator_lduw(env, >base, addr);
>  if (sign) {
> -r = cpu_ldsw_code(env, addr);
> -} else {
> -r = cpu_lduw_code(env, addr);
> +r = (int16_t)r;
>  }
>  break;
> -}
>  case 1:
> -{
> +r = translator_ldub(env, >base, addr);
>  if (sign) {
> -r = cpu_ldsb_code(env, addr);
> -} else {
> -r = cpu_ldub_code(env, addr);
> +r = (int8_t)r;
>  }
>  break;
> -}
>  default:
> -cpu_abort(CPU(dc->cpu), "Invalid fetch size %d\n", size);
> -break;
> +g_assert_not_reached();
>  }
>  return r;
>  }
> @@ -2868,7 +2859,7 @@ static unsigned int crisv32_decoder(CPUCRISState
> *env, DisasContext *dc)
>  int i;
>
>  /* Load a halfword onto the instruction register.  */
> -dc->ir = cris_fetch(env, dc, dc->pc, 2, 0);
> +dc->ir = cris_fetch(env, dc, dc->pc, 2, 0);
>
>  /* Now decode it.  */
>  dc->opcode   = EXTRACT_FIELD(dc->ir, 4, 11);
> --
> 2.34.1
>
>

Re: [PATCH-for-9.1 08/21] target/microblaze: Prefix MMU API with 'mb_'

[Edgar E. Iglesias]

On Thu, Mar 21, 2024 at 4:49 PM Philippe Mathieu-Daudé 
wrote:

> MicroBlaze MMU API is exposed in "mmu.h". In order to avoid
> name clashing with other targets, prefix the API with 'mb_'.
>


Reviewed-by: Edgar E. Iglesias 



>
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  target/microblaze/mmu.h   | 10 +-
>  target/microblaze/cpu.c   |  2 +-
>  target/microblaze/helper.c|  4 ++--
>  target/microblaze/mmu.c   | 14 +++---
>  target/microblaze/op_helper.c |  4 ++--
>  5 files changed, 17 insertions(+), 17 deletions(-)
>
> diff --git a/target/microblaze/mmu.h b/target/microblaze/mmu.h
> index 1068bd2d52..5b51e0a9c6 100644
> --- a/target/microblaze/mmu.h
> +++ b/target/microblaze/mmu.h
> @@ -85,10 +85,10 @@ typedef struct {
>  } err;
>  } MicroBlazeMMULookup;
>
> -unsigned int mmu_translate(MicroBlazeCPU *cpu, MicroBlazeMMULookup *lu,
> -   target_ulong vaddr, MMUAccessType rw, int
> mmu_idx);
> -uint32_t mmu_read(CPUMBState *env, bool ea, uint32_t rn);
> -void mmu_write(CPUMBState *env, bool ea, uint32_t rn, uint32_t v);
> -void mmu_init(MicroBlazeMMU *mmu);
> +unsigned int mb_mmu_translate(MicroBlazeCPU *cpu, MicroBlazeMMULookup *lu,
> +  target_ulong vaddr, MMUAccessType rw, int
> mmu_idx);
> +uint32_t mb_mmu_read(CPUMBState *env, bool ea, uint32_t rn);
> +void mb_mmu_write(CPUMBState *env, bool ea, uint32_t rn, uint32_t v);
> +void mb_mmu_init(MicroBlazeMMU *mmu);
>
>  #endif
> diff --git a/target/microblaze/cpu.c b/target/microblaze/cpu.c
> index 96c2b71f7f..59bfb5c45d 100644
> --- a/target/microblaze/cpu.c
> +++ b/target/microblaze/cpu.c
> @@ -205,7 +205,7 @@ static void mb_cpu_reset_hold(Object *obj)
>  mb_cpu_write_msr(env, MSR_EE | MSR_IE | MSR_VM | MSR_UM);
>  #else
>  mb_cpu_write_msr(env, 0);
> -mmu_init(>mmu);
> +mb_mmu_init(>mmu);
>  #endif
>  }
>
> diff --git a/target/microblaze/helper.c b/target/microblaze/helper.c
> index d25c9eb4d3..961687bae7 100644
> --- a/target/microblaze/helper.c
> +++ b/target/microblaze/helper.c
> @@ -57,7 +57,7 @@ bool mb_cpu_tlb_fill(CPUState *cs, vaddr address, int
> size,
>  return true;
>  }
>
> -hit = mmu_translate(cpu, , address, access_type, mmu_idx);
> +hit = mb_mmu_translate(cpu, , address, access_type, mmu_idx);
>  if (likely(hit)) {
>  uint32_t vaddr = address & TARGET_PAGE_MASK;
>  uint32_t paddr = lu.paddr + vaddr - lu.vaddr;
> @@ -238,7 +238,7 @@ hwaddr mb_cpu_get_phys_page_attrs_debug(CPUState *cs,
> vaddr addr,
>  attrs->secure = mb_cpu_access_is_secure(cpu, MMU_DATA_LOAD);
>
>  if (mmu_idx != MMU_NOMMU_IDX) {
> -hit = mmu_translate(cpu, , addr, 0, 0);
> +hit = mb_mmu_translate(cpu, , addr, 0, 0);
>  if (hit) {
>  vaddr = addr & TARGET_PAGE_MASK;
>  paddr = lu.paddr + vaddr - lu.vaddr;
> diff --git a/target/microblaze/mmu.c b/target/microblaze/mmu.c
> index 234006634e..5fb8ee8418 100644
> --- a/target/microblaze/mmu.c
> +++ b/target/microblaze/mmu.c
> @@ -74,8 +74,8 @@ static void mmu_change_pid(CPUMBState *env, unsigned int
> newpid)
>  }
>
>  /* rw - 0 = read, 1 = write, 2 = fetch.  */
> -unsigned int mmu_translate(MicroBlazeCPU *cpu, MicroBlazeMMULookup *lu,
> -   target_ulong vaddr, MMUAccessType rw, int
> mmu_idx)
> +unsigned int mb_mmu_translate(MicroBlazeCPU *cpu, MicroBlazeMMULookup *lu,
> +  target_ulong vaddr, MMUAccessType rw, int
> mmu_idx)
>  {
>  MicroBlazeMMU *mmu = >env.mmu;
>  unsigned int i, hit = 0;
> @@ -175,7 +175,7 @@ done:
>  }
>
>  /* Writes/reads to the MMU's special regs end up here.  */
> -uint32_t mmu_read(CPUMBState *env, bool ext, uint32_t rn)
> +uint32_t mb_mmu_read(CPUMBState *env, bool ext, uint32_t rn)
>  {
>  MicroBlazeCPU *cpu = env_archcpu(env);
>  unsigned int i;
> @@ -228,7 +228,7 @@ uint32_t mmu_read(CPUMBState *env, bool ext, uint32_t
> rn)
>  return r;
>  }
>
> -void mmu_write(CPUMBState *env, bool ext, uint32_t rn, uint32_t v)
> +void mb_mmu_write(CPUMBState *env, bool ext, uint32_t rn, uint32_t v)
>  {
>  MicroBlazeCPU *cpu = env_archcpu(env);
>  uint64_t tmp64;
> @@ -304,8 +304,8 @@ void mmu_write(CPUMBState *env, bool ext, uint32_t rn,
> uint32_t v)
>  return;
>  }
>
> -hit = mmu_translate(cpu, , v & TLB_EPN_MASK,
> -0, cpu_mmu_index(env_cpu(env), false));
> +hit = mb_mmu_translate(cpu, , v & TLB_EPN_MASK,
> +   0, cpu_mmu_ind

Re: [RFC PATCH-for-9.1 8/8] target/microblaze: Widen $ear to 64-bit

[Edgar E. Iglesias]

On Tue, Mar 19, 2024 at 07:28:55AM +0100, Philippe Mathieu-Daudé wrote:
> The Exception Address Register is 64-bit wide.
> User emulation only access the 32 lower bits.


Reviewed-by: Edgar E. Iglesias 

> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  target/microblaze/cpu.h   | 2 +-
>  linux-user/elfload.c  | 2 +-
>  target/microblaze/gdbstub.c   | 2 +-
>  target/microblaze/translate.c | 2 +-
>  4 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/target/microblaze/cpu.h b/target/microblaze/cpu.h
> index c3e2aba0ec..a9f93b37b7 100644
> --- a/target/microblaze/cpu.h
> +++ b/target/microblaze/cpu.h
> @@ -251,7 +251,7 @@ struct CPUArchState {
>  uint32_t pc;
>  uint32_t msr;/* All bits of MSR except MSR[C] and MSR[CC] */
>  uint32_t msr_c;  /* MSR[C], in low bit; other bits must be 0 */
> -target_ulong ear;
> +uint64_t ear;
>  uint32_t esr;
>  uint32_t fsr;
>  uint32_t btr;
> diff --git a/linux-user/elfload.c b/linux-user/elfload.c
> index 60cf55b36c..4612aef95a 100644
> --- a/linux-user/elfload.c
> +++ b/linux-user/elfload.c
> @@ -1498,7 +1498,7 @@ static void elf_core_copy_regs(target_elf_gregset_t 
> *regs, const CPUMBState *env
>  (*regs)[pos++] = tswapreg(env->pc);
>  (*regs)[pos++] = tswapreg(mb_cpu_read_msr(env));
>  (*regs)[pos++] = 0;
> -(*regs)[pos++] = tswapreg(env->ear);
> +(*regs)[pos++] = tswapreg((uint32_t)env->ear);
>  (*regs)[pos++] = 0;
>  (*regs)[pos++] = tswapreg(env->esr);
>  }
> diff --git a/target/microblaze/gdbstub.c b/target/microblaze/gdbstub.c
> index 09d74e164d..147d20c3e4 100644
> --- a/target/microblaze/gdbstub.c
> +++ b/target/microblaze/gdbstub.c
> @@ -63,7 +63,7 @@ int mb_cpu_gdb_read_register(CPUState *cs, GByteArray 
> *mem_buf, int n)
>  val = mb_cpu_read_msr(env);
>  break;
>  case GDB_EAR:
> -val = env->ear;
> +val = (uint32_t)env->ear;
>  break;
>  case GDB_ESR:
>  val = env->esr;
> diff --git a/target/microblaze/translate.c b/target/microblaze/translate.c
> index 493850c544..19b180501f 100644
> --- a/target/microblaze/translate.c
> +++ b/target/microblaze/translate.c
> @@ -1835,7 +1835,7 @@ void mb_cpu_dump_state(CPUState *cs, FILE *f, int flags)
>  }
>  
>  qemu_fprintf(f, "\nesr=0x%04x fsr=0x%02x btr=0x%08x edr=0x%x\n"
> - "ear=0x" TARGET_FMT_lx " slr=0x%x shr=0x%x\n",
> + "ear=0x%" PRIx64 " slr=0x%x shr=0x%x\n",
>   env->esr, env->fsr, env->btr, env->edr,
>   env->ear, env->slr, env->shr);
>  
> -- 
> 2.41.0
> 

Re: [PATCH-for-9.1 7/8] target/microblaze: Move MMU helpers to sys_helper.c

[Edgar E. Iglesias]

On Tue, Mar 19, 2024 at 07:28:54AM +0100, Philippe Mathieu-Daudé wrote:
> MMU helpers are only used during system emulation,
> move them to sys_helper.c.

Reviewed-by: Edgar E. Iglesias 


> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  target/microblaze/op_helper.c  | 48 --
>  target/microblaze/sys_helper.c | 47 +
>  2 files changed, 47 insertions(+), 48 deletions(-)
> 
> diff --git a/target/microblaze/op_helper.c b/target/microblaze/op_helper.c
> index f6378030b7..45dbed4aaa 100644
> --- a/target/microblaze/op_helper.c
> +++ b/target/microblaze/op_helper.c
> @@ -381,51 +381,3 @@ void helper_stackprot(CPUMBState *env, target_ulong addr)
>  cpu_loop_exit_restore(cs, GETPC());
>  }
>  }
> -
> -#if !defined(CONFIG_USER_ONLY)
> -/* Writes/reads to the MMU's special regs end up here.  */
> -uint32_t helper_mmu_read(CPUMBState *env, uint32_t ext, uint32_t rn)
> -{
> -return mmu_read(env, ext, rn);
> -}
> -
> -void helper_mmu_write(CPUMBState *env, uint32_t ext, uint32_t rn, uint32_t v)
> -{
> -mmu_write(env, ext, rn, v);
> -}
> -
> -void mb_cpu_transaction_failed(CPUState *cs, hwaddr physaddr, vaddr addr,
> -   unsigned size, MMUAccessType access_type,
> -   int mmu_idx, MemTxAttrs attrs,
> -   MemTxResult response, uintptr_t retaddr)
> -{
> -MicroBlazeCPU *cpu = MICROBLAZE_CPU(cs);
> -CPUMBState *env = >env;
> -
> -qemu_log_mask(CPU_LOG_INT, "Transaction failed: vaddr 0x%" VADDR_PRIx
> -  " physaddr 0x" HWADDR_FMT_plx " size %d access type %s\n",
> -  addr, physaddr, size,
> -  access_type == MMU_INST_FETCH ? "INST_FETCH" :
> -  (access_type == MMU_DATA_LOAD ? "DATA_LOAD" : 
> "DATA_STORE"));
> -
> -if (!(env->msr & MSR_EE)) {
> -return;
> -}
> -
> -if (access_type == MMU_INST_FETCH) {
> -if (!cpu->cfg.iopb_bus_exception) {
> -return;
> -}
> -env->esr = ESR_EC_INSN_BUS;
> -} else {
> -if (!cpu->cfg.dopb_bus_exception) {
> -return;
> -}
> -env->esr = ESR_EC_DATA_BUS;
> -}
> -
> -env->ear = addr;
> -cs->exception_index = EXCP_HW_EXCP;
> -cpu_loop_exit_restore(cs, retaddr);
> -}
> -#endif
> diff --git a/target/microblaze/sys_helper.c b/target/microblaze/sys_helper.c
> index 5180500354..7531f95ca7 100644
> --- a/target/microblaze/sys_helper.c
> +++ b/target/microblaze/sys_helper.c
> @@ -21,6 +21,7 @@
>  #include "qemu/osdep.h"
>  #include "cpu.h"
>  #include "exec/exec-all.h"
> +#include "exec/helper-proto.h"
>  #include "qemu/host-utils.h"
>  #include "exec/log.h"
>  
> @@ -292,3 +293,49 @@ void mb_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
>  cs->exception_index = EXCP_HW_EXCP;
>  cpu_loop_exit(cs);
>  }
> +
> +/* Writes/reads to the MMU's special regs end up here.  */
> +uint32_t helper_mmu_read(CPUMBState *env, uint32_t ext, uint32_t rn)
> +{
> +return mmu_read(env, ext, rn);
> +}
> +
> +void helper_mmu_write(CPUMBState *env, uint32_t ext, uint32_t rn, uint32_t v)
> +{
> +mmu_write(env, ext, rn, v);
> +}
> +
> +void mb_cpu_transaction_failed(CPUState *cs, hwaddr physaddr, vaddr addr,
> +   unsigned size, MMUAccessType access_type,
> +   int mmu_idx, MemTxAttrs attrs,
> +   MemTxResult response, uintptr_t retaddr)
> +{
> +MicroBlazeCPU *cpu = MICROBLAZE_CPU(cs);
> +CPUMBState *env = >env;
> +
> +qemu_log_mask(CPU_LOG_INT, "Transaction failed: vaddr 0x%" VADDR_PRIx
> +  " physaddr 0x" HWADDR_FMT_plx " size %d access type %s\n",
> +  addr, physaddr, size,
> +  access_type == MMU_INST_FETCH ? "INST_FETCH" :
> +  (access_type == MMU_DATA_LOAD ? "DATA_LOAD" : 
> "DATA_STORE"));
> +
> +if (!(env->msr & MSR_EE)) {
> +return;
> +}
> +
> +if (access_type == MMU_INST_FETCH) {
> +if (!cpu->cfg.iopb_bus_exception) {
> +return;
> +}
> +env->esr = ESR_EC_INSN_BUS;
> +} else {
> +if (!cpu->cfg.dopb_bus_exception) {
> +return;
> +}
> +env->esr = ESR_EC_DATA_BUS;
> +}
> +
> +env->ear = addr;
> +cs->exception_index = EXCP_HW_EXCP;
> +cpu_loop_exit_restore(cs, retaddr);
> +}
> -- 
> 2.41.0
> 

Re: [PATCH-for-9.1 6/8] target/microblaze: Rename helper.c -> sys_helper.c

[Edgar E. Iglesias]

On Tue, Mar 19, 2024 at 07:28:53AM +0100, Philippe Mathieu-Daudé wrote:
> helper.c only contains system emulation helpers,
> rename it as sys_helper.c.
> Adapt meson and remove pointless #ifdef'ry.

Reviewed-by: Edgar E. Iglesias 


> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  target/microblaze/{helper.c => sys_helper.c} | 5 +
>  target/microblaze/meson.build| 2 +-
>  2 files changed, 2 insertions(+), 5 deletions(-)
>  rename target/microblaze/{helper.c => sys_helper.c} (99%)
> 
> diff --git a/target/microblaze/helper.c b/target/microblaze/sys_helper.c
> similarity index 99%
> rename from target/microblaze/helper.c
> rename to target/microblaze/sys_helper.c
> index 3f410fc7b5..5180500354 100644
> --- a/target/microblaze/helper.c
> +++ b/target/microblaze/sys_helper.c
> @@ -1,5 +1,5 @@
>  /*
> - *  MicroBlaze helper routines.
> + *  MicroBlaze system helper routines.
>   *
>   *  Copyright (c) 2009 Edgar E. Iglesias 
>   *  Copyright (c) 2009-2012 PetaLogix Qld Pty Ltd.
> @@ -24,7 +24,6 @@
>  #include "qemu/host-utils.h"
>  #include "exec/log.h"
>  
> -#ifndef CONFIG_USER_ONLY
>  static bool mb_cpu_access_is_secure(MicroBlazeCPU *cpu,
>  MMUAccessType access_type)
>  {
> @@ -266,8 +265,6 @@ bool mb_cpu_exec_interrupt(CPUState *cs, int 
> interrupt_request)
>  return false;
>  }
>  
> -#endif /* !CONFIG_USER_ONLY */
> -
>  void mb_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
>  MMUAccessType access_type,
>  int mmu_idx, uintptr_t retaddr)
> diff --git a/target/microblaze/meson.build b/target/microblaze/meson.build
> index 3ed4fbb67a..013ea542be 100644
> --- a/target/microblaze/meson.build
> +++ b/target/microblaze/meson.build
> @@ -5,7 +5,6 @@ microblaze_ss.add(gen)
>  microblaze_ss.add(files(
>'cpu.c',
>'gdbstub.c',
> -  'helper.c',
>'op_helper.c',
>'translate.c',
>  ))
> @@ -14,6 +13,7 @@ microblaze_system_ss = ss.source_set()
>  microblaze_system_ss.add(files(
>'mmu.c',
>'machine.c',
> +  'sys_helper.c',
>  ))
>  
>  target_arch += {'microblaze': microblaze_ss}
> -- 
> 2.41.0
> 

Re: [PATCH-for-9.1 4/8] target/microblaze: Use 32-bit destination in gen_goto_tb()

[Edgar E. Iglesias]

On Tue, Mar 19, 2024 at 07:28:51AM +0100, Philippe Mathieu-Daudé wrote:
> cpu_pc and jmp_dest are 32-bit.
>

Reviewed-by: Edgar E. Iglesias 

 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  target/microblaze/translate.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/target/microblaze/translate.c b/target/microblaze/translate.c
> index 4e52ef32db..d6a42381bb 100644
> --- a/target/microblaze/translate.c
> +++ b/target/microblaze/translate.c
> @@ -121,7 +121,7 @@ static void gen_raise_hw_excp(DisasContext *dc, uint32_t 
> esr_ec)
>  gen_raise_exception_sync(dc, EXCP_HW_EXCP);
>  }
>  
> -static void gen_goto_tb(DisasContext *dc, int n, target_ulong dest)
> +static void gen_goto_tb(DisasContext *dc, int n, uint32_t dest)
>  {
>  if (translator_use_goto_tb(>base, dest)) {
>  tcg_gen_goto_tb(n);
> -- 
> 2.41.0
> 

Re: [PATCH-for-9.1 3/8] target/microblaze: Widen vaddr in mmu_translate()

[Edgar E. Iglesias]

On Tue, Mar 19, 2024 at 07:28:50AM +0100, Philippe Mathieu-Daudé wrote:
> Use 'vaddr' type for virtual addresses.


Reviewed-by: Edgar E. Iglesias 


> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  target/microblaze/mmu.h | 2 +-
>  target/microblaze/mmu.c | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/target/microblaze/mmu.h b/target/microblaze/mmu.h
> index 1068bd2d52..2aca39c923 100644
> --- a/target/microblaze/mmu.h
> +++ b/target/microblaze/mmu.h
> @@ -86,7 +86,7 @@ typedef struct {
>  } MicroBlazeMMULookup;
>  
>  unsigned int mmu_translate(MicroBlazeCPU *cpu, MicroBlazeMMULookup *lu,
> -   target_ulong vaddr, MMUAccessType rw, int 
> mmu_idx);
> +   vaddr vaddr, MMUAccessType rw, int mmu_idx);
>  uint32_t mmu_read(CPUMBState *env, bool ea, uint32_t rn);
>  void mmu_write(CPUMBState *env, bool ea, uint32_t rn, uint32_t v);
>  void mmu_init(MicroBlazeMMU *mmu);
> diff --git a/target/microblaze/mmu.c b/target/microblaze/mmu.c
> index 234006634e..eb7c683020 100644
> --- a/target/microblaze/mmu.c
> +++ b/target/microblaze/mmu.c
> @@ -75,7 +75,7 @@ static void mmu_change_pid(CPUMBState *env, unsigned int 
> newpid)
>  
>  /* rw - 0 = read, 1 = write, 2 = fetch.  */
>  unsigned int mmu_translate(MicroBlazeCPU *cpu, MicroBlazeMMULookup *lu,
> -   target_ulong vaddr, MMUAccessType rw, int mmu_idx)
> +   vaddr vaddr, MMUAccessType rw, int mmu_idx)
>  {
>  MicroBlazeMMU *mmu = >env.mmu;
>  unsigned int i, hit = 0;
> -- 
> 2.41.0
> 

Re: [PATCH-for-9.1 2/8] target/microblaze: Use hwaddr/vaddr in cpu_get_phys_page_attrs_debug()

[Edgar E. Iglesias]

On Tue, Mar 19, 2024 at 07:28:49AM +0100, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé 

Reviewed-by: Edgar E. Iglesias 


> ---
>  target/microblaze/helper.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/target/microblaze/helper.c b/target/microblaze/helper.c
> index 0a12c4ea94..3f410fc7b5 100644
> --- a/target/microblaze/helper.c
> +++ b/target/microblaze/helper.c
> @@ -228,7 +228,8 @@ hwaddr mb_cpu_get_phys_page_attrs_debug(CPUState *cs, 
> vaddr addr,
>  MemTxAttrs *attrs)
>  {
>  MicroBlazeCPU *cpu = MICROBLAZE_CPU(cs);
> -target_ulong vaddr, paddr = 0;
> +vaddr vaddr;
> +hwaddr paddr = 0;
>  MicroBlazeMMULookup lu;
>  int mmu_idx = cpu_mmu_index(cs, false);
>  unsigned int hit;
> -- 
> 2.41.0
> 

Re: [PATCH-for-9.1 1/8] target/microblaze: Use correct string format in do_unaligned_access()

[Edgar E. Iglesias]

On Tue, Mar 19, 2024 at 07:28:48AM +0100, Philippe Mathieu-Daudé wrote:
> 'addr' is of type 'vaddr'; no need to cast, use the
> corresponding format string.
> 
> Fixes: ab0c8d0f5b ("target/microblaze: Use cc->do_unaligned_access")
> Signed-off-by: Philippe Mathieu-Daudé 

Reviewed-by: Edgar E. Iglesias 


> ---
>  target/microblaze/helper.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/target/microblaze/helper.c b/target/microblaze/helper.c
> index d25c9eb4d3..0a12c4ea94 100644
> --- a/target/microblaze/helper.c
> +++ b/target/microblaze/helper.c
> @@ -279,8 +279,8 @@ void mb_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
>  iflags = cpu->env.iflags;
>  
>  qemu_log_mask(CPU_LOG_INT,
> -  "Unaligned access addr=" TARGET_FMT_lx " pc=%x 
> iflags=%x\n",
> -  (target_ulong)addr, cpu->env.pc, iflags);
> +  "Unaligned access addr=0x%"VADDR_PRIx" pc=0x%x 
> iflags=0x%x\n",
> +  addr, cpu->env.pc, iflags);
>  
>  esr = ESR_EC_UNALIGNED_DATA;
>  if (likely(iflags & ESR_ESS_FLAG)) {
> -- 
> 2.41.0
> 

Re: [PATCH-for-9.1 5/8] target/microblaze: Restrict 64-bit 'res_addr' to system emulation

[Edgar E. Iglesias]

On Tue, Mar 19, 2024 at 07:28:52AM +0100, Philippe Mathieu-Daudé wrote:
> 'res_addr' is only used in system emulation, where we have
> TARGET_LONG_BITS = 64, so we can directly use the native
> uint64_t type instead of target_ulong.


Hi Philippe,

This breaks linux-user, lwx and swx are valid user-mode insns.

Best regards,
Edgar


> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  target/microblaze/cpu.h   | 10 +-
>  target/microblaze/cpu.c   |  2 ++
>  target/microblaze/machine.c   |  2 +-
>  target/microblaze/translate.c |  9 +++--
>  4 files changed, 15 insertions(+), 8 deletions(-)
> 
> diff --git a/target/microblaze/cpu.h b/target/microblaze/cpu.h
> index c0c7574dbd..c3e2aba0ec 100644
> --- a/target/microblaze/cpu.h
> +++ b/target/microblaze/cpu.h
> @@ -260,11 +260,6 @@ struct CPUArchState {
>  /* Stack protectors. Yes, it's a hw feature.  */
>  uint32_t slr, shr;
>  
> -/* lwx/swx reserved address */
> -#define RES_ADDR_NONE 0x /* Use 0x to indicate no 
> reservation */
> -target_ulong res_addr;
> -uint32_t res_val;
> -
>  /* Internal flags.  */
>  #define IMM_FLAG(1 << 0)
>  #define BIMM_FLAG   (1 << 1)
> @@ -286,6 +281,11 @@ struct CPUArchState {
>  uint32_t iflags;
>  
>  #if !defined(CONFIG_USER_ONLY)
> +/* lwx/swx reserved address */
> +#define RES_ADDR_NONE 0x /* Use 0x to indicate no 
> reservation */
> +uint64_t res_addr;
> +uint32_t res_val;
> +
>  /* Unified MMU.  */
>  MicroBlazeMMU mmu;
>  #endif
> diff --git a/target/microblaze/cpu.c b/target/microblaze/cpu.c
> index 96c2b71f7f..9e393cf217 100644
> --- a/target/microblaze/cpu.c
> +++ b/target/microblaze/cpu.c
> @@ -193,7 +193,9 @@ static void mb_cpu_reset_hold(Object *obj)
>  }
>  
>  memset(env, 0, offsetof(CPUMBState, end_reset_fields));
> +#ifndef CONFIG_USER_ONLY
>  env->res_addr = RES_ADDR_NONE;
> +#endif
>  
>  /* Disable stack protector.  */
>  env->shr = ~0;
> diff --git a/target/microblaze/machine.c b/target/microblaze/machine.c
> index 51705e4f5c..4daf8a2471 100644
> --- a/target/microblaze/machine.c
> +++ b/target/microblaze/machine.c
> @@ -78,7 +78,7 @@ static const VMStateField vmstate_env_fields[] = {
>  VMSTATE_UINT32(iflags, CPUMBState),
>  
>  VMSTATE_UINT32(res_val, CPUMBState),
> -VMSTATE_UINTTL(res_addr, CPUMBState),
> +VMSTATE_UINT64(res_addr, CPUMBState),
>  
>  VMSTATE_STRUCT(mmu, CPUMBState, 0, vmstate_mmu, MicroBlazeMMU),
>  
> diff --git a/target/microblaze/translate.c b/target/microblaze/translate.c
> index d6a42381bb..493850c544 100644
> --- a/target/microblaze/translate.c
> +++ b/target/microblaze/translate.c
> @@ -1872,7 +1872,9 @@ void mb_tcg_init(void)
>  SP(iflags),
>  SP(bvalue),
>  SP(btarget),
> +#if !defined(CONFIG_USER_ONLY)
>  SP(res_val),
> +#endif
>  };
>  
>  #undef R
> @@ -1883,6 +1885,9 @@ void mb_tcg_init(void)
>tcg_global_mem_new_i32(tcg_env, i32s[i].ofs, i32s[i].name);
>  }
>  
> -cpu_res_addr =
> -tcg_global_mem_new(tcg_env, offsetof(CPUMBState, res_addr), 
> "res_addr");
> +#if !defined(CONFIG_USER_ONLY)
> +cpu_res_addr = tcg_global_mem_new_i64(tcg_env,
> +  offsetof(CPUMBState, res_addr),
> +  "res_addr");
> +#endif
>  }
> -- 
> 2.41.0
> 

Running x86 hypervisors inside QEMU x86 TCG outer guest?

[Edgar E. Iglesias]

Hi!

I'm trying to figure out if QEMU supports running x86 hypervisors (KVM and Xen) 
inside emulated (TCG) x86_64 guests.
I.e, using TCG emulated x86 hardware virtualization (Intel VMX/EPT or AMD 
SVM/NPT).

So far, I've been able to run nested x86 KVM when the host is on KVM but no 
luck with TCG.
With intel TCG cores kvm-ok in the guest says there's no KVM support.
With AMD TCG cores, kvm-ok looks fine but as soon as I try to run a guest, it 
doesn't run anything.

Am I doing something wrong or is this expected?

Some of the command-lines I've tried:
This is the command-line I use for AMD like outer TCG guest:
${QEMU} -M q35,accel=tcg -m 6G \
-cpu "EPYC-Rome-v2" \
-smp 1 \
-display none \
-serial mon:stdio \
-kernel ubuntu-23.10-server-cloudimg-amd64-vmlinuz-generic \
-append "root=/dev/sda1 console=ttyS0 intel_iommu=on" \
-initrd ubuntu-23.10-server-cloudimg-amd64-initrd-generic \
-device intel-iommu,intremap=on,device-iotlb=on \
-device virtio-net-pci,netdev=net0,romfile="" \
-netdev type=user,id=net0,hostfwd=tcp::2228-:22 \
-drive file=ubuntu-23.10-server-cloudimg-amd64.img \
-drive file=cloudimg-user-data.img,format=raw,snapshot=yes \
-drive file=hd.qcow2 \
$*

This is the one for Intel TCG guests:
${QEMU} -M q35,accel=tcg -m 6G \
-cpu "Haswell,vmx=on" \
-smp 1 \
-display none \
-serial mon:stdio \
-kernel ubuntu-23.10-server-cloudimg-amd64-vmlinuz-generic \
-append "root=/dev/sda1 console=ttyS0 intel_iommu=on" \
-initrd ubuntu-23.10-server-cloudimg-amd64-initrd-generic \
-device intel-iommu,intremap=on,device-iotlb=on \
-device virtio-net-pci,netdev=net0,romfile="" \
-netdev type=user,id=net0,hostfwd=tcp::2228-:22 \
-drive file=ubuntu-23.10-server-cloudimg-amd64.img \
-drive file=cloudimg-user-data.img,format=raw,snapshot=yes \
-drive file=hd.qcow2 \
$*

This is the one that works (nested KVM):
${QEMU} -M q35,accel=kvm,kernel-irqchip=split -m 6G \
-cpu host \
-smp 1 \
-display none \
-serial mon:stdio \
-kernel ubuntu-23.10-server-cloudimg-amd64-vmlinuz-generic \
-append "root=/dev/sda1 console=ttyS0 intel_iommu=on" \
-initrd ubuntu-23.10-server-cloudimg-amd64-initrd-generic \
-device intel-iommu,intremap=on,device-iotlb=on \
-device virtio-net-pci,netdev=net0,romfile="" \
-netdev type=user,id=net0,hostfwd=tcp::2228-:22 \
-drive file=ubuntu-23.10-server-cloudimg-amd64.img \
-drive file=cloudimg-user-data.img,format=raw,snapshot=yes \
-drive file=hd.qcow2 \
$*

Best regards,
Edgar

Re: [PATCH] hw/misc: zynq_slcr: set SLC_RST bit in REBOOT_STATUS register

[Edgar E. Iglesias]

On Fri, Mar 1, 2024 at 10:40 AM Peter Maydell 
wrote:

> On Wed, 28 Feb 2024 at 01:40, Gregory Anders  wrote:
> >
> > When the CPU is reset using PSS_RST_CTRL in the SLCR, bit 19 in
> > REBOOT_STATUS should be set.
> >

> Refer to page 1602 of the Xilinx Zynq 7000 Technical Reference Manual.
> >
> > Signed-off-by: Gregory Anders 
> > ---
> >  hw/misc/zynq_slcr.c | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/hw/misc/zynq_slcr.c b/hw/misc/zynq_slcr.c
> > index d2ac2e77f2..a8f1792bf6 100644
> > --- a/hw/misc/zynq_slcr.c
> > +++ b/hw/misc/zynq_slcr.c
> > @@ -120,6 +120,7 @@ REG32(RS_AWDT_CTRL, 0x24c)
> >  REG32(RST_REASON, 0x250)
> >
> >  REG32(REBOOT_STATUS, 0x258)
> > +FIELD(REBOOT_STATUS, SLC_RST, 19, 1)
> >  REG32(BOOT_MODE, 0x25c)
> >
> >  REG32(APU_CTRL, 0x300)
> > @@ -562,6 +563,7 @@ static void zynq_slcr_write(void *opaque, hwaddr
> offset,
> >  switch (offset) {
> >  case R_PSS_RST_CTRL:
> >  if (FIELD_EX32(val, PSS_RST_CTRL, SOFT_RST)) {
> > +s->regs[R_REBOOT_STATUS] |= R_REBOOT_STATUS_SLC_RST_MASK;
> >  qemu_system_reset_request(SHUTDOWN_CAUSE_GUEST_RESET);
> >  }
> >  break;
> > --
>
> The manual also says that "This field is written by ROM code",
> so as a model of the hardware should QEMU be writing it?
>
> I've cc'd the Zynq maintainers for their opinion.
>
>
Hi,

I don't have great answers unfortunately...

We haven't been super consistent with these things but on the ZynqMP we
sometimes require the user to apply ROM behaviour using -device loader on
the command-line (not great for this case since we wouldn't want the mask
to be set until a soft reset) or we conditionalize the ROM behaviour
checking if we're doing direct Linux boots..

Best regards,
Edgar




> thanks
> -- PMM
>

Re: [PATCH 04/31] docs: mark CRIS support as deprecated

[Edgar E. Iglesias]

On Wed, Jan 24, 2024 at 12:06 PM Philippe Mathieu-Daudé 
wrote:

> On 25/9/23 19:17, Edgar E. Iglesias wrote:
> >
> > On Mon, Sep 25, 2023 at 7:00 PM Alex Bennée  > <mailto:alex.ben...@linaro.org>> wrote:
> >
> >
> > Daniel P. Berrangé  > <mailto:berra...@redhat.com>> writes:
> >
> >  > On Mon, Sep 25, 2023 at 03:48:27PM +0100, Alex Bennée wrote:
> >  >> This might be premature but while streamling the avocado tests I
> >  >> realised the only tests we have are "check-tcg" ones. The aging
> >  >> fedora-criss-cross image works well enough for developers but
> > can't be
> >  >> used in CI as we need supported build platforms to build QEMU.
> >  >>
> >  >> Does this mean the writing is on the wall for this architecture?
> >  >>
> >  >> Signed-off-by: Alex Bennée  > <mailto:alex.ben...@linaro.org>>
> >  >> Cc: Rabin Vincent mailto:rab...@axis.com>>
> >  >> Cc: Edgar E. Iglesias  > <mailto:edgar.igles...@xilinx.com>>
> >  >> ---
> >  >>  docs/about/deprecated.rst | 11 +++
> >  >>  1 file changed, 11 insertions(+)
> >  >>
> >  >> diff --git a/docs/about/deprecated.rst
> b/docs/about/deprecated.rst
> >  >> index dc4da95329..7cfe313aa6 100644
> >  >> --- a/docs/about/deprecated.rst
> >  >> +++ b/docs/about/deprecated.rst
> >  >> @@ -399,6 +399,17 @@ Specifying the iSCSI password in plain text
> > on the command line using the
> >  >>  used instead, to refer to a ``--object secret...`` instance
> > that provides
> >  >>  a password via a file, or encrypted.
> >  >>
> >  >> +TCG CPUs
> >  >> +
> >  >> +
> >  >> +CRIS CPU architecture (since 8.1)
> >  >> +'
> >  >> +
> >  >> +The CRIS architecture was pulled from Linux in 4.17 and the
> > compiler
> >  >> +is no longer packaged in any distro making it harder to run the
> >  >> +``check-tcg`` tests. Unless we can improve the testing
> > situation there
> >  >> +is a chance the code will bitrot without anyone noticing.
> >  >
> >  > Deprecated is generally a warning that we intend to delete the
> >  > feature.   If we're just going to relegate it to untested
> >  > status (what I'd call "tier 3" quality), then we should document
> >  > that elsewhere.  I don't mind which way we go.
> >
> > We do have reasonably good coverage with tests/tcg/cris but of course
> > without a compiler we can't build them.
> >
> > Both nios2 and microblaze have build-toolchain scripts which can be
> used
> > to re-create containers. However my preference is having pre-built
> > toolchains hosted by others like we do for loongarch, hexagon, xtensa
> > and tricore. Then the docker image can simply curl them into an
> image.
> >
> >
> > Yeah, I guess it's time to deprecate it...
>
> Is that an informal Acked-by? (:
>
>
OK with me!

Acked-by: Edgar E. Iglesias 

Re: [PATCH] target/arm: HVC at EL3 should go to EL3, not EL2

[Edgar E. Iglesias]

On Thu, Nov 9, 2023 at 4:20 PM Peter Maydell 
wrote:

> AArch64 permits code at EL3 to use the HVC instruction; however the
> exception we take should go to EL3, not down to EL2 (see the pseudocode
> AArch64.CallHypervisor()). Fix the target EL.
>
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Peter Maydell 
>

Reviewed-by: Edgar E. Iglesias 



> ---
>  target/arm/tcg/translate-a64.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/target/arm/tcg/translate-a64.c
> b/target/arm/tcg/translate-a64.c
> index 41484d8ae54..a2e49c39f9f 100644
> --- a/target/arm/tcg/translate-a64.c
> +++ b/target/arm/tcg/translate-a64.c
> @@ -2351,6 +2351,8 @@ static bool trans_SVC(DisasContext *s, arg_i *a)
>
>  static bool trans_HVC(DisasContext *s, arg_i *a)
>  {
> +int target_el = s->current_el == 3 ? 3 : 2;
> +
>  if (s->current_el == 0) {
>  unallocated_encoding(s);
>  return true;
> @@ -2363,7 +2365,7 @@ static bool trans_HVC(DisasContext *s, arg_i *a)
>  gen_helper_pre_hvc(tcg_env);
>  /* Architecture requires ss advance before we do the actual work */
>  gen_ss_advance(s);
> -gen_exception_insn_el(s, 4, EXCP_HVC, syn_aa64_hvc(a->imm), 2);
> +gen_exception_insn_el(s, 4, EXCP_HVC, syn_aa64_hvc(a->imm),
> target_el);
>  return true;
>  }
>
> --
> 2.34.1
>
>
>

Re: [PATCH 2/9] target/cris: Use tcg_gen_extract_tl

[Edgar E. Iglesias]

On Tue, Oct 24, 2023 at 1:36 AM Richard Henderson <
richard.hender...@linaro.org> wrote:

> On 10/23/23 09:09, Philippe Mathieu-Daudé wrote:
> > Inspired-by: Richard Henderson 
> > Signed-off-by: Philippe Mathieu-Daudé 
> > ---
> >   target/cris/translate.c | 3 +--
> >   target/i386/tcg/translate.c | 9 +++--
> >   2 files changed, 4 insertions(+), 8 deletions(-)
>
> Accidental merge of two patches, but they both look fine.  :-)
>
> With the split, to both:
> Reviewed-by: Richard Henderson 
>
>
>
CRIS part looks good with Richard's request to split:
Reviewed-by: Edgar E. Iglesias 



> r~
>

Re: [RFC PATCH 8/9] target/cris: Use tcg_gen_sextract_tl

[Edgar E. Iglesias]

On Tue, Oct 24, 2023 at 2:26 AM Richard Henderson <
richard.hender...@linaro.org> wrote:

> On 10/23/23 09:09, Philippe Mathieu-Daudé wrote:
> > Inspired-by: Richard Henderson 
> > Signed-off-by: Philippe Mathieu-Daudé 
> > ---
> > RFC: please double-check bits
> > ---
> >   target/cris/translate.c | 3 +--
> >   1 file changed, 1 insertion(+), 2 deletions(-)
> >
> > diff --git a/target/cris/translate.c b/target/cris/translate.c
> > index 65b07e1d80..3a161f8f73 100644
> > --- a/target/cris/translate.c
> > +++ b/target/cris/translate.c
> > @@ -336,8 +336,7 @@ static void t_gen_cris_mstep(TCGv d, TCGv a, TCGv b,
> TCGv ccs)
> >*/
> >   t = tcg_temp_new();
> >   tcg_gen_shli_tl(d, a, 1);
> > -tcg_gen_shli_tl(t, ccs, 31 - 3);
> > -tcg_gen_sari_tl(t, t, 31);
> > +tcg_gen_sextract_tl(t, ccs, 3, 1);


> tcg_gen_sextract_tl(t, ccs, ctz32(N_FLAG), 1);
>

Looks good!

I think the intent was a branch-free version of:
if (ccs & N_FLAG) {
d += b;
}

Or:
t = ccs & N_FLAG ? UINT32_MAX : 0;
d += b & t;



>
> Also, it appears t_gen_cris_mstep consumes CCS without making sure that it
> is up-to-date.
> Edgar?
>
>
Yes, that looks suspicious!

Best regards,
Edgar


>
> r~
>