[Qemu-devel] [RFC 0/5] arm: Exclude reserved memory regions of iommu to avoid

[zhuyijun]

From: Zhu Yijun 

With kernel 4.11, iommu/smmu will populate the MSI IOVA reserved window and
PCI reserved window which has to be excluded from Guest iova allocations.

And on certain HiSilicon platforms (hip06/hip07), the GIC ITS and PCIe RC 
deviates from the standard implementation will reserve the hw msi regions in
the smmu-v3 driver which means these address regions will not be translated.

https://www.spinics.net/lists/linux-pci/msg65125.html

On such platforms, reserved memory regions will export like this:

root:~# cat /sys/kernel/iommu_groups/7/reserved_regions
0xa880 0xaffe reserved
0xc600 0xc601 msi

However, it falls within the Qemu default virtual memory address space.

Take a look at hw/arm/virt.c:

[VIRT_MEM] = { 0x4000, RAMLIMIT_BYTES },
. . .

memory_region_allocate_system_memory(ram, NULL, "mach-virt.ram",
 machine->ram_size);
memory_region_add_subregion(sysmem, vms->memmap[VIRT_MEM].base, ram);

So suppose we allocate 4GB mem to a VM, there is a chance that the reserved
regions will get allocated for a Guest VF DMA iova and it will fail.

This patchset create holes in the Qemu RAM address space which exclude
the sysfs exported regions.


Zhu Yijun (5):
  hw/vfio: Add function for getting reserved_region of device iommu
group
  hw/arm/virt: Enable dynamic generation of guest RAM memory regions
  hw/arm: add scattered RAM memory region support
  hw/arm/boot: set fdt size cell of memory node from mem_list
  hw/arm/virt-acpi-build: Build srat table according to mem_list

 hw/arm/boot.c | 155 +++---
 hw/arm/virt-acpi-build.c  |  40 +--
 hw/arm/virt.c | 120 ++--
 hw/vfio/common.c  |  67 ++
 hw/vfio/pci.c |   2 +
 hw/vfio/platform.c|   2 +
 include/exec/memory.h |   7 ++
 include/hw/arm/arm.h  |   1 +
 include/hw/arm/virt.h |   1 +
 include/hw/vfio/vfio-common.h |   3 +
 10 files changed, 347 insertions(+), 51 deletions(-)

-- 
1.8.3.1

Re: [Qemu-devel] [RFC 0/5] arm: Exclude reserved memory regions of iommu to avoid

[Auger Eric]

Hello Zhu,

On 13/11/2017 13:29, zhuyi...@huawei.com wrote:
> From: Zhu Yijun 
> 
> With kernel 4.11, iommu/smmu will populate the MSI IOVA reserved window and
> PCI reserved window which has to be excluded from Guest iova allocations.
> 
> And on certain HiSilicon platforms (hip06/hip07), the GIC ITS and PCIe RC 
> deviates from the standard implementation will reserve the hw msi regions in
> the smmu-v3 driver which means these address regions will not be translated.
> 
> https://www.spinics.net/lists/linux-pci/msg65125.html
> 
> On such platforms, reserved memory regions will export like this:
> 
> root:~# cat /sys/kernel/iommu_groups/7/reserved_regions
> 0xa880 0xaffe reserved
> 0xc600 0xc601 msi
> 
> However, it falls within the Qemu default virtual memory address space.
> 
> Take a look at hw/arm/virt.c:
> 
> [VIRT_MEM] = { 0x4000, RAMLIMIT_BYTES },
> . . .
> 
> memory_region_allocate_system_memory(ram, NULL, "mach-virt.ram",
>  machine->ram_size);
> memory_region_add_subregion(sysmem, vms->memmap[VIRT_MEM].base, ram);
> 
> So suppose we allocate 4GB mem to a VM, there is a chance that the reserved
> regions will get allocated for a Guest VF DMA iova and it will fail.
> 
> This patchset create holes in the Qemu RAM address space which exclude
> the sysfs exported regions.
> 
> 
> Zhu Yijun (5):
>   hw/vfio: Add function for getting reserved_region of device iommu
> group
>   hw/arm/virt: Enable dynamic generation of guest RAM memory regions
>   hw/arm: add scattered RAM memory region support
>   hw/arm/boot: set fdt size cell of memory node from mem_list
>   hw/arm/virt-acpi-build: Build srat table according to mem_list
I have not received all the patches of this series. I miss 2, 4, 5.

Looking at
https://lists.nongnu.org/archive/html/qemu-devel/2017-11/threads.html#02198,
I don't see them either.

Thanks

Eric
> 
>  hw/arm/boot.c | 155 
> +++---
>  hw/arm/virt-acpi-build.c  |  40 +--
>  hw/arm/virt.c | 120 ++--
>  hw/vfio/common.c  |  67 ++
>  hw/vfio/pci.c |   2 +
>  hw/vfio/platform.c|   2 +
>  include/exec/memory.h |   7 ++
>  include/hw/arm/arm.h  |   1 +
>  include/hw/arm/virt.h |   1 +
>  include/hw/vfio/vfio-common.h |   3 +
>  10 files changed, 347 insertions(+), 51 deletions(-)
> 

[Qemu-devel] [RFC 0/5] arm: Exclude reserved memory regions of iommu to avoid

[zhuyijun]

From: Zhu Yijun 

With kernel 4.11, iommu/smmu will populate the MSI IOVA reserved window and
PCI reserved window which has to be excluded from Guest iova allocations.

And on certain HiSilicon platforms (hip06/hip07), the GIC ITS and PCIe RC 
deviates from the standard implementation will reserve the hw msi regions in
the smmu-v3 driver which means these address regions will not be translated.

https://www.spinics.net/lists/linux-pci/msg65125.html

On such platforms, reserved memory regions will export like this:

root:~# cat /sys/kernel/iommu_groups/7/reserved_regions
0xa880 0xaffe reserved
0xc600 0xc601 msi

However, it falls within the Qemu default virtual memory address space.

Take a look at hw/arm/virt.c:

[VIRT_MEM] = { 0x4000, RAMLIMIT_BYTES },
. . .

memory_region_allocate_system_memory(ram, NULL, "mach-virt.ram",
 machine->ram_size);
memory_region_add_subregion(sysmem, vms->memmap[VIRT_MEM].base, ram);

So suppose we allocate 4GB mem to a VM, there is a chance that the reserved
regions will get allocated for a Guest VF DMA iova and it will fail.

This patchset create holes in the Qemu RAM address space which exclude
the sysfs exported regions.


Zhu Yijun (5):
  hw/vfio: Add function for getting reserved_region of device iommu
group
  hw/arm/virt: Enable dynamic generation of guest RAM memory regions
  hw/arm: add scattered RAM memory region support
  hw/arm/boot: set fdt size cell of memory node from mem_list
  hw/arm/virt-acpi-build: Build srat table according to mem_list

 hw/arm/boot.c | 155 +++---
 hw/arm/virt-acpi-build.c  |  40 +--
 hw/arm/virt.c | 120 ++--
 hw/vfio/common.c  |  67 ++
 hw/vfio/pci.c |   2 +
 hw/vfio/platform.c|   2 +
 include/exec/memory.h |   7 ++
 include/hw/arm/arm.h  |   1 +
 include/hw/arm/virt.h |   1 +
 include/hw/vfio/vfio-common.h |   3 +
 10 files changed, 347 insertions(+), 51 deletions(-)

-- 
1.8.3.1