Re: [Qemu-devel] [PATCH v4 0/5] ARM: Add NUMA support for machine virt

[Andrew Jones]

On Fri, Jan 29, 2016 at 02:52:35PM +0800, Shannon Zhao wrote:
> 
> 
> On 2016/1/29 14:32, Ashok Kumar wrote:
> > Hi, 
> > 
> > On Sat, Jan 23, 2016 at 07:36:41PM +0800, Shannon Zhao wrote:
> >> > From: Shannon Zhao 
> >> > 
> >> > Add NUMA support for machine virt. Tested successfully running a guest
> >> > Linux kernel with the following patch applied:
> >> > 
> >> > - [PATCH v9 0/6] arm64, numa: Add numa support for arm64 platforms
> >> > https://lwn.net/Articles/672329/
> >> > - [PATCH v2 0/4] ACPI based NUMA support for ARM64
> >> > http://www.spinics.net/lists/linux-acpi/msg61795.html
> >> > 
> >> > Changes since v3:
> >> > * based on new kernel driver and device bindings
> >> > * add ACPI part
> >> > 
> >> > Changes since v2:
> >> > * update to use NUMA node property arm,associativity.
> >> > 
> >> > Changes since v1:
> >> > Take into account Peter's comments:
> >> > * rename virt_memory_init to arm_generate_memory_dtb
> >> > * move arm_generate_memory_dtb to boot.c and make it a common func
> >> > * use a struct numa_map to generate numa dtb
> >> > 
> >> > Example qemu command line:
> >> > qemu-system-aarch64 \
> >> > -enable-kvm -smp 4\
> >> > -kernel Image \
> >> > -m 512 -machine virt,kernel_irqchip=on \
> >> > -initrd guestfs.cpio.gz \
> >> > -cpu host -nographic \
> >> > -numa node,mem=256M,cpus=0-1,nodeid=0 \
> >> > -numa node,mem=256M,cpus=2-3,nodeid=1 \
> >> > -append "console=ttyAMA0 root=/dev/ram"
> >> > 
> >> > Shannon Zhao (5):
> >> >   ARM: Virt: Add /distance-map node for NUMA
> >> >   ARM: Virt: Set numa-node-id for CPUs
> >> >   ARM: Add numa-node-id for /memory node
> >> >   include/hw/acpi/acpi-defs: Add GICC Affinity Structure
> >> >   hw/arm/virt-acpi-build: Generate SRAT table
> >> > 
> >> >  hw/arm/boot.c   | 29 ++-
> >> >  hw/arm/virt-acpi-build.c| 58 
> >> > +
> >> >  hw/arm/virt.c   | 37 +
> >> >  hw/i386/acpi-build.c|  2 +-
> >> >  include/hw/acpi/acpi-defs.h | 15 +++-
> >> >  5 files changed, 138 insertions(+), 3 deletions(-)
> >> > 
> >> > -- 
> >> > 2.0.4
> >> > 
> > Don't we need to populate the NUMA node in the Affinity byte of MPIDR?
> > Linux uses the Affinity information in MPIDR to build topology which
> > might go wrong for the guest in this case. 
> > Maybe a non Linux OS might be impacted more?
> > 
> Ah, yes. It needs to update the MPIDR. But currently QEMU uses the value
> from KVM when using KVM. It needs to call kvm_set_one_reg to set the
> MPIDR and I'm not sure if this will affect KVM by looking at following
> comments:
> /*
>  * When KVM is in use, PSCI is emulated in-kernel and not by qemu.
>  * Currently KVM has its own idea about MPIDR assignment, so we
>  * override our defaults with what we get from KVM.
>  */
> 
> Peter, do you have any suggestion?
> 
> > distance-map compatible string has been changed from
> > "numa,distance-map-v1" to "numa-distance-map-v1"
> Will update this.

Sigh... I've had the MPIDR rework (to let QEMU dictate it0 on my TODO list
for a loong time. I'll pick it up today and hopefully have something to
discuss next week. I'll keep this series in mind too.

Thanks,
drew

Re: [Qemu-devel] [PATCH v4 0/5] ARM: Add NUMA support for machine virt

[Ashok Kumar]

Hi, 

On Sat, Jan 23, 2016 at 07:36:41PM +0800, Shannon Zhao wrote:
> From: Shannon Zhao 
> 
> Add NUMA support for machine virt. Tested successfully running a guest
> Linux kernel with the following patch applied:
> 
> - [PATCH v9 0/6] arm64, numa: Add numa support for arm64 platforms
> https://lwn.net/Articles/672329/
> - [PATCH v2 0/4] ACPI based NUMA support for ARM64
> http://www.spinics.net/lists/linux-acpi/msg61795.html
> 
> Changes since v3:
> * based on new kernel driver and device bindings
> * add ACPI part
> 
> Changes since v2:
> * update to use NUMA node property arm,associativity.
> 
> Changes since v1:
> Take into account Peter's comments:
> * rename virt_memory_init to arm_generate_memory_dtb
> * move arm_generate_memory_dtb to boot.c and make it a common func
> * use a struct numa_map to generate numa dtb
> 
> Example qemu command line:
> qemu-system-aarch64 \
> -enable-kvm -smp 4\
> -kernel Image \
> -m 512 -machine virt,kernel_irqchip=on \
> -initrd guestfs.cpio.gz \
> -cpu host -nographic \
> -numa node,mem=256M,cpus=0-1,nodeid=0 \
> -numa node,mem=256M,cpus=2-3,nodeid=1 \
> -append "console=ttyAMA0 root=/dev/ram"
> 
> Shannon Zhao (5):
>   ARM: Virt: Add /distance-map node for NUMA
>   ARM: Virt: Set numa-node-id for CPUs
>   ARM: Add numa-node-id for /memory node
>   include/hw/acpi/acpi-defs: Add GICC Affinity Structure
>   hw/arm/virt-acpi-build: Generate SRAT table
> 
>  hw/arm/boot.c   | 29 ++-
>  hw/arm/virt-acpi-build.c| 58 
> +
>  hw/arm/virt.c   | 37 +
>  hw/i386/acpi-build.c|  2 +-
>  include/hw/acpi/acpi-defs.h | 15 +++-
>  5 files changed, 138 insertions(+), 3 deletions(-)
> 
> -- 
> 2.0.4
> 

Don't we need to populate the NUMA node in the Affinity byte of MPIDR?
Linux uses the Affinity information in MPIDR to build topology which
might go wrong for the guest in this case. 
Maybe a non Linux OS might be impacted more?

distance-map compatible string has been changed from
"numa,distance-map-v1" to "numa-distance-map-v1"

Thanks,
Ashok

Re: [Qemu-devel] [PATCH v4 0/5] ARM: Add NUMA support for machine virt

[Shannon Zhao]

On 2016/1/29 14:32, Ashok Kumar wrote:
> Hi, 
> 
> On Sat, Jan 23, 2016 at 07:36:41PM +0800, Shannon Zhao wrote:
>> > From: Shannon Zhao 
>> > 
>> > Add NUMA support for machine virt. Tested successfully running a guest
>> > Linux kernel with the following patch applied:
>> > 
>> > - [PATCH v9 0/6] arm64, numa: Add numa support for arm64 platforms
>> > https://lwn.net/Articles/672329/
>> > - [PATCH v2 0/4] ACPI based NUMA support for ARM64
>> > http://www.spinics.net/lists/linux-acpi/msg61795.html
>> > 
>> > Changes since v3:
>> > * based on new kernel driver and device bindings
>> > * add ACPI part
>> > 
>> > Changes since v2:
>> > * update to use NUMA node property arm,associativity.
>> > 
>> > Changes since v1:
>> > Take into account Peter's comments:
>> > * rename virt_memory_init to arm_generate_memory_dtb
>> > * move arm_generate_memory_dtb to boot.c and make it a common func
>> > * use a struct numa_map to generate numa dtb
>> > 
>> > Example qemu command line:
>> > qemu-system-aarch64 \
>> > -enable-kvm -smp 4\
>> > -kernel Image \
>> > -m 512 -machine virt,kernel_irqchip=on \
>> > -initrd guestfs.cpio.gz \
>> > -cpu host -nographic \
>> > -numa node,mem=256M,cpus=0-1,nodeid=0 \
>> > -numa node,mem=256M,cpus=2-3,nodeid=1 \
>> > -append "console=ttyAMA0 root=/dev/ram"
>> > 
>> > Shannon Zhao (5):
>> >   ARM: Virt: Add /distance-map node for NUMA
>> >   ARM: Virt: Set numa-node-id for CPUs
>> >   ARM: Add numa-node-id for /memory node
>> >   include/hw/acpi/acpi-defs: Add GICC Affinity Structure
>> >   hw/arm/virt-acpi-build: Generate SRAT table
>> > 
>> >  hw/arm/boot.c   | 29 ++-
>> >  hw/arm/virt-acpi-build.c| 58 
>> > +
>> >  hw/arm/virt.c   | 37 +
>> >  hw/i386/acpi-build.c|  2 +-
>> >  include/hw/acpi/acpi-defs.h | 15 +++-
>> >  5 files changed, 138 insertions(+), 3 deletions(-)
>> > 
>> > -- 
>> > 2.0.4
>> > 
> Don't we need to populate the NUMA node in the Affinity byte of MPIDR?
> Linux uses the Affinity information in MPIDR to build topology which
> might go wrong for the guest in this case. 
> Maybe a non Linux OS might be impacted more?
> 
Ah, yes. It needs to update the MPIDR. But currently QEMU uses the value
from KVM when using KVM. It needs to call kvm_set_one_reg to set the
MPIDR and I'm not sure if this will affect KVM by looking at following
comments:
/*
 * When KVM is in use, PSCI is emulated in-kernel and not by qemu.
 * Currently KVM has its own idea about MPIDR assignment, so we
 * override our defaults with what we get from KVM.
 */

Peter, do you have any suggestion?

> distance-map compatible string has been changed from
> "numa,distance-map-v1" to "numa-distance-map-v1"
Will update this.

Thanks,
-- 
Shannon

[Qemu-devel] [PATCH v4 0/5] ARM: Add NUMA support for machine virt

[Shannon Zhao]

From: Shannon Zhao 

Add NUMA support for machine virt. Tested successfully running a guest
Linux kernel with the following patch applied:

- [PATCH v9 0/6] arm64, numa: Add numa support for arm64 platforms
https://lwn.net/Articles/672329/
- [PATCH v2 0/4] ACPI based NUMA support for ARM64
http://www.spinics.net/lists/linux-acpi/msg61795.html

Changes since v3:
* based on new kernel driver and device bindings
* add ACPI part

Changes since v2:
* update to use NUMA node property arm,associativity.

Changes since v1:
Take into account Peter's comments:
* rename virt_memory_init to arm_generate_memory_dtb
* move arm_generate_memory_dtb to boot.c and make it a common func
* use a struct numa_map to generate numa dtb

Example qemu command line:
qemu-system-aarch64 \
-enable-kvm -smp 4\
-kernel Image \
-m 512 -machine virt,kernel_irqchip=on \
-initrd guestfs.cpio.gz \
-cpu host -nographic \
-numa node,mem=256M,cpus=0-1,nodeid=0 \
-numa node,mem=256M,cpus=2-3,nodeid=1 \
-append "console=ttyAMA0 root=/dev/ram"

Shannon Zhao (5):
  ARM: Virt: Add /distance-map node for NUMA
  ARM: Virt: Set numa-node-id for CPUs
  ARM: Add numa-node-id for /memory node
  include/hw/acpi/acpi-defs: Add GICC Affinity Structure
  hw/arm/virt-acpi-build: Generate SRAT table

 hw/arm/boot.c   | 29 ++-
 hw/arm/virt-acpi-build.c| 58 +
 hw/arm/virt.c   | 37 +
 hw/i386/acpi-build.c|  2 +-
 include/hw/acpi/acpi-defs.h | 15 +++-
 5 files changed, 138 insertions(+), 3 deletions(-)

-- 
2.0.4