Please test the PPA [1] if it would suit your needs. While you are doing so I have two MPs up for fellow packagers to review [2][3]
Bug status incomplete until testing on PPA confirmed to be working. [1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3671 [2]: https://code.launchpad.net/~paelzer/ubuntu/+source/numactl/+git/numactl/+merge/363778 [3]: https://code.launchpad.net/~paelzer/ubuntu/+source/numactl/+git/numactl/+merge/363777 ** Description changed: + Please test the PPA [1] if it would suit your needs. + + https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3671 + + + [Impact] + + * Some less common numa hierarchies break numactl + * backport upstream change to fix that + - https://github.com/numactl/numactl/commit/b60868703 + + [Test Case] + + * You have to create an "affected" Numa topology + E.g. on a full mem stacked P9 create a guest like: + <vcpu placement='static' current='4'>8</vcpu> + <cputune> + <vcpupin vcpu='0' cpuset='64'/> + <vcpupin vcpu='1' cpuset='65'/> + <vcpupin vcpu='2' cpuset='66'/> + <vcpupin vcpu='3' cpuset='67'/> + <emulatorpin cpuset='64-67'/> + </cputune> + <numatune> + <memory mode='strict' nodeset='8'/> + </numatune> + * Then after starting that numactl will fail to show it's data, for + example: + # numastat -c qemu-system-ppc64 + Per-node process memory usage (in MBs) for PID 5738 (qemu-system-ppc) + Node 0 Node 8 ... + ------ ------ ... + Huge 0 0 ... + Heap 0 0 ... + Stack 14 0 ... + Private 936 0 ... + ------- ------ ------ ... + Total 936 0 ... + (more details on the Host used below) + + [Regression Potential] + + * The change is rather small just skipping nodes not matching the + expected numbers (important to not crash). If anything then I can + think of a regression where it would skip a numa node that it + would need to access. But I have seen none such case in my tests, so + I hope it will be fine. + + [Other Info] + + * n/a + + --- + == Comment: #0 - SANTWANA SAMANTRAY <[email protected]> - 2019-02-20 23:48:43 == ---Problem Description--- numastat doesn't display correct information for kvm guests. The guest is configured with vcpupin from the node8: Snippet of the guest XML : <vcpu placement='static' current='4'>8</vcpu> - <cputune> - <vcpupin vcpu='0' cpuset='64'/> - <vcpupin vcpu='1' cpuset='65'/> - <vcpupin vcpu='2' cpuset='66'/> - <vcpupin vcpu='3' cpuset='67'/> - <emulatorpin cpuset='64-67'/> - </cputune> - <numatune> - <memory mode='strict' nodeset='8'/> - </numatune> + <cputune> + <vcpupin vcpu='0' cpuset='64'/> + <vcpupin vcpu='1' cpuset='65'/> + <vcpupin vcpu='2' cpuset='66'/> + <vcpupin vcpu='3' cpuset='67'/> + <emulatorpin cpuset='64-67'/> + </cputune> + <numatune> + <memory mode='strict' nodeset='8'/> + </numatune> == Host Details == - # lscpu + # lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 4 Core(s) per socket: 16 Socket(s): 2 NUMA node(s): 6 Model: 2.2 (pvr 004e 1202) Model name: POWER9, altivec supported CPU max MHz: 3800.0000 CPU min MHz: 2300.0000 L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 10240K NUMA node0 CPU(s): 0-63 NUMA node8 CPU(s): 64-127 - NUMA node252 CPU(s): - NUMA node253 CPU(s): - NUMA node254 CPU(s): - NUMA node255 CPU(s): + NUMA node252 CPU(s): + NUMA node253 CPU(s): + NUMA node254 CPU(s): + NUMA node255 CPU(s): After starting the guest, the numastat doesn't display the guest process to be running in node8. # numastat -c qemu-system-ppc64 Per-node process memory usage (in MBs) for PID 5738 (qemu-system-ppc) - Node 0 Node 8 Node 252 Node 253 Node 254 Node 255 Total - ------ ------ -------- -------- -------- -------- ----- + Node 0 Node 8 Node 252 Node 253 Node 254 Node 255 Total + ------ ------ -------- -------- -------- -------- ----- Huge 0 0 0 0 0 0 0 Heap 0 0 0 0 0 0 14 Stack 14 0 0 0 0 0 0 Private 936 0 0 0 0 0 3064 ------- ------ ------ -------- -------- -------- -------- ----- Total 936 0 0 0 0 0 3079 - # service numad status * numad.service - numad - The NUMA daemon that manages application locality. - Loaded: loaded (/lib/systemd/system/numad.service; enabled; vendor preset: enabled) - Active: active (running) since Thu 2019-02-21 00:43:54 EST; 20s ago - Docs: man:numad - Process: 8091 ExecStart=/usr/bin/numad $DAEMON_ARGS -i 15 (code=exited, status=0/SUCCESS) - Main PID: 8092 (numad) - Tasks: 2 (limit: 19660) - CGroup: /system.slice/numad.service - `-8092 /usr/bin/numad -i 15 + Loaded: loaded (/lib/systemd/system/numad.service; enabled; vendor preset: enabled) + Active: active (running) since Thu 2019-02-21 00:43:54 EST; 20s ago + Docs: man:numad + Process: 8091 ExecStart=/usr/bin/numad $DAEMON_ARGS -i 15 (code=exited, status=0/SUCCESS) + Main PID: 8092 (numad) + Tasks: 2 (limit: 19660) + CGroup: /system.slice/numad.service + `-8092 /usr/bin/numad -i 15 Feb 21 00:43:54 ltcgen3 systemd[1]: Starting numad - The NUMA daemon that manages application locality.... Feb 21 00:43:54 ltcgen3 systemd[1]: Started numad - The NUMA daemon that manages application locality.. - ---uname output--- Linux ltcgen3 4.15.0-1016-ibm-gt #18-Ubuntu SMP Thu Feb 7 16:58:31 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux - - Machine Type = Witherspoon + + Machine Type = Witherspoon == Versions Installed == qemu 1:2.11+dfsg-1ubuntu7.8-1ibm3 qemu-kvm 1:2.11+dfsg-1ubuntu7.9 qemu-system-ppc 1:2.11+dfsg-1ubuntu7.8-1ibm3 libvirt0:ppc64el 4.0.0-1ubuntu8.6 libnuma-dev:ppc64el 2.0.11-2.1 libnuma1:ppc64el 2.0.11-2.1 numactl 2.0.11-2.1 numad 0.5+20150602-5 ---Debugger--- A debugger is not configured - + ---Steps to Reproduce--- 1. Configure the guest with vcpu pinning and memory binding to one of the numa node. <cputune> - <vcpupin vcpu='0' cpuset='64'/> - <vcpupin vcpu='1' cpuset='65'/> - <vcpupin vcpu='2' cpuset='66'/> - <vcpupin vcpu='3' cpuset='67'/> - <emulatorpin cpuset='64-67'/> - </cputune> - <numatune> - <memory mode='strict' nodeset='8'/> - </numatune> + <vcpupin vcpu='0' cpuset='64'/> + <vcpupin vcpu='1' cpuset='65'/> + <vcpupin vcpu='2' cpuset='66'/> + <vcpupin vcpu='3' cpuset='67'/> + <emulatorpin cpuset='64-67'/> + </cputune> + <numatune> + <memory mode='strict' nodeset='8'/> + </numatune> 2. Start the guest. 3. Check the "numastat" for the guest details. 4. The output doesn't display any data for desired node. - + Contact Information = Santwana Samantray/[email protected] == Comment: #1 - SANTWANA SAMANTRAY <[email protected]> - 2019-02-20 23:49:58 == - == Comment: #4 - SEETEENA THOUFEEK <[email protected]> - 2019-02-22 00:40:37 == I am not able to replicate in upstream code - We need to cherry pick this patch in numactl 2.0.11-2.1 level. - + We need to cherry pick this patch in numactl 2.0.11-2.1 level. + https://github.com/numactl/numactl/commit/b608687037d873ad82d6318f231b3d6612e8601d - root@ltcgen3:~/numactl# ./numastat -c qemu-system-ppc64 Per-node process memory usage (in MBs) for PID 53294 (qemu-system-ppc) - Node 0 Node 8 Node 252 Node 253 Node 254 Node 255 Total - ------ ------ -------- -------- -------- -------- ------ + Node 0 Node 8 Node 252 Node 253 Node 254 Node 255 Total + ------ ------ -------- -------- -------- -------- ------ Huge 0 0 0 0 0 0 0 Heap 0 28 0 0 0 0 28 Stack 0 0 0 0 0 0 0 Private 0 103727 0 0 0 0 103727 ------- ------ ------ -------- -------- -------- -------- ------ Total 0 103755 0 0 0 0 103755 --------------------------------------------------------------------------------------------------- ** Description changed: - Please test the PPA [1] if it would suit your needs. + [Impact] - https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3671 - - - [Impact] - - * Some less common numa hierarchies break numactl - * backport upstream change to fix that - - https://github.com/numactl/numactl/commit/b60868703 + * Some less common numa hierarchies break numactl + * backport upstream change to fix that + - https://github.com/numactl/numactl/commit/b60868703 [Test Case] - * You have to create an "affected" Numa topology - E.g. on a full mem stacked P9 create a guest like: - <vcpu placement='static' current='4'>8</vcpu> - <cputune> - <vcpupin vcpu='0' cpuset='64'/> - <vcpupin vcpu='1' cpuset='65'/> - <vcpupin vcpu='2' cpuset='66'/> - <vcpupin vcpu='3' cpuset='67'/> - <emulatorpin cpuset='64-67'/> - </cputune> - <numatune> - <memory mode='strict' nodeset='8'/> - </numatune> - * Then after starting that numactl will fail to show it's data, for - example: + * You have to create an "affected" Numa topology + E.g. on a full mem stacked P9 create a guest like: + <vcpu placement='static' current='4'>8</vcpu> + <cputune> + <vcpupin vcpu='0' cpuset='64'/> + <vcpupin vcpu='1' cpuset='65'/> + <vcpupin vcpu='2' cpuset='66'/> + <vcpupin vcpu='3' cpuset='67'/> + <emulatorpin cpuset='64-67'/> + </cputune> + <numatune> + <memory mode='strict' nodeset='8'/> + </numatune> + * Then after starting that numactl will fail to show it's data, for + example: # numastat -c qemu-system-ppc64 Per-node process memory usage (in MBs) for PID 5738 (qemu-system-ppc) - Node 0 Node 8 ... - ------ ------ ... - Huge 0 0 ... - Heap 0 0 ... - Stack 14 0 ... - Private 936 0 ... - ------- ------ ------ ... - Total 936 0 ... - (more details on the Host used below) + Node 0 Node 8 ... + ------ ------ ... + Huge 0 0 ... + Heap 0 0 ... + Stack 14 0 ... + Private 936 0 ... + ------- ------ ------ ... + Total 936 0 ... + (more details on the Host used below) [Regression Potential] - * The change is rather small just skipping nodes not matching the - expected numbers (important to not crash). If anything then I can - think of a regression where it would skip a numa node that it - would need to access. But I have seen none such case in my tests, so - I hope it will be fine. + * The change is rather small just skipping nodes not matching the + expected numbers (important to not crash). If anything then I can + think of a regression where it would skip a numa node that it + would need to access. But I have seen none such case in my tests, so + I hope it will be fine. [Other Info] - - * n/a + + * n/a --- == Comment: #0 - SANTWANA SAMANTRAY <[email protected]> - 2019-02-20 23:48:43 == ---Problem Description--- numastat doesn't display correct information for kvm guests. The guest is configured with vcpupin from the node8: Snippet of the guest XML : <vcpu placement='static' current='4'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='64'/> <vcpupin vcpu='1' cpuset='65'/> <vcpupin vcpu='2' cpuset='66'/> <vcpupin vcpu='3' cpuset='67'/> <emulatorpin cpuset='64-67'/> </cputune> <numatune> <memory mode='strict' nodeset='8'/> </numatune> == Host Details == # lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 4 Core(s) per socket: 16 Socket(s): 2 NUMA node(s): 6 Model: 2.2 (pvr 004e 1202) Model name: POWER9, altivec supported CPU max MHz: 3800.0000 CPU min MHz: 2300.0000 L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 10240K NUMA node0 CPU(s): 0-63 NUMA node8 CPU(s): 64-127 NUMA node252 CPU(s): NUMA node253 CPU(s): NUMA node254 CPU(s): NUMA node255 CPU(s): After starting the guest, the numastat doesn't display the guest process to be running in node8. # numastat -c qemu-system-ppc64 Per-node process memory usage (in MBs) for PID 5738 (qemu-system-ppc) Node 0 Node 8 Node 252 Node 253 Node 254 Node 255 Total ------ ------ -------- -------- -------- -------- ----- Huge 0 0 0 0 0 0 0 Heap 0 0 0 0 0 0 14 Stack 14 0 0 0 0 0 0 Private 936 0 0 0 0 0 3064 ------- ------ ------ -------- -------- -------- -------- ----- Total 936 0 0 0 0 0 3079 # service numad status * numad.service - numad - The NUMA daemon that manages application locality. Loaded: loaded (/lib/systemd/system/numad.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-02-21 00:43:54 EST; 20s ago Docs: man:numad Process: 8091 ExecStart=/usr/bin/numad $DAEMON_ARGS -i 15 (code=exited, status=0/SUCCESS) Main PID: 8092 (numad) Tasks: 2 (limit: 19660) CGroup: /system.slice/numad.service `-8092 /usr/bin/numad -i 15 Feb 21 00:43:54 ltcgen3 systemd[1]: Starting numad - The NUMA daemon that manages application locality.... Feb 21 00:43:54 ltcgen3 systemd[1]: Started numad - The NUMA daemon that manages application locality.. ---uname output--- Linux ltcgen3 4.15.0-1016-ibm-gt #18-Ubuntu SMP Thu Feb 7 16:58:31 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux Machine Type = Witherspoon == Versions Installed == qemu 1:2.11+dfsg-1ubuntu7.8-1ibm3 qemu-kvm 1:2.11+dfsg-1ubuntu7.9 qemu-system-ppc 1:2.11+dfsg-1ubuntu7.8-1ibm3 libvirt0:ppc64el 4.0.0-1ubuntu8.6 libnuma-dev:ppc64el 2.0.11-2.1 libnuma1:ppc64el 2.0.11-2.1 numactl 2.0.11-2.1 numad 0.5+20150602-5 ---Debugger--- A debugger is not configured ---Steps to Reproduce--- 1. Configure the guest with vcpu pinning and memory binding to one of the numa node. <cputune> <vcpupin vcpu='0' cpuset='64'/> <vcpupin vcpu='1' cpuset='65'/> <vcpupin vcpu='2' cpuset='66'/> <vcpupin vcpu='3' cpuset='67'/> <emulatorpin cpuset='64-67'/> </cputune> <numatune> <memory mode='strict' nodeset='8'/> </numatune> 2. Start the guest. 3. Check the "numastat" for the guest details. 4. The output doesn't display any data for desired node. Contact Information = Santwana Samantray/[email protected] == Comment: #1 - SANTWANA SAMANTRAY <[email protected]> - 2019-02-20 23:49:58 == == Comment: #4 - SEETEENA THOUFEEK <[email protected]> - 2019-02-22 00:40:37 == I am not able to replicate in upstream code We need to cherry pick this patch in numactl 2.0.11-2.1 level. https://github.com/numactl/numactl/commit/b608687037d873ad82d6318f231b3d6612e8601d root@ltcgen3:~/numactl# ./numastat -c qemu-system-ppc64 Per-node process memory usage (in MBs) for PID 53294 (qemu-system-ppc) Node 0 Node 8 Node 252 Node 253 Node 254 Node 255 Total ------ ------ -------- -------- -------- -------- ------ Huge 0 0 0 0 0 0 0 Heap 0 28 0 0 0 0 28 Stack 0 0 0 0 0 0 0 Private 0 103727 0 0 0 0 103727 ------- ------ ------ -------- -------- -------- -------- ------ Total 0 103755 0 0 0 0 103755 --------------------------------------------------------------------------------------------------- ** Changed in: numactl (Ubuntu Bionic) Status: In Progress => Incomplete ** Changed in: numactl (Ubuntu Bionic) Assignee: Christian Ehrhardt (paelzer) => (unassigned) ** Changed in: numactl (Ubuntu Cosmic) Assignee: Christian Ehrhardt (paelzer) => (unassigned) ** Changed in: numactl (Ubuntu Cosmic) Status: In Progress => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1817258 Title: "numastat" doesn't display correct information for the guest. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1817258/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
