** Changed in: linux (Ubuntu)
Assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) =>
Canonical Kernel Team (canonical-kernel-team)
** Changed in: linux (Ubuntu)
Importance: Undecided => Critical
** Changed in: ubuntu-power-systems
Assignee: (unassigned) => Canonical Kernel Team (canonical-kernel-team)
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1819407
Title:
Guest with vfio device pass-through crashes during reboot operation.
Status in The Ubuntu-power-systems project:
Incomplete
Status in linux package in Ubuntu:
New
Bug description:
== Comment: #0 - SANTWANA SAMANTRAY <santwana.samant...@in.ibm.com> -
2019-02-18 22:08:48 ==
---Problem Description---
Guest with vfio device pass-through crashes during reboot operation.
The below error is noticed in the libvirt.log of the guest.
2019-02-18 09:43:55.348+0000: 19136: info : virObjectUnref:350 :
OBJECT_UNREF: obj=0x71bdb80fae00
2019-02-18T09:43:55.366229Z qemu-system-ppc64: -chardev pty,id=charserial0:
char device redirected to /dev/pts/8 (label charserial0)
2019-02-18T14:53:23.937306Z qemu-system-ppc64: Failed to create a window, ret
= -1 (Cannot allocate memory)
qemu: hardware error: vfio: DMA mapping failed, unable to continue
CPU #0:
NIP 000000000daf0010 LR 000000000000bbc8 CTR c00000000fa80000 XER
0000000020040000 CPU#0
MSR 0000000102801000 HID0 0000000000000000 HF 8000000000000000 iidx 3 didx 3
TB 00000000 00000000 DECR 00000000
GPR00 8000000102803031 c0000018e4b1ae80 c0000000016eba00 000000000000f000
GPR04 0000000001780ad0 000000000daf0000 0000000102801000 8000000102803033
GPR08 000000000a000000 8000000000002933 0000000000000010 3030382030303038
GPR12 0000000000008000 c00000000fa80000 0000000008000000 0000000000000000
GPR16 0000000020000001 0000000000000010 c00000000641e1a0 c0000018fd3dace0
GPR20 c0000000019a2ba0 c0000018fd05b098 c0000018ffff52a8 0000000000000029
GPR24 c0000018e4b1b154 0000000000000000 0000000000000004 0000000000000001
GPR28 0000000000000004 c0000018e4b1b154 c000000001780ab0 0000000000000004
CR 40000000 [ G - - - - - - - ] RES ffffffffffffffff
FPR00 8d73d0cfdf8626c9 0000000000000000 0000000000000000 0000000000000000
FPR04 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR08 0000000000000000 0000000000000000 6c7967656e657261 0000000000000000
FPR12 9265dacfc19031dd 0000000000000000 0000000000000000 0000000000000000
FPR16 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR20 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR24 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR28 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPSCR 0000000000000000
SRR0 000000000daf0010 SRR1 0000000102801000 PVR 00000000004e1202 VRSAVE
0000000000000000
SPRG0 0000000000000000 SPRG1 c00000000fa80000 SPRG2 c00000000fa80000 SPRG3
0000000000000000
SPRG4 0000000000000000 SPRG5 0000000000000000 SPRG6 0000000000000000 SPRG7
0000000000000000
HSRR0 0000000000000000 HSRR1 0000000000000000
CFAR 0000000000000000
LPCR 0000000003d4f41f
DAR 00000c8be5f8b8b0 DSISR 000000000a000000
In this case, NVIDIA GPU was pass-through'ed to the guest.
0004:04:00.0 3D controller [0302]: NVIDIA Corporation GV100 [Tesla V100 SXM2]
[10de:1db1] (rev a1)
0004:05:00.0 3D controller [0302]: NVIDIA Corporation GV100 [Tesla V100 SXM2]
[10de:1db1] (rev a1)
The initial few attempts of the guest reboot is successful, however in
subsequent trials of rebooting in a loop, the guest crashes.
== Versions Installed ==
qemu 1:2.11+dfsg-1ubuntu7.8-1ibm3
qemu-kvm 1:2.11+dfsg-1ubuntu7.9
qemu-system-ppc 1:2.11+dfsg-1ubuntu7.8-1ibm3
libvirt0:ppc64el 4.0.0-1ubuntu8.6
Contact Information = Santwana Samantray/santwana.samant...@in.ibm.com
---uname output---
Linux ltcgen3 4.15.0-1016-ibm-gt #18-Ubuntu SMP Thu Feb 7 16:58:31 UTC 2019
ppc64le ppc64le ppc64le GNU/Linux
Machine Type = Witherspoon
---Debugger---
A debugger is not configured
---Steps to Reproduce---
1. Configure the guest with vfio pass-through.
2. Start the guest.
3. While the guest is in a running state, reboot the guest in a loop.
[while true; do virsh reboot santwana_ubuntu; sleep 120; done]
== Comment: #1 - SANTWANA SAMANTRAY <santwana.samant...@in.ibm.com> -
2019-02-18 22:09:35 ==
== Comment: #2 - SANTWANA SAMANTRAY <santwana.samant...@in.ibm.com> -
2019-02-18 22:10:13 ==
== Comment: #3 - SANTWANA SAMANTRAY <santwana.samant...@in.ibm.com> -
2019-02-18 22:11:16 ==
== Comment: #9 - Alexey Kardashevskiy <ale...@au1.ibm.com> - 2019-02-20
19:09:22 ==
The patch from https://bugzilla.linux.ibm.com/show_bug.cgi?id=175550#c18
should fix this issue too, this bz is a duplicate really.
== Comment: #11 - SANTWANA SAMANTRAY <santwana.samant...@in.ibm.com> -
2019-02-21 22:07:20 ==
After installing the test kernel (https://ibm.ent.box.com/folder/67860346392)
, the issue is still reproducible. The guest crashes during reboot operation.
qemu: hardware error: vfio: DMA mapping failed, unable to continue
CPU #0:
NIP 000000000daf0010 LR 000000000000bbc8 CTR c000000003680000 XER
0000000020040000 CPU#0
MSR 0000000102801000 HID0 0000000000000000 HF 8000000000000000 iidx 3 didx 3
TB 00000000 00000000 DECR 00000000
GPR00 8000000102803031 c0000000061aee80 c0000000016eba00 000000000000f000
GPR04 0000000001780ad0 000000000daf0000 0000000102801000 8000000102803033
GPR08 000000000a000000 8000000000002933 0000000000000010 3030303820303038
GPR12 0000000000008000 c000000003680000 0000000008000000 0000000000000000
GPR16 0000000020000001 0000000000000010 c000000006539140 c00000000677a500
GPR20 c0000000019a2ba0 c0000018fd344098 c0000018ffff52b8 0000000000000029
GPR24 c0000000061af154 0000000000000000 0000000000000004 0000000000000001
GPR28 0000000000000004 c0000000061af154 c000000001780ab0 0000000000000004
CR 40000000 [ G - - - - - - - ] RES ffffffffffffffff
FPR00 4c8d21eebfdf167e 0000000000000000 0000000000000000 0000000000000000
FPR04 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR08 0000000000000000 0000000000000000 6c7967656e657261 0000000000000000
FPR12 539b2beea1c9016a 0000000000000000 0000000000000000 0000000000000000
FPR16 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR20 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR24 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR28 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPSCR 0000000000000000
SRR0 000000000daf0010 SRR1 0000000102801000 PVR 00000000004e1202 VRSAVE
00000000ffffffff
SPRG0 0000000000000000 SPRG1 c000000003680000 SPRG2 0000794fee76daa0 SPRG3
0000000000000000
SPRG4 0000000000000000 SPRG5 0000000000000000 SPRG6 0000000000000000 SPRG7
0000000000000000
HSRR0 0000000000000000 HSRR1 0000000000000000
CFAR 0000000000000000
LPCR 0000000003d4f41f
DAR 000009a9459ef6c8 DSISR 000000000a000000
2019-02-21 14:48:45.097+0000: shutting down, reason=crashed
== Versions Installed ==
Kernel: 4.15.0-1016.18-fix1-ibm-gt
qemu: 1:2.11+dfsg-1ubuntu7.8-1ibm3
libvirt-bin : 4.0.0-1ubuntu8.6
== Comment: #12 - SANTWANA SAMANTRAY <santwana.samant...@in.ibm.com> -
2019-02-21 22:10:05 ==
please cherry pick
https://patchwork.ozlabs.org/patch/1040938/
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1819407/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
