Public bug reported:
== Comment: #0 - Satheesh Rajendran <[email protected]> - 2017-03-27 12:30:45
==
---Problem Description---
Kernel hit with oops while running avocado(kvm) tests "Oops: Exception in
kernel mode, sig: 5 [#1]"
Contact Information = [email protected]
---uname output---
Linux ltc-test-ci1 4.10.0-14-generic #16-Ubuntu SMP Fri Mar 17 15:19:05 UTC
2017 ppc64le ppc64le ppc64le GNU/Linux
Machine Type = power 8 ppc64le
---Debugger---
A debugger is not configured
---Steps to Reproduce---
1. Run Avocado(kvm) tests
#git clone git://git.linux.ibm.com/ltc-test/avocado-fvt-wrapper.git;cd
avocado-fvt-wrapper
#python avocado-setup.py --bootstrap --run-suite guest_cpu --guest-os
Ubuntu.16.04.2.ppc64le --only-filter virtio_scsi virtio_net qcow2
2. After sometime the below mentioned traces were seen.
Stack trace output:
[20751.909458] ------------[ cut here ]------------
[20751.909461] kernel BUG at
/build/linux-HLNhAK/linux-4.10.0/include/linux/swapops.h:129!
[20751.909542] Oops: Exception in kernel mode, sig: 5 [#1]
[20751.909549] SMP NR_CPUS=2048
[20751.909549] NUMA
[20751.909555] PowerNV
[20751.909583] Modules linked in: vhost_net macvtap macvlan xt_CHECKSUM
ipt_MASQUERADE nf_nat_masquerade_ipv4 kvm_hv kvm_pr kvm tcm_fc libfc usb_f_tcm
tcm_usb_gadget libcomposite udc_core tcm_qla2xxx qla2xxx scsi_transport_fc
ib_srpt iscsi_target_mod tcm_loop vhost_scsi vhost target_core_user
target_core_file target_core_iblock target_core_pscsi target_core_mod
ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6t_REJECT nf_reject_ipv6 xt_conntrack
ip6t_rpfilter ip_set nfnetlink ebtable_broute bridge stp llc ebtable_nat
ip6table_security ip6table_mangle ip6table_raw ip6table_nat iptable_security
iptable_mangle iptable_raw iptable_nat ebtable_filter ebtables ip6table_filter
ip6_tables iptable_filter openvswitch nf_conntrack_ipv6 nf_nat_ipv6
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_defrag_ipv6 nf_nat nf_conntrack
[20751.910225] binfmt_misc powernv_rng powernv_op_panel ipmi_powernv
ipmi_devintf ipmi_msghandler uio_pdrv_genirq uio leds_powernv vmx_crypto
ib_iser rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace
configfs iscsi_tcp libiscsi_tcp sunrpc libiscsi scsi_transport_iscsi ip_tables
x_tables autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq
async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear ses
enclosure scsi_transport_sas crc32c_vpmsum tg3 ipr
[20751.910629] CPU: 24 PID: 6926 Comm: CPU 24/KVM Not tainted 4.10.0-14-generic
#16-Ubuntu
[20751.910700] task: c0000007b29ac000 task.stack: c0000007f07b8000
[20751.910759] NIP: c00000000030d748 LR: c00000000030d658 CTR: 0000000000000000
[20751.910828] REGS: c0000007f07bb3b0 TRAP: 0700 Not tainted
(4.10.0-14-generic)
[20751.910897] MSR: 9000000000029033 <SF,HV,EE,ME,IR,DR,RI,LE>
[20751.910903] CR: 44882882 XER: 00000000
[20751.910984] CFAR: c00000000030d884 SOFTE: 1
GPR00: c00000000030d658 c0000007f07bb630 c00000000144c900
f000000001f9f1f0
GPR04: c0000007e7c7e0e0 f000000001f9f1f0 000000001f001c61
00000000611c001f
GPR08: c0000000015bc900 0000000000000001 0000000000000001
0000000000e0c7e7
GPR12: 0000000000002200 c00000000fb8d800 0000000000000000
000000000007fe1d
GPR16: 0000000000010000 00003fff121c0000 0000000000000000
0000000088000000
GPR20: 0000000020000000 0000000088000000 0000000022000000
c000000004990000
GPR24: c0000000015be3d8 c0000007f05d3700 c0000007b972c030
0000000000000000
GPR28: c0000007f07bb710 3e000000000d611c f000000003584700
f000000001f9f1f0
[20751.911580] NIP [c00000000030d748] __migration_entry_wait+0x128/0x2a0
[20751.911639] LR [c00000000030d658] __migration_entry_wait+0x38/0x2a0
[20751.911697] Call Trace:
[20751.911722] [c0000007f07bb630] [c00000000030d658]
__migration_entry_wait+0x38/0x2a0 (unreliable)
[20751.911807] [c0000007f07bb670] [c0000000002bbc6c] do_swap_page+0x73c/0x9a0
[20751.911866] [c0000007f07bb6f0] [c0000000002bfa98]
handle_mm_fault+0xac8/0x1600
[20751.911937] [c0000007f07bb7e0] [c0000000002b4104]
__get_user_pages+0x194/0x4e0
[20751.912008] [c0000007f07bb890] [c0000000002b47e4]
get_user_pages_unlocked+0xf4/0x280
[20751.912079] [c0000007f07bb930] [c0000000002b59ac]
get_user_pages_fast+0xac/0x100
[20751.912152] [c0000007f07bb980] [d00000000f66ca74]
kvmppc_book3s_hv_page_fault+0x2bc/0xbb0 [kvm_hv]
[20751.912236] [c0000007f07bba70] [d00000000f6696f8]
kvmppc_vcpu_run_hv+0xe60/0x1220 [kvm_hv]
[20751.912312] [c0000007f07bbb80] [d00000000f6131ac] kvmppc_vcpu_run+0x34/0x48
[kvm]
[20751.912387] [c0000007f07bbba0] [d00000000f61030c]
kvm_arch_vcpu_ioctl_run+0x64/0x170 [kvm]
[20751.912462] [c0000007f07bbbe0] [d00000000f603db8] kvm_vcpu_ioctl+0x500/0x780
[kvm]
[20751.912534] [c0000007f07bbd40] [c00000000035b1f4] do_vfs_ioctl+0xd4/0x8c0
[20751.912594] [c0000007f07bbde0] [c00000000035bab4] SyS_ioctl+0xd4/0xf0
[20751.912654] [c0000007f07bbe30] [c00000000000b184] system_call+0x38/0xe0
[20751.912713] Instruction dump:
[20751.912749] 3d020017 79293448 39481a68 ebca0000 7fde4a14 e93e0020 712a0001
4082014c
[20751.912822] 7fc9f378 e9290000 7d2948f8 792907e0 <0b090000> 39400000 3bbe001c
39000001
[20751.912899] ---[ end trace 5eaae2f83c5daa20 ]---
System Dump Info:
The system is not configured to capture a system dump.
== Comment: #3 - IRANNA D. ANKAD <[email protected]> - 2017-03-28
01:50:15 ==
This is a regression from 4.10.0-13 kernel and blocking our regression tetsing
== Comment: #10 - VIPIN K. PARASHAR <[email protected]> - 2017-03-30 03:44:13
==
root@ltc-test-ci1:~# uname -a
Linux ltc-test-ci1 4.10.0-15-generic #17-Ubuntu SMP Fri Mar 24 17:50:37 UTC
2017 ppc64le ppc64le ppc64le GNU/Linux
root@ltc-test-ci1:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="17.04 (Zesty Zapus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu Zesty Zapus (development branch)"
VERSION_ID="17.04"
HOME_URL="https://www.ubuntu.com/";
SUPPORT_URL="https://help.ubuntu.com/";
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/";
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy";
VERSION_CODENAME=zesty
UBUNTU_CODENAME=zesty
root@ltc-test-ci1:~# tail /proc/cpuinfo
processor : 79
cpu : POWER8E (raw), altivec supported
clock : 2061.000000MHz
revision : 2.1 (pvr 004b 0201)
timebase : 512000000
platform : PowerNV
model : 8247-21L
machine : PowerNV 8247-21L
firmware : OPAL
root@ltc-test-ci1:~#
== Comment: #11 - VIPIN K. PARASHAR <[email protected]> - 2017-03-30 03:58:04
==
Mar 24 07:15:57
===========
[ 1955.041619] ------------[ cut here ]------------
[ 1955.041623] kernel BUG at
/build/linux-HLNhAK/linux-4.10.0/include/linux/swapops.h:129!
[ 1955.041633] Oops: Exception in kernel mode, sig: 5 [#1]
[ 1955.041637] SMP NR_CPUS=2048
[ 1955.041638] NUMA
[ 1955.041641] PowerNV
[ 1955.041645] Modules linked in: vhost_net macvtap macvlan rpcsec_gss_krb5
nfsv4 nfs fscache xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 kvm_hv
kvm_pr kvm tcm_fc libfc usb_f_tcm tcm_usb_gadget libcomposite udc_core
tcm_qla2xxx qla2xxx scsi_transport_fc ib_srpt iscsi_target_mod tcm_loop
vhost_scsi vhost target_core_user target_core_file target_core_iblock
target_core_pscsi target_core_mod ip6t_rpfilter ip6t_REJECT nf_reject_ipv6
xt_tcpudp ipt_REJECT nf_reject_ipv4 xt_conntrack ip_set nfnetlink
ebtable_broute bridge stp llc ebtable_nat ip6table_raw ip6table_security
ip6table_mangle ip6table_nat iptable_raw iptable_security iptable_mangle
iptable_nat ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter
openvswitch nf_conntrack_ipv6 nf_nat_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4
nf_nat_ipv4
[ 1955.041724] nf_defrag_ipv6 nf_nat nf_conntrack binfmt_misc powernv_rng
powernv_op_panel ipmi_powernv ipmi_devintf ipmi_msghandler leds_powernv
uio_pdrv_genirq uio vmx_crypto nfsd auth_rpcgss nfs_acl lockd grace sunrpc
ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi
scsi_transport_iscsi ip_tables x_tables autofs4 btrfs raid10 raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq
libcrc32c raid1 raid0 multipath linear ses enclosure scsi_transport_sas
crc32c_vpmsum tg3 ipr
[ 1955.041786] CPU: 40 PID: 7306 Comm: CPU 13/KVM Not tainted 4.10.0-14-generic
#16-Ubuntu
[ 1955.041792] task: c0000007937b5a00 task.stack: c0000007f2a48000
[ 1955.041796] NIP: c00000000030d748 LR: c00000000030d658 CTR: 0000000000000000
[ 1955.041801] REGS: c0000007f2a4b3b0 TRAP: 0700 Not tainted
(4.10.0-14-generic)
[ 1955.041805] MSR: 900000000282b033 <SF,HV,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>
[ 1955.041813] CR: 44882882 XER: 00000000
[ 1955.041819] CFAR: c00000000030d884 SOFTE: 1
[ 1955.041819] GPR00: c00000000030d658 c0000007f2a4b630 c00000000144c900
f000000001fb8b30
[ 1955.041819] GPR04: c0000007ee2cd2c8 f000000001fb8b30 000000001f005977
000000007759001f
[ 1955.041819] GPR08: c0000000015bc900 0000000000000001 0000000000000001
0000000000d02cee
[ 1955.041819] GPR12: 0000000000002200 c00000000fb96800 0000000000000000
000000000000005a
[ 1955.041819] GPR16: 0000000000010000 00003ffe30590000 0000000000000000
0000000088000000
[ 1955.041819] GPR20: 0000000020000000 0000000088000000 0000000022000000
c0000000fd3d0000
[ 1955.041819] GPR24: c0000000015be3d8 c0000007c3c66880 c0000007ed9b40d8
0000000000000000
[ 1955.041819] GPR28: c0000007f2a4b710 3e00000000077759 f000000001ddd640
f000000001fb8b30
[ 1955.041874] NIP [c00000000030d748] __migration_entry_wait+0x128/0x2a0
[ 1955.041879] LR [c00000000030d658] __migration_entry_wait+0x38/0x2a0
[ 1955.041883] Call Trace:
[ 1955.041886] [c0000007f2a4b630] [c00000000030d658]
__migration_entry_wait+0x38/0x2a0 (unreliable)
[ 1955.041894] [c0000007f2a4b670] [c0000000002bbc6c] do_swap_page+0x73c/0x9a0
[ 1955.041900] [c0000007f2a4b6f0] [c0000000002bfa98]
handle_mm_fault+0xac8/0x1600
[ 1955.041906] [c0000007f2a4b7e0] [c0000000002b4104]
__get_user_pages+0x194/0x4e0
[ 1955.041912] [c0000007f2a4b890] [c0000000002b47e4]
get_user_pages_unlocked+0xf4/0x280
[ 1955.041918] [c0000007f2a4b930] [c0000000002b59ac]
get_user_pages_fast+0xac/0x100
[ 1955.041927] [c0000007f2a4b980] [d00000000f7aca74]
kvmppc_book3s_hv_page_fault+0x2bc/0xbb0 [kvm_hv]
[ 1955.041935] [c0000007f2a4ba70] [d00000000f7a96f8]
kvmppc_vcpu_run_hv+0xe60/0x1220 [kvm_hv]
[ 1955.041947] [c0000007f2a4bb80] [d00000000f7531ac] kvmppc_vcpu_run+0x34/0x48
[kvm]
[ 1955.041958] [c0000007f2a4bba0] [d00000000f75030c]
kvm_arch_vcpu_ioctl_run+0x64/0x170 [kvm]
[ 1955.041967] [c0000007f2a4bbe0] [d00000000f743db8] kvm_vcpu_ioctl+0x500/0x780
[kvm]
[ 1955.041974] [c0000007f2a4bd40] [c00000000035b1f4] do_vfs_ioctl+0xd4/0x8c0
[ 1955.041980] [c0000007f2a4bde0] [c00000000035bab4] SyS_ioctl+0xd4/0xf0
[ 1955.041986] [c0000007f2a4be30] [c00000000000b184] system_call+0x38/0xe0
[ 1955.041990] Instruction dump:
[ 1955.041994] 3d020017 79293448 39481a68 ebca0000 7fde4a14 e93e0020 712a0001
4082014c
[ 1955.042003] 7fc9f378 e9290000 7d2948f8 792907e0 <0b090000> 39400000 3bbe001c
39000001
[ 1955.042016] ---[ end trace 1c0e9a056f95491f ]---
Mar 27 11:54:27
============
[20751.909458] ------------[ cut here ]------------
[20751.909461] kernel BUG at
/build/linux-HLNhAK/linux-4.10.0/include/linux/swapops.h:129!
[20751.909542] Oops: Exception in kernel mode, sig: 5 [#1]
[20751.909549] SMP NR_CPUS=2048
[20751.909549] NUMA
[20751.909555] PowerNV
[20751.909583] Modules linked in: vhost_net macvtap macvlan xt_CHECKSUM
ipt_MASQUERADE nf_nat_masquerade_ipv4 kvm_hv kvm_pr kvm tcm_fc libfc usb_f_tcm
tcm_usb_gadget libcomposite udc_core tcm_qla2xxx qla2xxx scsi_transport_fc
ib_srpt iscsi_target_mod tcm_loop vhost_scsi vhost target_core_user
target_core_file target_core_iblock target_core_pscsi target_core_mod
ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6t_REJECT nf_reject_ipv6 xt_conntrack
ip6t_rpfilter ip_set nfnetlink ebtable_broute bridge stp llc ebtable_nat
ip6table_security ip6table_mangle ip6table_raw ip6table_nat iptable_security
iptable_mangle iptable_raw iptable_nat ebtable_filter ebtables ip6table_filter
ip6_tables iptable_filter openvswitch nf_conntrack_ipv6 nf_nat_ipv6
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_defrag_ipv6 nf_nat nf_conntrack
[20751.910225] binfmt_misc powernv_rng powernv_op_panel ipmi_powernv
ipmi_devintf ipmi_msghandler uio_pdrv_genirq uio leds_powernv vmx_crypto
ib_iser rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace
configfs iscsi_tcp libiscsi_tcp sunrpc libiscsi scsi_transport_iscsi ip_tables
x_tables autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq
async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear ses
enclosure scsi_transport_sas crc32c_vpmsum tg3 ipr
[20751.910629] CPU: 24 PID: 6926 Comm: CPU 24/KVM Not tainted 4.10.0-14-generic
#16-Ubuntu
[20751.910700] task: c0000007b29ac000 task.stack: c0000007f07b8000
[20751.910759] NIP: c00000000030d748 LR: c00000000030d658 CTR: 0000000000000000
[20751.910828] REGS: c0000007f07bb3b0 TRAP: 0700 Not tainted
(4.10.0-14-generic)
[20751.910897] MSR: 9000000000029033 <SF,HV,EE,ME,IR,DR,RI,LE>
[20751.910903] CR: 44882882 XER: 00000000
[20751.910984] CFAR: c00000000030d884 SOFTE: 1
[20751.910984] GPR00: c00000000030d658 c0000007f07bb630 c00000000144c900
f000000001f9f1f0
[20751.910984] GPR04: c0000007e7c7e0e0 f000000001f9f1f0 000000001f001c61
00000000611c001f
[20751.910984] GPR08: c0000000015bc900 0000000000000001 0000000000000001
0000000000e0c7e7
[20751.910984] GPR12: 0000000000002200 c00000000fb8d800 0000000000000000
000000000007fe1d
[20751.910984] GPR16: 0000000000010000 00003fff121c0000 0000000000000000
0000000088000000
[20751.910984] GPR20: 0000000020000000 0000000088000000 0000000022000000
c000000004990000
[20751.910984] GPR24: c0000000015be3d8 c0000007f05d3700 c0000007b972c030
0000000000000000
[20751.910984] GPR28: c0000007f07bb710 3e000000000d611c f000000003584700
f000000001f9f1f0
[20751.911580] NIP [c00000000030d748] __migration_entry_wait+0x128/0x2a0
[20751.911639] LR [c00000000030d658] __migration_entry_wait+0x38/0x2a0
[20751.911697] Call Trace:
[20751.911722] [c0000007f07bb630] [c00000000030d658]
__migration_entry_wait+0x38/0x2a0 (unreliable)
[20751.911807] [c0000007f07bb670] [c0000000002bbc6c] do_swap_page+0x73c/0x9a0
[20751.911866] [c0000007f07bb6f0] [c0000000002bfa98]
handle_mm_fault+0xac8/0x1600
[20751.911937] [c0000007f07bb7e0] [c0000000002b4104]
__get_user_pages+0x194/0x4e0
[20751.912008] [c0000007f07bb890] [c0000000002b47e4]
get_user_pages_unlocked+0xf4/0x280
[20751.912079] [c0000007f07bb930] [c0000000002b59ac]
get_user_pages_fast+0xac/0x100
[20751.912152] [c0000007f07bb980] [d00000000f66ca74]
kvmppc_book3s_hv_page_fault+0x2bc/0xbb0 [kvm_hv]
[20751.912236] [c0000007f07bba70] [d00000000f6696f8]
kvmppc_vcpu_run_hv+0xe60/0x1220 [kvm_hv]
[20751.912312] [c0000007f07bbb80] [d00000000f6131ac] kvmppc_vcpu_run+0x34/0x48
[kvm]
[20751.912387] [c0000007f07bbba0] [d00000000f61030c]
kvm_arch_vcpu_ioctl_run+0x64/0x170 [kvm]
[20751.912462] [c0000007f07bbbe0] [d00000000f603db8] kvm_vcpu_ioctl+0x500/0x780
[kvm]
[20751.912534] [c0000007f07bbd40] [c00000000035b1f4] do_vfs_ioctl+0xd4/0x8c0
[20751.912594] [c0000007f07bbde0] [c00000000035bab4] SyS_ioctl+0xd4/0xf0
[20751.912654] [c0000007f07bbe30] [c00000000000b184] system_call+0x38/0xe0
[20751.912713] Instruction dump:
[20751.912749] 3d020017 79293448 39481a68 ebca0000 7fde4a14 e93e0020 712a0001
4082014c
[20751.912822] 7fc9f378 e9290000 7d2948f8 792907e0 <0b090000> 39400000 3bbe001c
39000001
[20751.912899] ---[ end trace 5eaae2f83c5daa20 ]---
As pasted above, two instances of Oops are seen in kernel logs.
== Comment: #15 - VIPIN K. PARASHAR <[email protected]> - 2017-03-30 11:34:06
==
>From Linux source
============
/*
* Something used the pte of a page under migration. We need to
* get to the page and wait until migration is finished.
* When we return from this function the fault will be retried.
*/
void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep,
spinlock_t *ptl)
{
..
..
page = migration_entry_to_page(entry);
static inline struct page *migration_entry_to_page(swp_entry_t entry)
{
struct page *p = pfn_to_page(swp_offset(entry));
/*
* Any use of migration entries may only occur while the
* corresponding page is locked
*/
BUG_ON(!PageLocked(p)); <------ Oops here
return p;
}
Kernel Oops is getting invoked, due to hitting a BUG_ON in kernel, while
servicing KVM ioctl and subsequent page fault with pages being migrated.
Same issue has been noticed on intel as well
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1677057
** Affects: linux (Ubuntu)
Importance: Undecided
Assignee: Taco Screen team (taco-screen-team)
Status: New
** Tags: architecture-ppc64le bugnameltc-152928 severity-critical
targetmilestone-inin1704
** Tags added: architecture-ppc64le bugnameltc-152928 severity-critical
targetmilestone-inin1704
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1680390
Title:
Ubuntu17.04: Kernel Oops: Exception in kernel mode, sig: 5 [#1] during
Avocado KVM Test runs [Regression]
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1680390/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
