Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Wei Huang]

On 03/30/2017 05:51 AM, Marc Zyngier wrote:
> On 29/03/17 19:56, Christoffer Dall wrote:
>> On Tue, Mar 28, 2017 at 01:24:15PM -0700, Radha Mohan wrote:
>>> On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  wrote:
 Hi Radha,

 On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
> Hi,
> I am seeing an issue with qemu-system-aarch64 when using pflash
> (booting kernel via UEFI bios).
>
> Host kernel: 4.11.0-rc3-next-20170323
> Qemu version: v2.9.0-rc1
>
> Command used:
> ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
> virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
> if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
> virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
> /root/flash1.img
>
>
> As soon as the guest kernel boots the host starts to stall and prints
> the below messages. And the system never recovers. I can neither
> poweroff the guest nor the host. So I have resort to external power
> reset of the host.
>
> ==
> [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
> [kworker/25:1:454]
> [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
> shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
> sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
> d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
> raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
> drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
> ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
> mdio_thunder mdio_cavium
>
> [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
> 4.11.0-rc3-next-20170323 #1
> [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 
> 23 2017
> [  116.207010] Workqueue: events netstamp_clear
> [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
> [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
> [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
> [  116.207026] pc : [] lr : []
> pstate: 8145
> [  116.207028] sp : 801f901a7be0
> [  116.207030] x29: 801f901a7be0 x28: 09139000
> [  116.207036] x27: 09139434 x26: 0080
> [  116.207041] x25:  x24: 081565d0
> [  116.207047] x23: 0001 x22: 08e11e00
> [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
> [  116.207057] x19: 09138e38 x18: 0a03
> [  116.207063] x17: b77c9028 x16: 082e81d8
> [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
> [  116.207073] x13: 58dabe4c x12: 0018
> [  116.207079] x11: 366e2f04 x10: 09f0
> [  116.207084] x9 : 801f901a7d30 x8 : 0002
> [  116.207089] x7 :  x6 : 
> [  116.207095] x5 :  x4 : 0020
> [  116.207100] x3 : 0020 x2 : 
> [  116.207105] x1 : 801f6d682578 x0 : 0003
>
> [  150.443116] INFO: rcu_sched self-detected stall on CPU
> [  150.448261]  25-...: (14997 ticks this GP)
> idle=47a/141/0 softirq=349/349 fqs=7495
> [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
> [  150.451123]  25-...: (14997 ticks this GP)
> idle=47a/141/0 softirq=349/349 fqs=7495
> [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
> [  150.451136] Task dump for CPU 25:
> [  150.451138] kworker/25:1R  running task0   454  2 
> 0x0002
> [  150.451155] Workqueue: events netstamp_clear
> [  150.451158] Call trace:
> [  150.451164] [] __switch_to+0x90/0xa8
> [  150.451172] [] static_key_slow_inc+0x128/0x138
> [  150.451175] [] static_key_enable+0x34/0x60
> [  150.451178] [] netstamp_clear+0x68/0x80
> [  150.451181] [] process_one_work+0x158/0x478
> [  150.451183] [] worker_thread+0x50/0x4a8
> [  150.451187] [] kthread+0x108/0x138
> [  150.451190] [] ret_from_fork+0x10/0x50
> [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
> [  150.482242] Task dump for CPU 25:
> [  150.482245] kworker/25:1R  running task0   454  2 
> 0x0002
> [  150.482259] Workqueue: events netstamp_clear
> [  150.482264] Call trace:
> [  150.482271] [] dump_backtrace+0x0/0x2b0
> [  150.482277] [] show_stack+0x24/0x30
> [  150.482281] [] sched_show_task+0x128/0x178
> [  150.482285] [] dump_cpu_task+0x48/0x58
> [  150.482288] [] rcu_dump_cpu_stacks+0xa0/0xe8
> [  150.482297] [] rcu_check_callbacks+0x774/0x938
> [  150.482305] [] update_process_times+0x34/0x60
> [  

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Radha Mohan]

On Fri, Mar 31, 2017 at 4:16 PM, Radha Mohan  wrote:
> On Thu, Mar 30, 2017 at 9:47 AM, Laszlo Ersek  wrote:
>> On 03/29/17 20:56, Christoffer Dall wrote:
>>> On Tue, Mar 28, 2017 at 01:24:15PM -0700, Radha Mohan wrote:
 On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  wrote:
> Hi Radha,
>
> On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
>> Hi,
>> I am seeing an issue with qemu-system-aarch64 when using pflash
>> (booting kernel via UEFI bios).
>>
>> Host kernel: 4.11.0-rc3-next-20170323
>> Qemu version: v2.9.0-rc1
>>
>> Command used:
>> ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
>> virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
>> if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
>> virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
>> /root/flash1.img
>>
>>
>> As soon as the guest kernel boots the host starts to stall and prints
>> the below messages. And the system never recovers. I can neither
>> poweroff the guest nor the host. So I have resort to external power
>> reset of the host.
>>
>> ==
>> [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
>> [kworker/25:1:454]
>> [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
>> shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
>> sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
>> d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
>> raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
>> drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
>> ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
>> mdio_thunder mdio_cavium
>>
>> [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
>> 4.11.0-rc3-next-20170323 #1
>> [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 
>> 23 2017
>> [  116.207010] Workqueue: events netstamp_clear
>> [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
>> [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
>> [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
>> [  116.207026] pc : [] lr : []
>> pstate: 8145
>> [  116.207028] sp : 801f901a7be0
>> [  116.207030] x29: 801f901a7be0 x28: 09139000
>> [  116.207036] x27: 09139434 x26: 0080
>> [  116.207041] x25:  x24: 081565d0
>> [  116.207047] x23: 0001 x22: 08e11e00
>> [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
>> [  116.207057] x19: 09138e38 x18: 0a03
>> [  116.207063] x17: b77c9028 x16: 082e81d8
>> [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
>> [  116.207073] x13: 58dabe4c x12: 0018
>> [  116.207079] x11: 366e2f04 x10: 09f0
>> [  116.207084] x9 : 801f901a7d30 x8 : 0002
>> [  116.207089] x7 :  x6 : 
>> [  116.207095] x5 :  x4 : 0020
>> [  116.207100] x3 : 0020 x2 : 
>> [  116.207105] x1 : 801f6d682578 x0 : 0003
>>
>> [  150.443116] INFO: rcu_sched self-detected stall on CPU
>> [  150.448261]  25-...: (14997 ticks this GP)
>> idle=47a/141/0 softirq=349/349 fqs=7495
>> [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
>> [  150.451123]  25-...: (14997 ticks this GP)
>> idle=47a/141/0 softirq=349/349 fqs=7495
>> [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
>> [  150.451136] Task dump for CPU 25:
>> [  150.451138] kworker/25:1R  running task0   454  2 
>> 0x0002
>> [  150.451155] Workqueue: events netstamp_clear
>> [  150.451158] Call trace:
>> [  150.451164] [] __switch_to+0x90/0xa8
>> [  150.451172] [] static_key_slow_inc+0x128/0x138
>> [  150.451175] [] static_key_enable+0x34/0x60
>> [  150.451178] [] netstamp_clear+0x68/0x80
>> [  150.451181] [] process_one_work+0x158/0x478
>> [  150.451183] [] worker_thread+0x50/0x4a8
>> [  150.451187] [] kthread+0x108/0x138
>> [  150.451190] [] ret_from_fork+0x10/0x50
>> [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
>> [  150.482242] Task dump for CPU 25:
>> [  150.482245] kworker/25:1R  running task0   454  2 
>> 0x0002
>> [  150.482259] Workqueue: events netstamp_clear
>> [  150.482264] Call trace:
>> [  150.482271] [] dump_backtrace+0x0/0x2b0
>> [  150.482277] [] show_stack+0x24/0x30
>> [  150.482281] [] sched_show_task+0x128/0x178
>> [  150.482285] [] 

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Radha Mohan]

On Thu, Mar 30, 2017 at 9:47 AM, Laszlo Ersek  wrote:
> On 03/29/17 20:56, Christoffer Dall wrote:
>> On Tue, Mar 28, 2017 at 01:24:15PM -0700, Radha Mohan wrote:
>>> On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  wrote:
 Hi Radha,

 On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
> Hi,
> I am seeing an issue with qemu-system-aarch64 when using pflash
> (booting kernel via UEFI bios).
>
> Host kernel: 4.11.0-rc3-next-20170323
> Qemu version: v2.9.0-rc1
>
> Command used:
> ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
> virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
> if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
> virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
> /root/flash1.img
>
>
> As soon as the guest kernel boots the host starts to stall and prints
> the below messages. And the system never recovers. I can neither
> poweroff the guest nor the host. So I have resort to external power
> reset of the host.
>
> ==
> [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
> [kworker/25:1:454]
> [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
> shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
> sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
> d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
> raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
> drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
> ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
> mdio_thunder mdio_cavium
>
> [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
> 4.11.0-rc3-next-20170323 #1
> [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 
> 23 2017
> [  116.207010] Workqueue: events netstamp_clear
> [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
> [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
> [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
> [  116.207026] pc : [] lr : []
> pstate: 8145
> [  116.207028] sp : 801f901a7be0
> [  116.207030] x29: 801f901a7be0 x28: 09139000
> [  116.207036] x27: 09139434 x26: 0080
> [  116.207041] x25:  x24: 081565d0
> [  116.207047] x23: 0001 x22: 08e11e00
> [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
> [  116.207057] x19: 09138e38 x18: 0a03
> [  116.207063] x17: b77c9028 x16: 082e81d8
> [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
> [  116.207073] x13: 58dabe4c x12: 0018
> [  116.207079] x11: 366e2f04 x10: 09f0
> [  116.207084] x9 : 801f901a7d30 x8 : 0002
> [  116.207089] x7 :  x6 : 
> [  116.207095] x5 :  x4 : 0020
> [  116.207100] x3 : 0020 x2 : 
> [  116.207105] x1 : 801f6d682578 x0 : 0003
>
> [  150.443116] INFO: rcu_sched self-detected stall on CPU
> [  150.448261]  25-...: (14997 ticks this GP)
> idle=47a/141/0 softirq=349/349 fqs=7495
> [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
> [  150.451123]  25-...: (14997 ticks this GP)
> idle=47a/141/0 softirq=349/349 fqs=7495
> [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
> [  150.451136] Task dump for CPU 25:
> [  150.451138] kworker/25:1R  running task0   454  2 
> 0x0002
> [  150.451155] Workqueue: events netstamp_clear
> [  150.451158] Call trace:
> [  150.451164] [] __switch_to+0x90/0xa8
> [  150.451172] [] static_key_slow_inc+0x128/0x138
> [  150.451175] [] static_key_enable+0x34/0x60
> [  150.451178] [] netstamp_clear+0x68/0x80
> [  150.451181] [] process_one_work+0x158/0x478
> [  150.451183] [] worker_thread+0x50/0x4a8
> [  150.451187] [] kthread+0x108/0x138
> [  150.451190] [] ret_from_fork+0x10/0x50
> [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
> [  150.482242] Task dump for CPU 25:
> [  150.482245] kworker/25:1R  running task0   454  2 
> 0x0002
> [  150.482259] Workqueue: events netstamp_clear
> [  150.482264] Call trace:
> [  150.482271] [] dump_backtrace+0x0/0x2b0
> [  150.482277] [] show_stack+0x24/0x30
> [  150.482281] [] sched_show_task+0x128/0x178
> [  150.482285] [] dump_cpu_task+0x48/0x58
> [  150.482288] [] rcu_dump_cpu_stacks+0xa0/0xe8
> [  150.482297] [] rcu_check_callbacks+0x774/0x938
> [  150.482305] [] 

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Laszlo Ersek]

On 03/29/17 20:56, Christoffer Dall wrote:
> On Tue, Mar 28, 2017 at 01:24:15PM -0700, Radha Mohan wrote:
>> On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  wrote:
>>> Hi Radha,
>>>
>>> On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
 Hi,
 I am seeing an issue with qemu-system-aarch64 when using pflash
 (booting kernel via UEFI bios).

 Host kernel: 4.11.0-rc3-next-20170323
 Qemu version: v2.9.0-rc1

 Command used:
 ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
 virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
 if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
 virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
 /root/flash1.img


 As soon as the guest kernel boots the host starts to stall and prints
 the below messages. And the system never recovers. I can neither
 poweroff the guest nor the host. So I have resort to external power
 reset of the host.

 ==
 [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
 [kworker/25:1:454]
 [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
 shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
 sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
 d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
 raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
 ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
 mdio_thunder mdio_cavium

 [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
 4.11.0-rc3-next-20170323 #1
 [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 
 23 2017
 [  116.207010] Workqueue: events netstamp_clear
 [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
 [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
 [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
 [  116.207026] pc : [] lr : []
 pstate: 8145
 [  116.207028] sp : 801f901a7be0
 [  116.207030] x29: 801f901a7be0 x28: 09139000
 [  116.207036] x27: 09139434 x26: 0080
 [  116.207041] x25:  x24: 081565d0
 [  116.207047] x23: 0001 x22: 08e11e00
 [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
 [  116.207057] x19: 09138e38 x18: 0a03
 [  116.207063] x17: b77c9028 x16: 082e81d8
 [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
 [  116.207073] x13: 58dabe4c x12: 0018
 [  116.207079] x11: 366e2f04 x10: 09f0
 [  116.207084] x9 : 801f901a7d30 x8 : 0002
 [  116.207089] x7 :  x6 : 
 [  116.207095] x5 :  x4 : 0020
 [  116.207100] x3 : 0020 x2 : 
 [  116.207105] x1 : 801f6d682578 x0 : 0003

 [  150.443116] INFO: rcu_sched self-detected stall on CPU
 [  150.448261]  25-...: (14997 ticks this GP)
 idle=47a/141/0 softirq=349/349 fqs=7495
 [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
 [  150.451123]  25-...: (14997 ticks this GP)
 idle=47a/141/0 softirq=349/349 fqs=7495
 [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
 [  150.451136] Task dump for CPU 25:
 [  150.451138] kworker/25:1R  running task0   454  2 
 0x0002
 [  150.451155] Workqueue: events netstamp_clear
 [  150.451158] Call trace:
 [  150.451164] [] __switch_to+0x90/0xa8
 [  150.451172] [] static_key_slow_inc+0x128/0x138
 [  150.451175] [] static_key_enable+0x34/0x60
 [  150.451178] [] netstamp_clear+0x68/0x80
 [  150.451181] [] process_one_work+0x158/0x478
 [  150.451183] [] worker_thread+0x50/0x4a8
 [  150.451187] [] kthread+0x108/0x138
 [  150.451190] [] ret_from_fork+0x10/0x50
 [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
 [  150.482242] Task dump for CPU 25:
 [  150.482245] kworker/25:1R  running task0   454  2 
 0x0002
 [  150.482259] Workqueue: events netstamp_clear
 [  150.482264] Call trace:
 [  150.482271] [] dump_backtrace+0x0/0x2b0
 [  150.482277] [] show_stack+0x24/0x30
 [  150.482281] [] sched_show_task+0x128/0x178
 [  150.482285] [] dump_cpu_task+0x48/0x58
 [  150.482288] [] rcu_dump_cpu_stacks+0xa0/0xe8
 [  150.482297] [] rcu_check_callbacks+0x774/0x938
 [  150.482305] [] update_process_times+0x34/0x60
 [  150.482314] [] tick_sched_handle.isra.7+0x38/0x70
 [  150.482319] [] tick_sched_timer+0x4c/0x98
 [  150.482324] [] 

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Marc Zyngier]

On 29/03/17 19:56, Christoffer Dall wrote:
> On Tue, Mar 28, 2017 at 01:24:15PM -0700, Radha Mohan wrote:
>> On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  wrote:
>>> Hi Radha,
>>>
>>> On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
 Hi,
 I am seeing an issue with qemu-system-aarch64 when using pflash
 (booting kernel via UEFI bios).

 Host kernel: 4.11.0-rc3-next-20170323
 Qemu version: v2.9.0-rc1

 Command used:
 ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
 virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
 if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
 virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
 /root/flash1.img


 As soon as the guest kernel boots the host starts to stall and prints
 the below messages. And the system never recovers. I can neither
 poweroff the guest nor the host. So I have resort to external power
 reset of the host.

 ==
 [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
 [kworker/25:1:454]
 [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
 shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
 sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
 d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
 raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
 ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
 mdio_thunder mdio_cavium

 [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
 4.11.0-rc3-next-20170323 #1
 [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 
 23 2017
 [  116.207010] Workqueue: events netstamp_clear
 [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
 [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
 [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
 [  116.207026] pc : [] lr : []
 pstate: 8145
 [  116.207028] sp : 801f901a7be0
 [  116.207030] x29: 801f901a7be0 x28: 09139000
 [  116.207036] x27: 09139434 x26: 0080
 [  116.207041] x25:  x24: 081565d0
 [  116.207047] x23: 0001 x22: 08e11e00
 [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
 [  116.207057] x19: 09138e38 x18: 0a03
 [  116.207063] x17: b77c9028 x16: 082e81d8
 [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
 [  116.207073] x13: 58dabe4c x12: 0018
 [  116.207079] x11: 366e2f04 x10: 09f0
 [  116.207084] x9 : 801f901a7d30 x8 : 0002
 [  116.207089] x7 :  x6 : 
 [  116.207095] x5 :  x4 : 0020
 [  116.207100] x3 : 0020 x2 : 
 [  116.207105] x1 : 801f6d682578 x0 : 0003

 [  150.443116] INFO: rcu_sched self-detected stall on CPU
 [  150.448261]  25-...: (14997 ticks this GP)
 idle=47a/141/0 softirq=349/349 fqs=7495
 [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
 [  150.451123]  25-...: (14997 ticks this GP)
 idle=47a/141/0 softirq=349/349 fqs=7495
 [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
 [  150.451136] Task dump for CPU 25:
 [  150.451138] kworker/25:1R  running task0   454  2 
 0x0002
 [  150.451155] Workqueue: events netstamp_clear
 [  150.451158] Call trace:
 [  150.451164] [] __switch_to+0x90/0xa8
 [  150.451172] [] static_key_slow_inc+0x128/0x138
 [  150.451175] [] static_key_enable+0x34/0x60
 [  150.451178] [] netstamp_clear+0x68/0x80
 [  150.451181] [] process_one_work+0x158/0x478
 [  150.451183] [] worker_thread+0x50/0x4a8
 [  150.451187] [] kthread+0x108/0x138
 [  150.451190] [] ret_from_fork+0x10/0x50
 [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
 [  150.482242] Task dump for CPU 25:
 [  150.482245] kworker/25:1R  running task0   454  2 
 0x0002
 [  150.482259] Workqueue: events netstamp_clear
 [  150.482264] Call trace:
 [  150.482271] [] dump_backtrace+0x0/0x2b0
 [  150.482277] [] show_stack+0x24/0x30
 [  150.482281] [] sched_show_task+0x128/0x178
 [  150.482285] [] dump_cpu_task+0x48/0x58
 [  150.482288] [] rcu_dump_cpu_stacks+0xa0/0xe8
 [  150.482297] [] rcu_check_callbacks+0x774/0x938
 [  150.482305] [] update_process_times+0x34/0x60
 [  150.482314] [] tick_sched_handle.isra.7+0x38/0x70
 [  150.482319] [] tick_sched_timer+0x4c/0x98
 [  150.482324] [] 

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Radha Mohan]

On Wed, Mar 29, 2017 at 2:06 PM, Christoffer Dall  wrote:
> On Wed, Mar 29, 2017 at 01:51:19PM -0700, Radha Mohan wrote:
>> On Wed, Mar 29, 2017 at 11:56 AM, Christoffer Dall  wrote:
>> > On Tue, Mar 28, 2017 at 01:24:15PM -0700, Radha Mohan wrote:
>> >> On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  
>> >> wrote:
>> >> > Hi Radha,
>> >> >
>> >> > On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
>> >> >> Hi,
>> >> >> I am seeing an issue with qemu-system-aarch64 when using pflash
>> >> >> (booting kernel via UEFI bios).
>> >> >>
>> >> >> Host kernel: 4.11.0-rc3-next-20170323
>> >> >> Qemu version: v2.9.0-rc1
>> >> >>
>> >> >> Command used:
>> >> >> ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
>> >> >> virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
>> >> >> if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
>> >> >> virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
>> >> >> /root/flash1.img
>> >> >>
>> >> >>
>> >> >> As soon as the guest kernel boots the host starts to stall and prints
>> >> >> the below messages. And the system never recovers. I can neither
>> >> >> poweroff the guest nor the host. So I have resort to external power
>> >> >> reset of the host.
>> >> >>
>> >> >> ==
>> >> >> [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
>> >> >> [kworker/25:1:454]
>> >> >> [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
>> >> >> shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
>> >> >> sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
>> >> >> d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
>> >> >> raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
>> >> >> drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
>> >> >> ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
>> >> >> mdio_thunder mdio_cavium
>> >> >>
>> >> >> [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
>> >> >> 4.11.0-rc3-next-20170323 #1
>> >> >> [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 
>> >> >> Feb 23 2017
>> >> >> [  116.207010] Workqueue: events netstamp_clear
>> >> >> [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
>> >> >> [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
>> >> >> [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
>> >> >> [  116.207026] pc : [] lr : []
>> >> >> pstate: 8145
>> >> >> [  116.207028] sp : 801f901a7be0
>> >> >> [  116.207030] x29: 801f901a7be0 x28: 09139000
>> >> >> [  116.207036] x27: 09139434 x26: 0080
>> >> >> [  116.207041] x25:  x24: 081565d0
>> >> >> [  116.207047] x23: 0001 x22: 08e11e00
>> >> >> [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
>> >> >> [  116.207057] x19: 09138e38 x18: 0a03
>> >> >> [  116.207063] x17: b77c9028 x16: 082e81d8
>> >> >> [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
>> >> >> [  116.207073] x13: 58dabe4c x12: 0018
>> >> >> [  116.207079] x11: 366e2f04 x10: 09f0
>> >> >> [  116.207084] x9 : 801f901a7d30 x8 : 0002
>> >> >> [  116.207089] x7 :  x6 : 
>> >> >> [  116.207095] x5 :  x4 : 0020
>> >> >> [  116.207100] x3 : 0020 x2 : 
>> >> >> [  116.207105] x1 : 801f6d682578 x0 : 0003
>> >> >>
>> >> >> [  150.443116] INFO: rcu_sched self-detected stall on CPU
>> >> >> [  150.448261]  25-...: (14997 ticks this GP)
>> >> >> idle=47a/141/0 softirq=349/349 fqs=7495
>> >> >> [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
>> >> >> [  150.451123]  25-...: (14997 ticks this GP)
>> >> >> idle=47a/141/0 softirq=349/349 fqs=7495
>> >> >> [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
>> >> >> [  150.451136] Task dump for CPU 25:
>> >> >> [  150.451138] kworker/25:1R  running task0   454  2 
>> >> >> 0x0002
>> >> >> [  150.451155] Workqueue: events netstamp_clear
>> >> >> [  150.451158] Call trace:
>> >> >> [  150.451164] [] __switch_to+0x90/0xa8
>> >> >> [  150.451172] [] static_key_slow_inc+0x128/0x138
>> >> >> [  150.451175] [] static_key_enable+0x34/0x60
>> >> >> [  150.451178] [] netstamp_clear+0x68/0x80
>> >> >> [  150.451181] [] process_one_work+0x158/0x478
>> >> >> [  150.451183] [] worker_thread+0x50/0x4a8
>> >> >> [  150.451187] [] kthread+0x108/0x138
>> >> >> [  150.451190] [] ret_from_fork+0x10/0x50
>> >> >> [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
>> >> >> [  150.482242] Task dump for CPU 25:
>> >> >> [  150.482245] kworker/25:1R  running task0   454  2 
>> >> >> 0x0002
>> >> >> [  150.482259] Workqueue: events netstamp_clear
>> >> >> 

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Christoffer Dall]

On Wed, Mar 29, 2017 at 01:51:19PM -0700, Radha Mohan wrote:
> On Wed, Mar 29, 2017 at 11:56 AM, Christoffer Dall  wrote:
> > On Tue, Mar 28, 2017 at 01:24:15PM -0700, Radha Mohan wrote:
> >> On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  wrote:
> >> > Hi Radha,
> >> >
> >> > On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
> >> >> Hi,
> >> >> I am seeing an issue with qemu-system-aarch64 when using pflash
> >> >> (booting kernel via UEFI bios).
> >> >>
> >> >> Host kernel: 4.11.0-rc3-next-20170323
> >> >> Qemu version: v2.9.0-rc1
> >> >>
> >> >> Command used:
> >> >> ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
> >> >> virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
> >> >> if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
> >> >> virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
> >> >> /root/flash1.img
> >> >>
> >> >>
> >> >> As soon as the guest kernel boots the host starts to stall and prints
> >> >> the below messages. And the system never recovers. I can neither
> >> >> poweroff the guest nor the host. So I have resort to external power
> >> >> reset of the host.
> >> >>
> >> >> ==
> >> >> [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
> >> >> [kworker/25:1:454]
> >> >> [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
> >> >> shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
> >> >> sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
> >> >> d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
> >> >> raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
> >> >> drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
> >> >> ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
> >> >> mdio_thunder mdio_cavium
> >> >>
> >> >> [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
> >> >> 4.11.0-rc3-next-20170323 #1
> >> >> [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 
> >> >> Feb 23 2017
> >> >> [  116.207010] Workqueue: events netstamp_clear
> >> >> [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
> >> >> [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
> >> >> [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
> >> >> [  116.207026] pc : [] lr : []
> >> >> pstate: 8145
> >> >> [  116.207028] sp : 801f901a7be0
> >> >> [  116.207030] x29: 801f901a7be0 x28: 09139000
> >> >> [  116.207036] x27: 09139434 x26: 0080
> >> >> [  116.207041] x25:  x24: 081565d0
> >> >> [  116.207047] x23: 0001 x22: 08e11e00
> >> >> [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
> >> >> [  116.207057] x19: 09138e38 x18: 0a03
> >> >> [  116.207063] x17: b77c9028 x16: 082e81d8
> >> >> [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
> >> >> [  116.207073] x13: 58dabe4c x12: 0018
> >> >> [  116.207079] x11: 366e2f04 x10: 09f0
> >> >> [  116.207084] x9 : 801f901a7d30 x8 : 0002
> >> >> [  116.207089] x7 :  x6 : 
> >> >> [  116.207095] x5 :  x4 : 0020
> >> >> [  116.207100] x3 : 0020 x2 : 
> >> >> [  116.207105] x1 : 801f6d682578 x0 : 0003
> >> >>
> >> >> [  150.443116] INFO: rcu_sched self-detected stall on CPU
> >> >> [  150.448261]  25-...: (14997 ticks this GP)
> >> >> idle=47a/141/0 softirq=349/349 fqs=7495
> >> >> [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
> >> >> [  150.451123]  25-...: (14997 ticks this GP)
> >> >> idle=47a/141/0 softirq=349/349 fqs=7495
> >> >> [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
> >> >> [  150.451136] Task dump for CPU 25:
> >> >> [  150.451138] kworker/25:1R  running task0   454  2 
> >> >> 0x0002
> >> >> [  150.451155] Workqueue: events netstamp_clear
> >> >> [  150.451158] Call trace:
> >> >> [  150.451164] [] __switch_to+0x90/0xa8
> >> >> [  150.451172] [] static_key_slow_inc+0x128/0x138
> >> >> [  150.451175] [] static_key_enable+0x34/0x60
> >> >> [  150.451178] [] netstamp_clear+0x68/0x80
> >> >> [  150.451181] [] process_one_work+0x158/0x478
> >> >> [  150.451183] [] worker_thread+0x50/0x4a8
> >> >> [  150.451187] [] kthread+0x108/0x138
> >> >> [  150.451190] [] ret_from_fork+0x10/0x50
> >> >> [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
> >> >> [  150.482242] Task dump for CPU 25:
> >> >> [  150.482245] kworker/25:1R  running task0   454  2 
> >> >> 0x0002
> >> >> [  150.482259] Workqueue: events netstamp_clear
> >> >> [  150.482264] Call trace:
> >> >> [  150.482271] [] dump_backtrace+0x0/0x2b0
> >> >> [  150.482277] [] show_stack+0x24/0x30
> >> >> [  150.482281] [] 

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Radha Mohan]

On Wed, Mar 29, 2017 at 11:56 AM, Christoffer Dall  wrote:
> On Tue, Mar 28, 2017 at 01:24:15PM -0700, Radha Mohan wrote:
>> On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  wrote:
>> > Hi Radha,
>> >
>> > On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
>> >> Hi,
>> >> I am seeing an issue with qemu-system-aarch64 when using pflash
>> >> (booting kernel via UEFI bios).
>> >>
>> >> Host kernel: 4.11.0-rc3-next-20170323
>> >> Qemu version: v2.9.0-rc1
>> >>
>> >> Command used:
>> >> ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
>> >> virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
>> >> if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
>> >> virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
>> >> /root/flash1.img
>> >>
>> >>
>> >> As soon as the guest kernel boots the host starts to stall and prints
>> >> the below messages. And the system never recovers. I can neither
>> >> poweroff the guest nor the host. So I have resort to external power
>> >> reset of the host.
>> >>
>> >> ==
>> >> [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
>> >> [kworker/25:1:454]
>> >> [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
>> >> shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
>> >> sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
>> >> d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
>> >> raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
>> >> drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
>> >> ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
>> >> mdio_thunder mdio_cavium
>> >>
>> >> [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
>> >> 4.11.0-rc3-next-20170323 #1
>> >> [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 
>> >> 23 2017
>> >> [  116.207010] Workqueue: events netstamp_clear
>> >> [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
>> >> [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
>> >> [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
>> >> [  116.207026] pc : [] lr : []
>> >> pstate: 8145
>> >> [  116.207028] sp : 801f901a7be0
>> >> [  116.207030] x29: 801f901a7be0 x28: 09139000
>> >> [  116.207036] x27: 09139434 x26: 0080
>> >> [  116.207041] x25:  x24: 081565d0
>> >> [  116.207047] x23: 0001 x22: 08e11e00
>> >> [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
>> >> [  116.207057] x19: 09138e38 x18: 0a03
>> >> [  116.207063] x17: b77c9028 x16: 082e81d8
>> >> [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
>> >> [  116.207073] x13: 58dabe4c x12: 0018
>> >> [  116.207079] x11: 366e2f04 x10: 09f0
>> >> [  116.207084] x9 : 801f901a7d30 x8 : 0002
>> >> [  116.207089] x7 :  x6 : 
>> >> [  116.207095] x5 :  x4 : 0020
>> >> [  116.207100] x3 : 0020 x2 : 
>> >> [  116.207105] x1 : 801f6d682578 x0 : 0003
>> >>
>> >> [  150.443116] INFO: rcu_sched self-detected stall on CPU
>> >> [  150.448261]  25-...: (14997 ticks this GP)
>> >> idle=47a/141/0 softirq=349/349 fqs=7495
>> >> [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
>> >> [  150.451123]  25-...: (14997 ticks this GP)
>> >> idle=47a/141/0 softirq=349/349 fqs=7495
>> >> [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
>> >> [  150.451136] Task dump for CPU 25:
>> >> [  150.451138] kworker/25:1R  running task0   454  2 
>> >> 0x0002
>> >> [  150.451155] Workqueue: events netstamp_clear
>> >> [  150.451158] Call trace:
>> >> [  150.451164] [] __switch_to+0x90/0xa8
>> >> [  150.451172] [] static_key_slow_inc+0x128/0x138
>> >> [  150.451175] [] static_key_enable+0x34/0x60
>> >> [  150.451178] [] netstamp_clear+0x68/0x80
>> >> [  150.451181] [] process_one_work+0x158/0x478
>> >> [  150.451183] [] worker_thread+0x50/0x4a8
>> >> [  150.451187] [] kthread+0x108/0x138
>> >> [  150.451190] [] ret_from_fork+0x10/0x50
>> >> [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
>> >> [  150.482242] Task dump for CPU 25:
>> >> [  150.482245] kworker/25:1R  running task0   454  2 
>> >> 0x0002
>> >> [  150.482259] Workqueue: events netstamp_clear
>> >> [  150.482264] Call trace:
>> >> [  150.482271] [] dump_backtrace+0x0/0x2b0
>> >> [  150.482277] [] show_stack+0x24/0x30
>> >> [  150.482281] [] sched_show_task+0x128/0x178
>> >> [  150.482285] [] dump_cpu_task+0x48/0x58
>> >> [  150.482288] [] rcu_dump_cpu_stacks+0xa0/0xe8
>> >> [  150.482297] [] rcu_check_callbacks+0x774/0x938
>> >> [  150.482305] [] update_process_times+0x34/0x60
>> >> [  150.482314] [] 

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Christoffer Dall]

On Tue, Mar 28, 2017 at 01:24:15PM -0700, Radha Mohan wrote:
> On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  wrote:
> > Hi Radha,
> >
> > On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
> >> Hi,
> >> I am seeing an issue with qemu-system-aarch64 when using pflash
> >> (booting kernel via UEFI bios).
> >>
> >> Host kernel: 4.11.0-rc3-next-20170323
> >> Qemu version: v2.9.0-rc1
> >>
> >> Command used:
> >> ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
> >> virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
> >> if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
> >> virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
> >> /root/flash1.img
> >>
> >>
> >> As soon as the guest kernel boots the host starts to stall and prints
> >> the below messages. And the system never recovers. I can neither
> >> poweroff the guest nor the host. So I have resort to external power
> >> reset of the host.
> >>
> >> ==
> >> [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
> >> [kworker/25:1:454]
> >> [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
> >> shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
> >> sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
> >> d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
> >> raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
> >> drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
> >> ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
> >> mdio_thunder mdio_cavium
> >>
> >> [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
> >> 4.11.0-rc3-next-20170323 #1
> >> [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 
> >> 23 2017
> >> [  116.207010] Workqueue: events netstamp_clear
> >> [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
> >> [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
> >> [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
> >> [  116.207026] pc : [] lr : []
> >> pstate: 8145
> >> [  116.207028] sp : 801f901a7be0
> >> [  116.207030] x29: 801f901a7be0 x28: 09139000
> >> [  116.207036] x27: 09139434 x26: 0080
> >> [  116.207041] x25:  x24: 081565d0
> >> [  116.207047] x23: 0001 x22: 08e11e00
> >> [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
> >> [  116.207057] x19: 09138e38 x18: 0a03
> >> [  116.207063] x17: b77c9028 x16: 082e81d8
> >> [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
> >> [  116.207073] x13: 58dabe4c x12: 0018
> >> [  116.207079] x11: 366e2f04 x10: 09f0
> >> [  116.207084] x9 : 801f901a7d30 x8 : 0002
> >> [  116.207089] x7 :  x6 : 
> >> [  116.207095] x5 :  x4 : 0020
> >> [  116.207100] x3 : 0020 x2 : 
> >> [  116.207105] x1 : 801f6d682578 x0 : 0003
> >>
> >> [  150.443116] INFO: rcu_sched self-detected stall on CPU
> >> [  150.448261]  25-...: (14997 ticks this GP)
> >> idle=47a/141/0 softirq=349/349 fqs=7495
> >> [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
> >> [  150.451123]  25-...: (14997 ticks this GP)
> >> idle=47a/141/0 softirq=349/349 fqs=7495
> >> [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
> >> [  150.451136] Task dump for CPU 25:
> >> [  150.451138] kworker/25:1R  running task0   454  2 
> >> 0x0002
> >> [  150.451155] Workqueue: events netstamp_clear
> >> [  150.451158] Call trace:
> >> [  150.451164] [] __switch_to+0x90/0xa8
> >> [  150.451172] [] static_key_slow_inc+0x128/0x138
> >> [  150.451175] [] static_key_enable+0x34/0x60
> >> [  150.451178] [] netstamp_clear+0x68/0x80
> >> [  150.451181] [] process_one_work+0x158/0x478
> >> [  150.451183] [] worker_thread+0x50/0x4a8
> >> [  150.451187] [] kthread+0x108/0x138
> >> [  150.451190] [] ret_from_fork+0x10/0x50
> >> [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
> >> [  150.482242] Task dump for CPU 25:
> >> [  150.482245] kworker/25:1R  running task0   454  2 
> >> 0x0002
> >> [  150.482259] Workqueue: events netstamp_clear
> >> [  150.482264] Call trace:
> >> [  150.482271] [] dump_backtrace+0x0/0x2b0
> >> [  150.482277] [] show_stack+0x24/0x30
> >> [  150.482281] [] sched_show_task+0x128/0x178
> >> [  150.482285] [] dump_cpu_task+0x48/0x58
> >> [  150.482288] [] rcu_dump_cpu_stacks+0xa0/0xe8
> >> [  150.482297] [] rcu_check_callbacks+0x774/0x938
> >> [  150.482305] [] update_process_times+0x34/0x60
> >> [  150.482314] [] tick_sched_handle.isra.7+0x38/0x70
> >> [  150.482319] [] tick_sched_timer+0x4c/0x98
> >> [  150.482324] [] __hrtimer_run_queues+0xd8/0x2b8
> >> [  150.482328] [] 

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Peter Maydell]

On 29 March 2017 at 19:17, Radha Mohan  wrote:
>> I will also try with qemu 2.8 and see.
>
> Same issue with 2.8.0 and 2.7.1 qemu version.

Thanks for checking -- good to know it's not a regression
on the QEMU side. (If you can soft-lockup the host it's
clearly a host kernel bug, but it might have been that
we changed something recently to provoke it.)

thanks
-- PMM

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Radha Mohan]

On Tue, Mar 28, 2017 at 1:24 PM, Radha Mohan  wrote:
> On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  wrote:
>> Hi Radha,
>>
>> On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
>>> Hi,
>>> I am seeing an issue with qemu-system-aarch64 when using pflash
>>> (booting kernel via UEFI bios).
>>>
>>> Host kernel: 4.11.0-rc3-next-20170323
>>> Qemu version: v2.9.0-rc1
>>>
>>> Command used:
>>> ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
>>> virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
>>> if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
>>> virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
>>> /root/flash1.img
>>>
>>>
>>> As soon as the guest kernel boots the host starts to stall and prints
>>> the below messages. And the system never recovers. I can neither
>>> poweroff the guest nor the host. So I have resort to external power
>>> reset of the host.
>>>
>>> ==
>>> [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
>>> [kworker/25:1:454]
>>> [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
>>> shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
>>> sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
>>> d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
>>> raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
>>> drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
>>> ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
>>> mdio_thunder mdio_cavium
>>>
>>> [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
>>> 4.11.0-rc3-next-20170323 #1
>>> [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 23 
>>> 2017
>>> [  116.207010] Workqueue: events netstamp_clear
>>> [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
>>> [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
>>> [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
>>> [  116.207026] pc : [] lr : []
>>> pstate: 8145
>>> [  116.207028] sp : 801f901a7be0
>>> [  116.207030] x29: 801f901a7be0 x28: 09139000
>>> [  116.207036] x27: 09139434 x26: 0080
>>> [  116.207041] x25:  x24: 081565d0
>>> [  116.207047] x23: 0001 x22: 08e11e00
>>> [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
>>> [  116.207057] x19: 09138e38 x18: 0a03
>>> [  116.207063] x17: b77c9028 x16: 082e81d8
>>> [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
>>> [  116.207073] x13: 58dabe4c x12: 0018
>>> [  116.207079] x11: 366e2f04 x10: 09f0
>>> [  116.207084] x9 : 801f901a7d30 x8 : 0002
>>> [  116.207089] x7 :  x6 : 
>>> [  116.207095] x5 :  x4 : 0020
>>> [  116.207100] x3 : 0020 x2 : 
>>> [  116.207105] x1 : 801f6d682578 x0 : 0003
>>>
>>> [  150.443116] INFO: rcu_sched self-detected stall on CPU
>>> [  150.448261]  25-...: (14997 ticks this GP)
>>> idle=47a/141/0 softirq=349/349 fqs=7495
>>> [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
>>> [  150.451123]  25-...: (14997 ticks this GP)
>>> idle=47a/141/0 softirq=349/349 fqs=7495
>>> [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
>>> [  150.451136] Task dump for CPU 25:
>>> [  150.451138] kworker/25:1R  running task0   454  2 
>>> 0x0002
>>> [  150.451155] Workqueue: events netstamp_clear
>>> [  150.451158] Call trace:
>>> [  150.451164] [] __switch_to+0x90/0xa8
>>> [  150.451172] [] static_key_slow_inc+0x128/0x138
>>> [  150.451175] [] static_key_enable+0x34/0x60
>>> [  150.451178] [] netstamp_clear+0x68/0x80
>>> [  150.451181] [] process_one_work+0x158/0x478
>>> [  150.451183] [] worker_thread+0x50/0x4a8
>>> [  150.451187] [] kthread+0x108/0x138
>>> [  150.451190] [] ret_from_fork+0x10/0x50
>>> [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
>>> [  150.482242] Task dump for CPU 25:
>>> [  150.482245] kworker/25:1R  running task0   454  2 
>>> 0x0002
>>> [  150.482259] Workqueue: events netstamp_clear
>>> [  150.482264] Call trace:
>>> [  150.482271] [] dump_backtrace+0x0/0x2b0
>>> [  150.482277] [] show_stack+0x24/0x30
>>> [  150.482281] [] sched_show_task+0x128/0x178
>>> [  150.482285] [] dump_cpu_task+0x48/0x58
>>> [  150.482288] [] rcu_dump_cpu_stacks+0xa0/0xe8
>>> [  150.482297] [] rcu_check_callbacks+0x774/0x938
>>> [  150.482305] [] update_process_times+0x34/0x60
>>> [  150.482314] [] tick_sched_handle.isra.7+0x38/0x70
>>> [  150.482319] [] tick_sched_timer+0x4c/0x98
>>> [  150.482324] [] __hrtimer_run_queues+0xd8/0x2b8
>>> [  150.482328] [] hrtimer_interrupt+0xa8/0x228
>>> [  150.482334] [] arch_timer_handler_phys+0x3c/0x50
>>> [  150.482341] [] 

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Christoffer Dall]

Hi Radha,

On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
> Hi,
> I am seeing an issue with qemu-system-aarch64 when using pflash
> (booting kernel via UEFI bios).
> 
> Host kernel: 4.11.0-rc3-next-20170323
> Qemu version: v2.9.0-rc1
> 
> Command used:
> ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
> virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
> if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
> virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
> /root/flash1.img
> 
> 
> As soon as the guest kernel boots the host starts to stall and prints
> the below messages. And the system never recovers. I can neither
> poweroff the guest nor the host. So I have resort to external power
> reset of the host.
> 
> ==
> [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
> [kworker/25:1:454]
> [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
> shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
> sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
> d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
> raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
> drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
> ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
> mdio_thunder mdio_cavium
> 
> [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
> 4.11.0-rc3-next-20170323 #1
> [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 23 
> 2017
> [  116.207010] Workqueue: events netstamp_clear
> [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
> [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
> [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
> [  116.207026] pc : [] lr : []
> pstate: 8145
> [  116.207028] sp : 801f901a7be0
> [  116.207030] x29: 801f901a7be0 x28: 09139000
> [  116.207036] x27: 09139434 x26: 0080
> [  116.207041] x25:  x24: 081565d0
> [  116.207047] x23: 0001 x22: 08e11e00
> [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
> [  116.207057] x19: 09138e38 x18: 0a03
> [  116.207063] x17: b77c9028 x16: 082e81d8
> [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
> [  116.207073] x13: 58dabe4c x12: 0018
> [  116.207079] x11: 366e2f04 x10: 09f0
> [  116.207084] x9 : 801f901a7d30 x8 : 0002
> [  116.207089] x7 :  x6 : 
> [  116.207095] x5 :  x4 : 0020
> [  116.207100] x3 : 0020 x2 : 
> [  116.207105] x1 : 801f6d682578 x0 : 0003
> 
> [  150.443116] INFO: rcu_sched self-detected stall on CPU
> [  150.448261]  25-...: (14997 ticks this GP)
> idle=47a/141/0 softirq=349/349 fqs=7495
> [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
> [  150.451123]  25-...: (14997 ticks this GP)
> idle=47a/141/0 softirq=349/349 fqs=7495
> [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
> [  150.451136] Task dump for CPU 25:
> [  150.451138] kworker/25:1R  running task0   454  2 
> 0x0002
> [  150.451155] Workqueue: events netstamp_clear
> [  150.451158] Call trace:
> [  150.451164] [] __switch_to+0x90/0xa8
> [  150.451172] [] static_key_slow_inc+0x128/0x138
> [  150.451175] [] static_key_enable+0x34/0x60
> [  150.451178] [] netstamp_clear+0x68/0x80
> [  150.451181] [] process_one_work+0x158/0x478
> [  150.451183] [] worker_thread+0x50/0x4a8
> [  150.451187] [] kthread+0x108/0x138
> [  150.451190] [] ret_from_fork+0x10/0x50
> [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
> [  150.482242] Task dump for CPU 25:
> [  150.482245] kworker/25:1R  running task0   454  2 
> 0x0002
> [  150.482259] Workqueue: events netstamp_clear
> [  150.482264] Call trace:
> [  150.482271] [] dump_backtrace+0x0/0x2b0
> [  150.482277] [] show_stack+0x24/0x30
> [  150.482281] [] sched_show_task+0x128/0x178
> [  150.482285] [] dump_cpu_task+0x48/0x58
> [  150.482288] [] rcu_dump_cpu_stacks+0xa0/0xe8
> [  150.482297] [] rcu_check_callbacks+0x774/0x938
> [  150.482305] [] update_process_times+0x34/0x60
> [  150.482314] [] tick_sched_handle.isra.7+0x38/0x70
> [  150.482319] [] tick_sched_timer+0x4c/0x98
> [  150.482324] [] __hrtimer_run_queues+0xd8/0x2b8
> [  150.482328] [] hrtimer_interrupt+0xa8/0x228
> [  150.482334] [] arch_timer_handler_phys+0x3c/0x50
> [  150.482341] [] handle_percpu_devid_irq+0x8c/0x230
> [  150.482344] [] generic_handle_irq+0x34/0x50
> [  150.482347] [] __handle_domain_irq+0x68/0xc0
> [  150.482351] [] gic_handle_irq+0xc4/0x170
> [  150.482356] Exception stack(0x801f901a7ab0 to 0x801f901a7be0)
> [  150.482360] 7aa0:
> 0003 801f6d682578
> [  150.482364] 7ac0: 

Re: [Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Radha Mohan]

On Tue, Mar 28, 2017 at 1:16 PM, Christoffer Dall  wrote:
> Hi Radha,
>
> On Tue, Mar 28, 2017 at 12:58:24PM -0700, Radha Mohan wrote:
>> Hi,
>> I am seeing an issue with qemu-system-aarch64 when using pflash
>> (booting kernel via UEFI bios).
>>
>> Host kernel: 4.11.0-rc3-next-20170323
>> Qemu version: v2.9.0-rc1
>>
>> Command used:
>> ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
>> virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
>> if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
>> virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
>> /root/flash1.img
>>
>>
>> As soon as the guest kernel boots the host starts to stall and prints
>> the below messages. And the system never recovers. I can neither
>> poweroff the guest nor the host. So I have resort to external power
>> reset of the host.
>>
>> ==
>> [  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
>> [kworker/25:1:454]
>> [  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
>> shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
>> sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
>> d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
>> raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
>> drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
>> ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
>> mdio_thunder mdio_cavium
>>
>> [  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
>> 4.11.0-rc3-next-20170323 #1
>> [  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 23 
>> 2017
>> [  116.207010] Workqueue: events netstamp_clear
>> [  116.207015] task: 801f906b5400 task.stack: 801f901a4000
>> [  116.207020] PC is at smp_call_function_many+0x284/0x2e8
>> [  116.207023] LR is at smp_call_function_many+0x244/0x2e8
>> [  116.207026] pc : [] lr : []
>> pstate: 8145
>> [  116.207028] sp : 801f901a7be0
>> [  116.207030] x29: 801f901a7be0 x28: 09139000
>> [  116.207036] x27: 09139434 x26: 0080
>> [  116.207041] x25:  x24: 081565d0
>> [  116.207047] x23: 0001 x22: 08e11e00
>> [  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
>> [  116.207057] x19: 09138e38 x18: 0a03
>> [  116.207063] x17: b77c9028 x16: 082e81d8
>> [  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
>> [  116.207073] x13: 58dabe4c x12: 0018
>> [  116.207079] x11: 366e2f04 x10: 09f0
>> [  116.207084] x9 : 801f901a7d30 x8 : 0002
>> [  116.207089] x7 :  x6 : 
>> [  116.207095] x5 :  x4 : 0020
>> [  116.207100] x3 : 0020 x2 : 
>> [  116.207105] x1 : 801f6d682578 x0 : 0003
>>
>> [  150.443116] INFO: rcu_sched self-detected stall on CPU
>> [  150.448261]  25-...: (14997 ticks this GP)
>> idle=47a/141/0 softirq=349/349 fqs=7495
>> [  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
>> [  150.451123]  25-...: (14997 ticks this GP)
>> idle=47a/141/0 softirq=349/349 fqs=7495
>> [  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
>> [  150.451136] Task dump for CPU 25:
>> [  150.451138] kworker/25:1R  running task0   454  2 
>> 0x0002
>> [  150.451155] Workqueue: events netstamp_clear
>> [  150.451158] Call trace:
>> [  150.451164] [] __switch_to+0x90/0xa8
>> [  150.451172] [] static_key_slow_inc+0x128/0x138
>> [  150.451175] [] static_key_enable+0x34/0x60
>> [  150.451178] [] netstamp_clear+0x68/0x80
>> [  150.451181] [] process_one_work+0x158/0x478
>> [  150.451183] [] worker_thread+0x50/0x4a8
>> [  150.451187] [] kthread+0x108/0x138
>> [  150.451190] [] ret_from_fork+0x10/0x50
>> [  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
>> [  150.482242] Task dump for CPU 25:
>> [  150.482245] kworker/25:1R  running task0   454  2 
>> 0x0002
>> [  150.482259] Workqueue: events netstamp_clear
>> [  150.482264] Call trace:
>> [  150.482271] [] dump_backtrace+0x0/0x2b0
>> [  150.482277] [] show_stack+0x24/0x30
>> [  150.482281] [] sched_show_task+0x128/0x178
>> [  150.482285] [] dump_cpu_task+0x48/0x58
>> [  150.482288] [] rcu_dump_cpu_stacks+0xa0/0xe8
>> [  150.482297] [] rcu_check_callbacks+0x774/0x938
>> [  150.482305] [] update_process_times+0x34/0x60
>> [  150.482314] [] tick_sched_handle.isra.7+0x38/0x70
>> [  150.482319] [] tick_sched_timer+0x4c/0x98
>> [  150.482324] [] __hrtimer_run_queues+0xd8/0x2b8
>> [  150.482328] [] hrtimer_interrupt+0xa8/0x228
>> [  150.482334] [] arch_timer_handler_phys+0x3c/0x50
>> [  150.482341] [] handle_percpu_devid_irq+0x8c/0x230
>> [  150.482344] [] generic_handle_irq+0x34/0x50
>> [  150.482347] [] __handle_domain_irq+0x68/0xc0
>> [  150.482351] [] 

[Qemu-devel] host stalls when qemu-system-aarch64 with kvm and pflash

[Radha Mohan]

Hi,
I am seeing an issue with qemu-system-aarch64 when using pflash
(booting kernel via UEFI bios).

Host kernel: 4.11.0-rc3-next-20170323
Qemu version: v2.9.0-rc1

Command used:
./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm -M
virt,gic_version=3 -nographic -smp 1 -m 2048 -drive
if=none,id=hd0,file=/root/zesty-server-cloudimg-arm64.img,id=0 -device
virtio-blk-device,drive=hd0 -pflash /root/flash0.img -pflash
/root/flash1.img


As soon as the guest kernel boots the host starts to stall and prints
the below messages. And the system never recovers. I can neither
poweroff the guest nor the host. So I have resort to external power
reset of the host.

==
[  116.199077] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s!
[kworker/25:1:454]
[  116.206901] Modules linked in: binfmt_misc nls_iso8859_1 aes_ce_blk
shpchp crypto_simd gpio_keys cryptd aes_ce_cipher ghash_ce sha2_ce
sha1_ce uio_pdrv_genirq uio autofs4 btrfs raid10 rai
d456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor
raid6_pq libcrc32c raid1 raid0 multipath linear ast i2c_algo_bit ttm
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_s
ys_fops drm nicvf ahci nicpf libahci thunder_bgx thunder_xcv
mdio_thunder mdio_cavium

[  116.206995] CPU: 25 PID: 454 Comm: kworker/25:1 Not tainted
4.11.0-rc3-next-20170323 #1
[  116.206997] Hardware name: www.cavium.com crb-1s/crb-1s, BIOS 0.3 Feb 23 2017
[  116.207010] Workqueue: events netstamp_clear
[  116.207015] task: 801f906b5400 task.stack: 801f901a4000
[  116.207020] PC is at smp_call_function_many+0x284/0x2e8
[  116.207023] LR is at smp_call_function_many+0x244/0x2e8
[  116.207026] pc : [] lr : []
pstate: 8145
[  116.207028] sp : 801f901a7be0
[  116.207030] x29: 801f901a7be0 x28: 09139000
[  116.207036] x27: 09139434 x26: 0080
[  116.207041] x25:  x24: 081565d0
[  116.207047] x23: 0001 x22: 08e11e00
[  116.207052] x21: 801f6d5cff00 x20: 801f6d5cff08
[  116.207057] x19: 09138e38 x18: 0a03
[  116.207063] x17: b77c9028 x16: 082e81d8
[  116.207068] x15: 3d0d6dd44d08 x14: 0036312196549b4a
[  116.207073] x13: 58dabe4c x12: 0018
[  116.207079] x11: 366e2f04 x10: 09f0
[  116.207084] x9 : 801f901a7d30 x8 : 0002
[  116.207089] x7 :  x6 : 
[  116.207095] x5 :  x4 : 0020
[  116.207100] x3 : 0020 x2 : 
[  116.207105] x1 : 801f6d682578 x0 : 0003

[  150.443116] INFO: rcu_sched self-detected stall on CPU
[  150.448261]  25-...: (14997 ticks this GP)
idle=47a/141/0 softirq=349/349 fqs=7495
[  150.451115] INFO: rcu_sched detected stalls on CPUs/tasks:
[  150.451123]  25-...: (14997 ticks this GP)
idle=47a/141/0 softirq=349/349 fqs=7495
[  150.451124]  (detected by 13, t=15002 jiffies, g=805, c=804, q=8384)
[  150.451136] Task dump for CPU 25:
[  150.451138] kworker/25:1R  running task0   454  2 0x0002
[  150.451155] Workqueue: events netstamp_clear
[  150.451158] Call trace:
[  150.451164] [] __switch_to+0x90/0xa8
[  150.451172] [] static_key_slow_inc+0x128/0x138
[  150.451175] [] static_key_enable+0x34/0x60
[  150.451178] [] netstamp_clear+0x68/0x80
[  150.451181] [] process_one_work+0x158/0x478
[  150.451183] [] worker_thread+0x50/0x4a8
[  150.451187] [] kthread+0x108/0x138
[  150.451190] [] ret_from_fork+0x10/0x50
[  150.477451]   (t=15008 jiffies g=805 c=804 q=8384)
[  150.482242] Task dump for CPU 25:
[  150.482245] kworker/25:1R  running task0   454  2 0x0002
[  150.482259] Workqueue: events netstamp_clear
[  150.482264] Call trace:
[  150.482271] [] dump_backtrace+0x0/0x2b0
[  150.482277] [] show_stack+0x24/0x30
[  150.482281] [] sched_show_task+0x128/0x178
[  150.482285] [] dump_cpu_task+0x48/0x58
[  150.482288] [] rcu_dump_cpu_stacks+0xa0/0xe8
[  150.482297] [] rcu_check_callbacks+0x774/0x938
[  150.482305] [] update_process_times+0x34/0x60
[  150.482314] [] tick_sched_handle.isra.7+0x38/0x70
[  150.482319] [] tick_sched_timer+0x4c/0x98
[  150.482324] [] __hrtimer_run_queues+0xd8/0x2b8
[  150.482328] [] hrtimer_interrupt+0xa8/0x228
[  150.482334] [] arch_timer_handler_phys+0x3c/0x50
[  150.482341] [] handle_percpu_devid_irq+0x8c/0x230
[  150.482344] [] generic_handle_irq+0x34/0x50
[  150.482347] [] __handle_domain_irq+0x68/0xc0
[  150.482351] [] gic_handle_irq+0xc4/0x170
[  150.482356] Exception stack(0x801f901a7ab0 to 0x801f901a7be0)
[  150.482360] 7aa0:
0003 801f6d682578
[  150.482364] 7ac0:  0020
0020 
[  150.482367] 7ae0:  
0002 801f901a7d30
[  150.482371] 7b00: 09f0 366e2f04
0018 58dabe4c
[  150.482375] 7b20: 0036312196549b4a