On 23/01/2019 04:10, jianchao.wang wrote:
> Looking through the log
> https://pastebin.ubuntu.com/p/YSm82GxhNW/
>
> rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
> rcu: 6-...0: (13995 ticks this GP) idle=e16/1/0x4000000000000000
> softirq=155/155 fqs=655
> rcu: (detected by 4, t=576151 jiffies, g=-391, q=18)
> Task dump for CPU 6:
> dd R running task 0 677 671 0x00000002
> Call trace:
> __switch_to+0x174/0x1e0
> ufshcd_queuecommand+0x84c/0x9a8
>
> The task is in RUNNING state when it was scheduled out.
> So it should be a preempt (the path is under preemptible rcu).
>
> And I wonder why it is not scheduled back for so long time that rcu stall was
> triggered.
> And who was occupying the cpu all the time ?
>
> Would you please try to show all running tasks on all cpu ?
>
> echo l > /proc/sysrq-trigger
>
> In addition, since the rcu grace period didn't pass, a lot of things could
> not be moved
> forward.
# echo w > /proc/sysrq-trigger
[ 517.619818] sysrq: Show Blocked State
[ 517.620542] task PC stack pid father
# echo l > /proc/sysrq-trigger
[ 530.692149] sysrq: Show backtrace of all active CPUs
# ps -o pid,user,stat,args
PID USER STAT COMMAND
1 root S init
2 root SW [kthreadd]
3 root IW< [rcu_gp]
4 root IW< [rcu_par_gp]
5 root IW [kworker/0:0-eve]
6 root IW< [kworker/0:0H]
7 root IW [kworker/u16:0-u]
8 root IW< [mm_percpu_wq]
9 root SW [ksoftirqd/0]
10 root IW [rcu_preempt]
11 root SW [migration/0]
12 root IW [kworker/0:1-rcu]
13 root SW [cpuhp/0]
14 root SW [cpuhp/1]
15 root SW [migration/1]
16 root SW [ksoftirqd/1]
17 root IW [kworker/1:0-eve]
18 root IW< [kworker/1:0H]
19 root SW [cpuhp/2]
20 root SW [migration/2]
21 root SW [ksoftirqd/2]
22 root IW [kworker/2:0-eve]
23 root IW< [kworker/2:0H]
24 root SW [cpuhp/3]
25 root SW [migration/3]
26 root SW [ksoftirqd/3]
27 root IW [kworker/3:0-eve]
28 root IW< [kworker/3:0H]
29 root SW [cpuhp/4]
30 root SW [migration/4]
31 root SW [ksoftirqd/4]
32 root IW [kworker/4:0-eve]
33 root IW< [kworker/4:0H-kb]
34 root SW [cpuhp/5]
35 root SW [migration/5]
36 root SW [ksoftirqd/5]
37 root IW [kworker/5:0-eve]
38 root IW< [kworker/5:0H]
39 root SW [cpuhp/6]
40 root SW [migration/6]
41 root SW [ksoftirqd/6]
42 root IW [kworker/6:0-eve]
43 root IW< [kworker/6:0H]
44 root SW [cpuhp/7]
45 root SW [migration/7]
46 root SW [ksoftirqd/7]
47 root IW [kworker/7:0-eve]
48 root IW< [kworker/7:0H]
49 root SW [kdevtmpfs]
50 root IW [kworker/u16:1-e]
52 root SW [rcu_tasks_kthre]
61 root IW [kworker/3:1-mm_]
68 root IW [kworker/5:1-eve]
76 root IW [kworker/6:1-eve]
77 root IW [kworker/7:1-mm_]
85 root IW [kworker/1:1-eve]
93 root IW [kworker/2:1-eve]
102 root IW [kworker/4:1-mm_]
202 root SW [khungtaskd]
203 root SW [oom_reaper]
204 root IW< [writeback]
206 root SW [kcompactd0]
208 root IW< [kblockd]
209 root IW< [devfreq_wq]
211 root IW [kworker/u16:2-e]
214 root IW [kworker/u16:3-e]
224 root IW [kworker/u16:4-e]
235 root IW [kworker/u16:5-e]
259 root IW [kworker/u16:6-e]
380 root SW [kswapd0]
509 root SW [scsi_eh_0]
510 root IW< [scsi_tmf_0]
511 root IW< [ufs_clk_gating_]
512 root IW< [ufs_clkscaling_]
551 root IW [kworker/u16:7-e]
635 root IW [kworker/u16:8-u]
656 root S /sbin/klogd -n
674 root S /bin/sh
675 root IW< [kworker/4:1H]
677 root R ps -o pid,user,stat,args
# dd if=/dev/sde of=/dev/null bs=1M & while true; do echo l >
/proc/sysrq-trigger; sleep 1; done
https://pastebin.ubuntu.com/p/Sx4hXRbHby/
What information can I provide to help debug this issue?
Regards.
