Hello everybody!
I've bought recently OrangePi PC2 with Allwinner H5 CPU to develop a small
data gathering machine but got stuck with some difficulties with Armbian.
Board is OrangePi PC2 v 1.1.
The problem is random crashes (kernel oops and kernel panics then
sometimes).
What I have done: made SD card with *Armbian "next" 5.65 distro* (kernel
*4.14.78-sunxi64*). No changes, no overlays etc.
What happens: the system crashes randomly. It happens never during
initial kernel boot but it starts happening after systemd load.
Sometimes (like 1-2 of 10) system cannot boot at all without crash. When
system boots okay I use a script to test:
while true; do
date
for I in `seq 1 4`; do
timeout $I cat /dev/zero > /dev/null &
cat /var/log/syslog | grep 123 > /dev/null &
done
wait
sleep 1
done
The script runs well for a while (like 5-15 mins) and then crash appears in
system console and dmesg. Sometimes system hangs immediately but sometimes
it works then. I check then the state of the system using ambianmonitor -m
(temp and cpu freq) and also top (f -> P to watch CPU id of the process)
When there's no direct oops crash in system log, some local crashes can
occur, i.e. it can be segfault of some tool (ssh, top, sudo etc).
What I have already tried: updated kernel to dev-version (*4.19*) but got
the same random crashes.
Downgraded to *4.10.0* and got the same things.
Tried to upgrade u-boot to *dev* version, tried to downgrade u-boot.
Uploaded u-boot to NOR flash and used USB for armbian (no SD card at all) -
random crashes appear.
"Official" distro (Debian 8, kernel 3.10) works with no issue. But it's too
old to use with 1-wire, I2C and other sensors.
Then I spent hours to check PC and LR values, also checked processes which
was involved to crash issue but nothing common.
I tried to downclock CPU (to 480 MHz min and max) - no luck. Checked
voltages (1.1 Vcore, 1.2Vdram) - everything is okay. Changed PSU - no
effect.
Changed the board to another OPI PC2 one - no effect, same crashes.
Then I tried to run the system on a single core by kernel parameter
*maxcpus=1*. And everything was FINE. No crashes for 24h.
Then I made CPU1 online (*echo 1 > /sys/bus/cpu/devices/cpu1/online*) and
got crash after like 5 minutes of running the script (non stop).
Then I tried to play with kernel parameter *isolcpus=1-3*. I saw then some
interrupts (cat /proc/interrupts) got served by all CPU cores, but all
software were running at CPU0 core. Then no crash for like 12h. Then I
started the same script on CPU2 using *taskset -c 4 ./test.sh*, saw it
running on CPU2 (using top) and then got a crash again in 15 minutes.
If I set *isolcpus=0,2-3*, all system services run on CPU1, and then if I
start script on CPU2 using *taskset -c 4 ./test.sh*, I see it running on
CPU2 (using top) and then a crash appears again in like 15 minutes.
Also I tried memtester on both boards I have - everything is fine.
Conclusion: it seems it's not hardware issue, something prevents apps work
normally when there's high I/O load when multiple cores are used.
Please help me to figure out what's happening. Thank you so much.
Some crash oops logs are attached.
--
You received this message because you are subscribed to the Google Groups
"linux-sunxi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to linux-sunxi+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[ OK ] Started LSB: Load kernel modules needed to enable cpufreq scaling.
Starting LSB: set CPUFreq kernel parameters...
[ 11.379399] Unable to handle kernel NULL pointer dereference at virtual
address 00000038
[ 11.379403] Unable to handle kernel NULL pointer dereference at virtual
address 00000038
[ 11.379406] Unable to handle kernel NULL pointer dereference at virtual
address 00000038
[ 11.379409] Unable to handle kernel NULL pointer dereference at virtual
address 00000038
[ 11.379417] Mem abort info:
[ 11.379418] Mem abort info:
[ 11.379419] Mem abort info:
[ 11.379422] Exception class = DABT (current EL), IL = 32 bits
[ 11.379424] Exception class = DABT (current EL), IL = 32 bits
[ 11.379426] Exception class = DABT (current EL), IL = 32 bits
[ 11.379428] SET = 0, FnV = 0
[ 11.379429] SET = 0, FnV = 0
[ 11.379431] SET = 0, FnV = 0
[ 11.379432] EA = 0, S1PTW = 0
[ 11.379433] EA = 0, S1PTW = 0
[ 11.379434] EA = 0, S1PTW = 0
[ 11.379435] Data abort info:
[ 11.379436] Data abort info:
[ 11.379437] Data abort info:
[ 11.379438] ISV = 0, ISS = 0x00000006
[ 11.379439] ISV = 0, ISS = 0x00000004
[ 11.379440] ISV = 0, ISS = 0x00000004
[ 11.379442] CM = 0, WnR = 0
[ 11.379443] CM = 0, WnR = 0
[ 11.379444] CM = 0, WnR = 0
[ 11.379448] user pgtable: 4k pages, 48-bit VAs, pgd = ffff80003508c000
[ 11.379451] user pgtable: 4k pages, 48-bit VAs, pgd = ffff8000351b8000
[ 11.379454] user pgtable: 4k pages, 48-bit VAs, pgd = ffff800034572000
[ 11.379456] [0000000000000038] *pgd=000000007568b003
[ 11.379458] [0000000000000038] *pgd=0000000000000000
[ 11.379460] [0000000000000038] *pgd=0000000000000000
[ 11.379461] , *pud=0000000075699003
[ 11.379464] , *pmd=0000000000000000
[ 11.379467] Internal error: Oops: 96000004 [#1] SMP
[ 11.379469] Modules linked in: sun8i_codec_analog sun4i_i2s ir_lirc_codec
lirc_dev snd_soc_hdmi_codec sun4i_codec snd_soc_simple_card
snd_soc_simple_card_utils snd_soc_core sunxi_cir sun4i_gpadc_iio
snd_pcm_dmaengine snd_pcm snd_timer iio_hwmon industrialio zram sch_fq_codel
dw_hdmi_cec dw_hdmi_i2s_audio sy8106a_regulator sun8i_dw_hdmi dw_hdmi cec
rc_core uas sunxi musb_hdrc sun4i_tcon sun8i_mixer sun4i_drm
[ 11.379524] CPU: 3 PID: 724 Comm: armbian-hardwar Not tainted
4.14.78-sunxi64 #416
[ 11.379526] Hardware name: Xunlong Orange Pi PC 2 (DT)
[ 11.379528] task: ffff80003539db00 task.stack: ffff00000a5d8000
[ 11.379541] PC is at cpu_load_update_active+0x4c/0xa8
[ 11.379546] LR is at cpu_load_update_active+0x9c/0xa8
[ 11.379549] pc : [<ffff0000080e3064>] lr : [<ffff0000080e30b4>] pstate:
800001c5
[ 11.379550] sp : ffff00000801bd90
[ 11.379551] x29: 0000000000000000 x28: ffff80003539db00
[ 11.379556] x27: ffff800037fb6dc0 x26: ffff800037fb6dc0
[ 11.379560] x25: ffff800037fb6dd0 x24: 00000002a600d616
[ 11.379563] x23: ffff80003539db00 x22: 0000000000000003
[ 11.379567] x21: 0000000000000404 x20: ffff800037fbb000
[ 11.379571] x19: ffff000008c68d88 x18: 0000ffffbc77fa70
[ 11.379575] x17: 0000ffffbc6ceb60 x16: ffff0000081ff200
[ 11.379579] x15: 0000000000000406 x14: 0000000000000000
[ 11.379582] x13: ffff000008998918 x12: 0000000000000001
[ 11.379586] x11: ffff000008998920 x10: 0000000000000406
[ 11.379590] x9 : 0000000000000004 x8 : ffff800037fbb030
[ 11.379594] x7 : 0000000000000020 x6 : 0000000000000404
[ 11.379597] x5 : 00000002a6438871 x4 : 000000000000000f
[ 11.379601] x3 : 000000001dcd6500 x2 : 0000000016349bec
[ 11.379604] x1 : 00000002900eec85 x0 : ffff800037fbb000
[ 11.379610] Process armbian-hardwar (pid: 724, stack limit =
0xffff00000a5d8000)
[ 11.379612] Call trace:
[ 11.379616] Exception stack(0xffff00000801bc50 to 0xffff00000801bd90)
[ 11.379620] bc40: ffff800037fbb000
00000002900eec85
[ 11.379624] bc60: 0000000016349bec 000000001dcd6500 000000000000000f
00000002a6438871
[ 11.379628] bc80: 0000000000000404 0000000000000020 ffff800037fbb030
0000000000000004
[ 11.379633] bca0: 0000000000000406 ffff000008998920 0000000000000001
ffff000008998918
[ 11.379637] bcc0: 0000000000000000 0000000000000406 ffff0000081ff200
0000ffffbc6ceb60
[ 11.379641] bce0: 0000ffffbc77fa70 ffff000008c68d88 ffff800037fbb000
0000000000000404
[ 11.379645] bd00: 0000000000000003 ffff80003539db00 00000002a600d616
ffff800037fb6dd0
[ 11.379649] bd20: ffff800037fb6dc0 ffff800037fb6dc0 ffff80003539db00
0000000000000000
[ 11.379653] bd40: ffff0000080e30b4 ffff00000801bd90 ffff0000080e3064
00000000800001c5
[ 11.379657] bd60: ffff800037fbb000 ffff80003539db80 0000ffffffffffff
ffff0000080e1a84
[ 11.379660] bd80: 0000000000000000 ffff0000080e3064
[ 11.379666] [<ffff0000080e3064>] cpu_load_update_active+0x4c/0xa8
[ 11.379672] Code: f9454000 eb020002 54000141 91362273 (f9401fa1)
[ 11.379676] ---[ end trace a5890dadb94a76a7 ]---
[ 11.379679] Kernel panic - not syncing: Fatal exception in interrupt
[ 11.379684] SMP: stopping secondary CPUs
[ 11.812830] Mem abort info:
[ 11.815620] Exception class = DABT (current EL), IL = 32 bits
[ 11.821531] SET = 0, FnV = 0
[ 11.824581] EA = 0, S1PTW = 0
[ 11.827716] Data abort info:
[ 11.830592] ISV = 0, ISS = 0x00000006
[ 11.834422] CM = 0, WnR = 0
[ 11.837386] user pgtable: 4k pages, 48-bit VAs, pgd = ffff8000350b9000
[ 11.843904] [0000000000000038] *pgd=0000000075795003, *pud=00000000757a4003,
*pmd=0000000000000000
f[ 12.422302] SMP: failed to stop secondary CPUs 0-3
[ 12.427089] Kernel Offset: disabled
[ 12.430574] CPU features: 0x0802004
[ 12.434058] Memory Limit: none
[ 12.437110] Rebooting in 10 seconds..
[ 22.441187] SMP: stopping secondary CPUs
[ 23.487718] SMP: failed to stop secondary CPUs 0-3 [ 3.319363] sun8i-emac 1c30000.ethernet: PHY regulator found
[ 3.325754] input: r_gpio_keys as /devices/platform/r_gpio_keys/input/input0
[ 3.333011] of_cfs_init
[ 3.335489] of_cfs_init: OK
[ 3.338387] gmac-3v3: disabling
[ 3.341539] ALSA device list:
[ 3.344505] No soundcards found.
[ 3.348283] Freeing unused kernel memory: 448K
Loading, please wait...
starting version 237
[ 3.417575] systemd-udevd[203]: unhandled level 0 translation fault (11) at
0x00000038, esr 0x92000004
[ 3.426960] pgd = ffff80003be82000
[ 3.430373] [00000038] *pgd=0000000000000000
[ 3.434643]
[ 3.436140] CPU: 1 PID: 203 Comm: systemd-udevd Not tainted 4.10.0-sun50iw2
#7
[ 3.443356] Hardware name: Xunlong Orange Pi PC 2 (DT)
[ 3.448490] task: ffff80003c19f080 task.stack: ffff80003bec4000
[ 3.454409] PC is at 0xaaaad4fce1c8
[ 3.457896] LR is at 0xaaaad4fce1b0
[ 3.461382] pc : [<0000aaaad4fce1c8>] lr : [<0000aaaad4fce1b0>] pstate:
60000000
[ 3.468768] sp : 0000ffffeba07510
[ 3.472080] x29: 0000000000000000 x28: 000000000000000e
[ 3.477389] x27: 0000aaaaeca4a140 x26: 00000000000002ea
[ 3.482697] x25: 0000aaaaeca6a180 x24: 0000000000000054
[ 3.488006] x23: 0000aaaaeca6a4a0 x22: 0000aaaaeca66b80
[ 3.493340] x21: 0000aaaaeca6b2d0 x20: 000000000000007c
[ 3.498655] x19: 0000aaaad5036000 x18: 0000ffff9d690a70
[ 3.503963] x17: 0000ffff9d60e5f8 x16: 0000aaaad5036a58
[ 3.509269] x15: 0000000000000000 x14: 0000000000000002
[ 3.514574] x13: 000000000000270f x12: 0000000000000001
[ 3.519880] x11: 0000000000000000 x10: 0000000000000001
[ 3.525186] x9 : 0000ffff9d6b9c90 x8 : 00000000000000d3
[ 3.530492] x7 : 7f7fffffff7f7fff x6 : fefffefe9c6809cf
[ 100.848748] Unable to handle kernel NULL pointer dereference at virtual
address 000000d0
[ 100.856945] pgd = ffff80003c641000
[ 100.860346] [000000d0] *pgd=0000000000000000
[ 100.864667] Internal error: Oops: 96000044 [#1] SMP
[ 100.869541] Modules linked in: fuse tun sun4i_codec sun8i_codec_analog
snd_soc_core snd_pcm_dmaengine snd_pcm snd_timer sch_fq_codel
[ 100.881456] CPU: 2 PID: 1439 Comm: test.sh Not tainted 4.10.0-sun50iw2 #7
[ 100.888232] Hardware name: Xunlong Orange Pi PC 2 (DT)
[ 100.893362] task: ffff80003ca0d780 task.stack: ffff80003c614000
[ 100.899281] PC is at copy_process.isra.5.part.6+0x150/0x1470
[ 100.904934] LR is at copy_process.isra.5.part.6+0x134/0x1470
[ 100.910584] pc : [<ffff0000080a6bd8>] lr : [<ffff0000080a6bbc>] pstate:
40000145
[ 100.917965] sp : ffff80003c617d50
[ 100.921273] x29: 0000000000000000 x28: 0000000001200011
[ 100.926579] x27: ffff80003c880000 x26: ffff80003c810c80
[ 100.931883] x25: 0000000000000000 x24: 0000000000000000
[ 100.937186] x23: 0000000000000000 x22: 0000000000000000
[ 100.942490] x21: 0000000000000000 x20: 0000ffff8d4cac80
[ 100.947793] x19: 0000000000000000 x18: 0000000000000000
[ 100.953098] x17: 0000ffff8d3f27c0 x16: ffff0000080a8370
[ 100.958402] x15: 0000000000000000 x14: 0000000000000000
[ 100.963705] x13: 0000000000000000 x12: 0000000000000000
[ 100.969009] x11: 0000000000000000 x10: 0000000000000000
[ 100.974312] x9 : 0000000000000000 x8 : 0000000000000000
[ 100.979614] x7 : 0000000000000000 x6 : ffff80003c811900
[ 100.984916] x5 : 0000000000000000 x4 : 0000000000000000
[ 100.990221] x3 : ffff000008af5739 x2 : 0000000000000001
[ 100.995524] x1 : ffff80003c810c80 x0 : 0000000000000001
[ 101.000826]
[ 101.002315] Process test.sh (pid: 1439, stack limit = 0xffff80003c614000)
[ 101.009093] Stack: (0xffff80003c617d50 to 0xffff80003c618000)
[ 101.014831] 7d40: ffff80003c617e40
ffff0000080a8058
[ 101.022648] 7d60: 0000000001200011 0000000000000000 ffffffffffffffff
0000000000000000
[ 101.030465] 7d80: 0000000040000000 0000000000000015 0000000000000123
00000000000000dc
[ 101.038281] 7da0: ffff000008822000 ffff80003ca0d780 ffff000008828178
000000009200004f
[ 101.046098] 7dc0: 0000aaaab2f5c908 ffff80003c617ec0 0000000000000000
0000000000000024
[ 101.053914] 7de0: 000000009200004f 0000aaaab2f5c908 0000aaaac955f8b0
ffff80003ca0d780
[ 101.061730] 7e00: 0000000001200011 0000000000000002 0000000000000000
ffff00000808236c
[ 101.069547] 7e20: 0000000000000000 ffff000008ba4000 ffffffffffffffff
0000aaaab2f39ac4
[ 101.077363] 7e40: ffff80003c617eb0 ffff0000080a8388 0000000000000000
00008000364c5000
[ 101.085180] 7e60: ffffffffffffffff 0000ffff8d3f28a4 0000000040000000
0000000000000015
[ 101.092998] 7e80: 0000000000000123 00000000000000dc ffff80003c617eb0
ffff0000081c75d0
[ 101.100817] 7ea0: 0000000000000000 00008000364c5000 0000000000000000
ffff000008082730
[ 101.108634] 7ec0: 0000000001200011 0000000000000000 0000000000000000
0000000000000000
[ 101.116451] 7ee0: 0000ffff8d4cac80 0000ffff8d4cabb0 0000aaaac9561120
000000000003e9a3
[ 101.124269] 7f00: 00000000000000dc 0000aaaab2f5c000 0101010101010101
0000000000000000
[ 101.132085] 7f20: 0101010101010101 000000000000270f 0000000000000001
0000000000000020
[ 101.139901] 7f40: 0000aaaab2f5baf0 0000ffff8d3f27c0 0000ffff8d4a3a70
0000000000000000
[ 101.147716] 7f60: 0000000000000000 0000ffff8d4a2000 0000aaaac955f8b0
0000ffff8d4caaa0
[ 101.155532] 7f80: 0000ffff8d4cb2a0 0000000000000000 0000aaaab2f5c0d8
0000aaaac955f8b0
[ 101.163348] 7fa0: 0000ffffc0c044a0 0000ffffc0c04390 0000aaaab2f3f480
0000ffffc0c04390
[ 101.171165] 7fc0: 0000ffff8d3f28a4 0000000040000000 0000000001200011
00000000000000dc
[ 101.178981] 7fe0: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 101.186796] Call trace:
[ 101.189239] Exception stack(0xffff80003c617b80 to 0xffff80003c617cb0)
[ 101.195672] 7b80: 0000000000000000 0001000000000000 ffff80003c617d50
ffff0000080a6bd8
[ 101.203488] 7ba0: ffff80003c617c20 ffff000008184af8 ffff7e00017000c0
ffff7e000074fb80
[ 101.211305] 7bc0: ffff80003c617d08 0000000000000180 000000003ca07840
ffff000008b9b600
[ 101.219122] 7be0: 0000000000000000 0000000000000140 ffff800000000001
ffff000008184b6c
[ 101.226937] 7c00: ffff80003c617d50 ffff0000080a6ba4 0000000000000000
0000ffff8d4cac80
[ 101.234754] 7c20: 0000000000000001 ffff80003c810c80 0000000000000001
ffff000008af5739
[ 101.242569] 7c40: 0000000000000000 0000000000000000 ffff80003c811900
0000000000000000
[ 101.250385] 7c60: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 101.258201] 7c80: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 101.266016] 7ca0: ffff0000080a8370 0000ffff8d3f27c0
[ 101.270890] [<ffff0000080a6bd8>] copy_process.isra.5.part.6+0x150/0x1470
[ 101.277583] Code: 350019a0 2a0203e0 aa1a03e1 f903435f (b900d3a2)
[ 101.283806] ---[ end trace 2625318557f33a56 ]---
[ OK ] Started Serial Getty on ttyS0.
[ OK ] Started Set console scheme.
[ OK ] Created slice system-getty.slice.
[ OK ] Started Getty on tty1.
[ OK ] Reached target Login Prompts.
[ OK ] Started OpenBSD Secure Shell server.
[ OK ] Reached target Multi-User System.
[ OK ] Reached target Graphical Interface.
Starting Update UTMP about System Runlevel Changes...
[ OK ] Started Update UTMP about System Runlevel Changes.
Ubuntu 18.04.1 LTS orangepipc2 ttyS0
orangepipc2 login: [ 38.155840] Unable to handle kernel NULL pointer
dereference at virtual address 00000077
[ 38.163950] pgd = ffff000008bef000
[ 38.167350] [00000077] *pgd=000000007effe003, *pud=000000007effd003,
*pmd=0000000000000000
[ 38.175639] Internal error: Oops: 96000044 [#1] SMP
[ 38.180518] Modules linked in: tun sun8i_codec_analog sun4i_codec
snd_soc_core snd_pcm_dmaengine snd_pcm snd_timer sch_fq_codel
[ 38.192001] CPU: 1 PID: 7 Comm: rcu_sched Not tainted 4.10.0-sun50iw2 #7
[ 38.198692] Hardware name: Xunlong Orange Pi PC 2 (DT)
[ 38.203824] task: ffff80003d864b00 task.stack: ffff80003d898000
[ 38.209741] PC is at rcu_gp_kthread+0x3c8/0x7c8
[ 38.214263] LR is at rcu_gp_kthread+0x394/0x7c8
[ 38.218792] pc : [<ffff0000080ff3c8>] lr : [<ffff0000080ff394>] pstate:
20000145
[ 38.226175] sp : ffff80003d89bdd0
[ 38.229483] x29: 0000000000000000 x28: 0000000000000000
[ 38.234786] x27: 0000000000000000 x26: ffff000008b32000
[ 38.240092] x25: 0000000000000003 x24: ffff80003d89be50
[ 38.245407] x23: 0000000000000001 x22: 0000000000000004
[ 38.250713] x21: ffff000008b16000 x20: ffff000008b32b20
[ 38.256015] x19: ffff000008b32800 x18: ffff000008ba7a00
[ 38.261330] x17: 0000ffff9d099010 x16: ffff0000080ad170
[ 38.266639] x15: fffffffffffffc68 x14: 000000000000024d
[ 38.271947] x13: ffff000008a3c000 x12: dead000000000100
[ 38.277252] x11: ffff7e0000000000 x10: 0000000000000860
[ 38.282564] x9 : ffff80003d89bcb0 x8 : ffff80003d8653c0
[ 38.287871] x7 : ffff80003ef99c80 x6 : 00000008e2435045
[ 38.293174] x5 : 0000000000000000 x4 : 0000000000000000
[ 38.298476] x3 : 0000000000000078 x2 : 0000000000000077
[ 38.303778] x1 : 00000000ffff004b x0 : 00000000000005f2
[ 38.309088]
[ 38.310581] Process rcu_sched (pid: 7, stack limit = 0xffff80003d898000)
[ 38.317269] Stack: (0xffff80003d89bdd0 to 0xffff80003d89c000)
[ 38.323015] bdc0: ffff80003d89be60
ffff0000080c5a28
[ 38.330847] bde0: ffff80003d831300 ffff80003d831200 ffff000008ba5860
ffff80003d864b00
[ 38.338669] be00: ffff0000089ca578 ffff000008b32800 ffff0000080ff000
ffff80003d831338
[ 38.346484] be20: ffff80003d857ce0 0000000000000000 ffff000008b37d0d
ffff80003d864b00
[ 38.354301] be40: 00ff80003d831300 ffff80003d864b00 ffff80003d89be50
ffff80003d89be50
[ 38.362118] be60: 0000000000000000 ffff0000080826c0 ffff0000080c5938
ffff80003d831200
[ 38.369934] be80: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 38.377751] bea0: 0000000000000000 0000000000000000 0000000000000000
ffffffffffffffff
[ 38.385566] bec0: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 38.393381] bee0: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 38.401196] bf00: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 38.409011] bf20: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 38.416826] bf40: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 38.424641] bf60: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 38.432457] bf80: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 38.440272] bfa0: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 38.448087] bfc0: 0000000000000000 0000000000000005 0000000000000000
0000000000000000
[ 38.455901] bfe0: 0000000000000000 0000000000000000 ffffffffffffffff
ffffffffffffffff
[ 38.463715] Call trace:
[ 38.466156] Exception stack(0xffff80003d89bc00 to 0xffff80003d89bd30)
[ 38.472585] bc00: ffff000008b32800 0001000000000000 ffff80003d89bdd0
ffff0000080ff3c8
[ 38.480400] bc20: ffff80003d89bc30 ffff0000080df768 ffff80003d89bcd0
ffff000008811d7c
[ 38.488214] bc40: ffff80003ef9f900 ffff000008b18000 ffff000008af1000
ffff80003d864b00
[ 38.496031] bc60: ffff80003d89bc90 ffff0000080db368 ffff80003d864b80
ffff000008b18000
[ 38.503845] bc80: 0000000000000001 0000000000000000 ffff80003d89bcd0
ffff000008811e30
[ 38.511660] bca0: 00000000000005f2 00000000ffff004b 0000000000000077
0000000000000078
[ 38.519475] bcc0: 0000000000000000 0000000000000000 00000008e2435045
ffff80003ef99c80
[ 38.527291] bce0: ffff80003d8653c0 ffff80003d89bcb0 0000000000000860
ffff7e0000000000
[ 38.535106] bd00: dead000000000100 ffff000008a3c000 000000000000024d
fffffffffffffc68
[ 38.542920] bd20: ffff0000080ad170 0000ffff9d099010
[ 38.547795] [<ffff0000080ff3c8>] rcu_gp_kthread+0x3c8/0x7c8
[ 38.553359] Code: 9101e3a3 f94542a1 9101dfa2 91000400 (3901dfbf)
[ 38.559577] ---[ end trace f49e754a16b6761c ]---
[ 43.200819] fuse init (API version 7.26)
[ 59.156035] INFO: rcu_sched detected stalls on CPUs/tasks:
[ 59.161537] (detected by 2, t=5253 jiffies, g=1185, c=1184, q=6363)
[ 59.167896] All QSes seen, last rcu_sched kthread activity 5254
(4294907087-4294901833), jiffies_till_next_fqs=1, root ->qsmask 0x0
[ 59.179697] cat R running task 0 1638 1637 0x00000000
[ 59.186736] Call trace:
[ 59.189189] [<ffff000008087a20>] dump_backtrace+0x0/0x220
[ 59.194579] [<ffff000008087c54>] show_stack+0x14/0x20
[ 59.199624] [<ffff0000080d0c84>] sched_show_task+0x10c/0x170
[ 59.205279] [<ffff0000081002e4>] rcu_check_callbacks+0x854/0x858
[ 59.211277] [<ffff000008102c74>] update_process_times+0x2c/0x58
[ 59.217189] [<ffff00000811205c>] tick_sched_handle.isra.7+0x24/0x70
[ 59.223444] [<ffff0000081120ec>] tick_sched_timer+0x44/0x90
[ 59.229029] [<ffff0000081037a8>] __hrtimer_run_queues+0xf0/0x178
[ 59.235034] [<ffff000008103ae8>] hrtimer_interrupt+0xa0/0x220
[ 59.240774] [<ffff00000869b970>] arch_timer_handler_phys+0x30/0x40
[ 59.246946] [<ffff0000080f61b8>] handle_percpu_devid_irq+0x78/0x130
[ 59.253204] [<ffff0000080f0eac>] generic_handle_irq+0x24/0x38
[ 59.258943] [<ffff0000080f152c>] __handle_domain_irq+0x5c/0xb8
[ 59.264768] [<ffff000008080d24>] gic_handle_irq+0x54/0xa8
[ 59.270157] Exception stack(0xffff80003c423be0 to 0xffff80003c423d10)
[ 59.276586] 3be0: 0000ffffa7556d40 00000000000002c0 0000000000001000
0000000000000001
[ 59.284418] 3c00: 0000000000020000 0001000000000000 0000000000000020
0000ffffa753d000
[ 59.292240] 3c20: 000000000000003f 0000ffffa78742a0 0000000000000840
0000000000000000
[ 59.300057] 3c40: 0000000000000001 000000000000270f 0000000000000002
0000000000000000
[ 59.307874] 3c60: ffff0000081cb7b0 0000ffffa77bba78 0000ffffa784ca70
0000000000001000
[ 59.315689] 3c80: 0000000000019000 ffff80003c423de0 ffff80003c8a0000
ffff80003c423eb8
[ 59.323503] 3ca0: 0000000000000015 0000000000000123 000000000000003f
ffff80003c423e18
[ 59.331320] 3cc0: 0000000000001000 ffff80003c423d10 ffff0000084c7fa0
ffff80003c423d10
[ 59.339136] 3ce0: ffff0000084b2388 0000000020000145 ffff80003c0f5780
ffff80003c8a0000
[ 59.346951] 3d00: 0001000000000000 ffff000008812500
[ 59.351825] [<ffff000008081ff4>] el1_irq+0xb4/0x140
[ 59.356700] [<ffff0000084b2388>] __clear_user+0x18/0x90
[ 59.361919] [<ffff00000856882c>] read_iter_zero+0x4c/0xc0
[ 59.367312] [<ffff0000081c9064>] __vfs_read+0xac/0xf8
[ 59.372354] [<ffff0000081ca2fc>] vfs_read+0x84/0x140
[ 59.377310] [<ffff0000081cb7f4>] SyS_read+0x44/0xa0
[ 59.382179] [<ffff000008082730>] el0_svc_naked+0x24/0x28
[ 59.387484] rcu_sched kthread starved for 5309 jiffies! g1185 c1184 f0x2
RCU_GP_DOING_FQS(4) ->state=0x40
