Dear All,
I did some soak testing of our application (running on linux) over the weekend.
Our application runs on a dedicated linux system.
This morning, I saw in "/var/log/message" the following message repeated 70
times "rcu_sched kthread starved for N jiffies!" (N being a number in the tens
of thousands). It also comes with a stack trace. Please find a copy/paste of
the latest error message below.
For information, we are using SCHED_FIFO for some threads that needs to run
with very low-latency (video card captures).
If anyone could shed some light on what that means, that would be greatly
appreciated!
Many thanks,
Fabrice Triboix
ftenc:~# grep kernel /var/log/messages | tail -30
Nov 30 09:13:33 plink-buildroot user.warn kernel: [332879.369148]
[<ffffffff81887e0f>] apic_timer_interrupt+0x6f/0x80
Nov 30 09:13:33 plink-buildroot user.warn kernel: [332879.369149] <EOI>
[<ffffffff81887b42>] ? retint_restore_args+0xe/0xe
Nov 30 09:13:33 plink-buildroot user.warn kernel: [332879.369154]
[<ffffffff8100cae5>] ? default_idle+0x25/0xb0
Nov 30 09:13:33 plink-buildroot user.warn kernel: [332879.369156]
[<ffffffff8100cae3>] ? default_idle+0x23/0xb0
Nov 30 09:13:33 plink-buildroot user.warn kernel: [332879.369158]
[<ffffffff8100d42f>] arch_cpu_idle+0xf/0x20
Nov 30 09:13:33 plink-buildroot user.warn kernel: [332879.369162]
[<ffffffff810f64a5>] cpu_startup_entry+0x2e5/0x400
Nov 30 09:13:33 plink-buildroot user.warn kernel: [332879.369164]
[<ffffffff81127103>] ? clockevents_register_device+0xe3/0x140
Nov 30 09:13:33 plink-buildroot user.warn kernel: [332879.369167]
[<ffffffff8112718c>] ? clockevents_config_and_register+0x2c/0x40
Nov 30 09:13:33 plink-buildroot user.warn kernel: [332879.369169]
[<ffffffff8103448a>] start_secondary+0x13a/0x150
Nov 30 09:13:33 plink-buildroot user.err kernel: [332879.369171] rcu_sched
kthread starved for 20999 jiffies!
Nov 30 09:32:14 plink-buildroot user.err kernel: [334000.974890] INFO:
rcu_sched detected stalls on CPUs/tasks: {} (detected by 5, t=21002 jiffies,
g=6612437, c=6612436, q=11267)
Nov 30 09:32:14 plink-buildroot user.err kernel: [334000.974900] All QSes
seen, last rcu_sched kthread activity 20999 (4628342351-4628321352),
jiffies_till_next_fqs=3
Nov 30 09:32:14 plink-buildroot user.info kernel: [334000.974903] mux-0
R running task 0 10129 8797 0x00000000
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974907]
ffffffff81e4c3c0 ffff88089fc83dd8 ffffffff810e53ef ffff88089fc943c0
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974909]
ffffffff81e4c3c0 ffff88089fc83e48 ffffffff811157cc ffff880837edbf00
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974911]
ffff88089fc943c0 0000000000002c03 000000000064e5d5 0000000000000000
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974914] Call
Trace:
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974916] <IRQ>
[<ffffffff810e53ef>] sched_show_task+0xaf/0x120
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974934]
[<ffffffff811157cc>] rcu_check_callbacks+0x70c/0x780
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974937]
[<ffffffff8111a5e9>] update_process_times+0x39/0x60
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974942]
[<ffffffff811296f6>] tick_sched_handle.isra.16+0x36/0x50
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974945]
[<ffffffff81129754>] tick_sched_timer+0x44/0x80
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974947]
[<ffffffff8111b2f0>] __run_hrtimer+0x80/0x1c0
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974950]
[<ffffffff81129710>] ? tick_sched_handle.isra.16+0x50/0x50
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974952]
[<ffffffff8111b693>] hrtimer_interrupt+0xf3/0x210
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974958]
[<ffffffff81035c99>] local_apic_timer_interrupt+0x39/0x60
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974964]
[<ffffffff81889a25>] smp_apic_timer_interrupt+0x45/0x60
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974967]
[<ffffffff81887e0f>] apic_timer_interrupt+0x6f/0x80
Nov 30 09:32:14 plink-buildroot user.warn kernel: [334000.974968] <EOI>
[<ffffffff81887abe>] ? retint_swapgs+0xe/0x44
Nov 30 09:32:14 plink-buildroot user.err kernel: [334000.974971] rcu_sched
kthread starved for 20999 jiffies!
ftenc:~#
ftenc:~#
ftenc:~# uname -a
Linux plink-buildroot 4.0.0 #2 SMP Thu Nov 26 10:34:48 GMT 2015 x86_64
GNU/Linux
Company registration number: 08121803
Place of registration: England
This email and any attachments are sent in strictest confidence for the sole
use of the addressee
and may contain legally privileged, confidential, and proprietary data. If you
are not the intended
recipient, please advise the sender by replying promptly to this email and then
delete and destroy
this email and any attachments without any further use, copying or forwarding.
Please consider your environmental responsibility before printing this e-mail
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
