Ok... it happened again overnight.
bladernr@klaatu:~$ uname -a
Linux klaatu 3.2.0-25-generic #40-Ubuntu SMP Wed May 23 20:30:51 UTC 2012
x86_64 x86_64 x86_64 GNU/Linux
Fully updated Precise system :( at 0854 this morning, the first SysRq
message appeared in the log:
Jun 20 08:54:04 klaatu kernel: [85700.545264] SysRq : HELP : loglevel(0-9)
reBoot Crash terminate-all-tasks(E) memory-full-oom-kill(F) kill-all-tasks(I)
thaw-filesystems(J) saK show-backtrace-all-active-cpus(L) show-memory-usage(M)
nice-all-RT-tasks(N) powerOff show-registers(P) show-all-timers(Q) unRaw Sync
show-task-states(T) Unmount show-blocked-tasks(W) dump-ftrace-buffer(Z)
Jun 20 08:54:04 klaatu kernel: [85700.548084] atkbd serio2: Unknown key pressed
(raw set 2, code 0x0 on isa0060/serio2).
Jun 20 08:54:04 klaatu kernel: [85700.548093] atkbd serio2: Use 'setkeycodes 00
<keycode>' to make it known.
Jun 20 08:54:04 klaatu kernel: [85700.645263] atkbd serio2: Unknown key pressed
(raw set 2, code 0x18 on isa0060/serio2).
Jun 20 08:54:04 klaatu kernel: [85700.645272] atkbd serio2: Use 'setkeycodes 18
<keycode>' to make it known.
Jun 20 08:54:04 klaatu kernel: [85700.647803] atkbd serio2: Unknown key pressed
(raw set 2, code 0x174 on isa0060/serio2).
Then there was this next SysRq message:
Jun 20 08:54:06 klaatu kernel: [85702.601971] atkbd serio2: Use 'setkeycodes 00
<keycode>' to make it known.
Jun 20 08:54:06 klaatu kernel: [85702.678295] atkbd serio2: Unknown key pressed
(raw set 2, code 0x8 on isa0060/serio2).
Jun 20 08:54:06 klaatu kernel: [85702.678303] atkbd serio2: Use 'setkeycodes 08
<keycode>' to make it known.
Jun 20 08:54:06 klaatu kernel: [85702.679470] SysRq : Changing Loglevel
Jun 20 08:54:06 klaatu kernel: [85702.679479] Loglevel set to 0
Jun 20 08:54:06 klaatu kernel: [85702.681515] atkbd serio2: Unknown key pressed
(raw set 2, code 0x0 on isa0060/serio2).
Jun 20 08:54:06 klaatu kernel: [85702.681522] atkbd serio2: Use 'setkeycodes 00
<keycode>' to make it known.
Jun 20 08:54:06 klaatu kernel: [85702.826266] atkbd serio2: Unknown key pressed
(raw set 2, code 0x8 on isa0060/serio2).
Jun 20 08:54:06 klaatu kernel: [85702.826274] atkbd serio2: Use 'setkeycodes 08
<keycode>' to make it known.
and more:
Jun 20 08:54:07 klaatu kernel: [85703.533880] atkbd serio2: Unknown key pressed
(raw set 2, code 0x8 on isa0060/serio2).
Jun 20 08:54:07 klaatu kernel: [85703.533889] atkbd serio2: Use 'setkeycodes 08
<keycode>' to make it known.
Jun 20 08:54:07 klaatu kernel: [85703.535377] SysRq : Show Blocked State
Jun 20 08:54:07 klaatu kernel: [85703.535385] task PC
stack pid father
Jun 20 08:54:07 klaatu kernel: [85703.535612] Sched Debug Version: v0.10,
3.2.0-25-generic #40-Ubuntu
Jun 20 08:54:07 klaatu kernel: [85703.535617] ktime
: 85788353.151918
Jun 20 08:54:07 klaatu kernel: [85703.535622] sched_clk
: 85703535.609556
Jun 20 08:54:07 klaatu kernel: [85703.535626] cpu_clk
: 85703535.609650
Jun 20 08:54:07 klaatu kernel: [85703.535630] jiffies
: 4316339384
Jun 20 08:54:07 klaatu kernel: [85703.535634] sched_clock_stable
: 1
Jun 20 08:54:07 klaatu kernel: [85703.535637]
And even more disturbingly, this:
Jun 20 08:54:12 klaatu kernel: [85708.455046] atkbd serio2: Use 'setkeycodes 00
<keycode>' to make it known.
Jun 20 08:54:12 klaatu kernel: [85708.474757] atkbd serio2: Unknown key pressed
(raw set 2, code 0x8 on isa0060/serio2).
Jun 20 08:54:12 klaatu kernel: [85708.474765] atkbd serio2: Use 'setkeycodes 08
<keycode>' to make it known.
Jun 20 08:54:12 klaatu kernel: [85708.476031] SysRq : Manual OOM execution
Jun 20 08:54:12 klaatu kernel: [85708.476557] kworker/0:1 invoked oom-killer:
gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0
Jun 20 08:54:12 klaatu kernel: [85708.476564] kworker/0:1 cpuset=/
mems_allowed=0
Jun 20 08:54:12 klaatu kernel: [85708.476571] Pid: 10320, comm: kworker/0:1
Tainted: P O 3.2.0-25-generic #40-Ubuntu
Jun 20 08:54:12 klaatu kernel: [85708.476576] Call Trace:
Jun 20 08:54:12 klaatu kernel: [85708.476589] [<ffffffff810bffad>] ?
cpuset_print_task_mems_allowed+0x9d/0xb0
Jun 20 08:54:12 klaatu kernel: [85708.476601] [<ffffffff8111ad91>]
dump_header+0x91/0xe0
Jun 20 08:54:12 klaatu kernel: [85708.476607] [<ffffffff8111b115>]
oom_kill_process+0x85/0xb0
Jun 20 08:54:12 klaatu kernel: [85708.476613] [<ffffffff8111b4ba>]
out_of_memory+0xfa/0x220
Jun 20 08:54:12 klaatu kernel: [85708.476623] [<ffffffff813c5ec0>] ?
sysrq_handle_term+0x20/0x20
Jun 20 08:54:12 klaatu kernel: [85708.476629] [<ffffffff813c5ee5>]
moom_callback+0x25/0x30
Jun 20 08:54:12 klaatu kernel: [85708.476638] [<ffffffff81084f9a>]
process_one_work+0x11a/0x480
Jun 20 08:54:12 klaatu kernel: [85708.476646] [<ffffffff81085d44>]
worker_thread+0x164/0x370
Jun 20 08:54:12 klaatu kernel: [85708.476654] [<ffffffff81085be0>] ?
manage_workers.isra.29+0x130/0x130
Jun 20 08:54:12 klaatu kernel: [85708.476661] [<ffffffff8108a59c>]
kthread+0x8c/0xa0
Jun 20 08:54:12 klaatu kernel: [85708.476671] [<ffffffff81667db4>]
kernel_thread_helper+0x4/0x10
Jun 20 08:54:12 klaatu kernel: [85708.476678] [<ffffffff8108a510>] ?
flush_kthread_worker+0xa0/0xa0
Jun 20 08:54:12 klaatu kernel: [85708.476685] [<ffffffff81667db0>] ?
gs_change+0x13/0x13
Jun 20 08:54:12 klaatu kernel: [85708.476689] Mem-Info:
Jun 20 08:54:12 klaatu kernel: [85708.476692] Node 0 DMA per-cpu:
Jun 20 08:54:12 klaatu kernel: [85708.476697] CPU 0: hi: 0, btch: 1
usd: 0
Jun 20 08:54:12 klaatu kernel: [85708.476702] CPU 1: hi: 0, btch: 1
usd: 0
Jun 20 08:54:12 klaatu kernel: [85708.476706] CPU 2: hi: 0, btch: 1
usd: 0
Jun 20 08:54:12 klaatu kernel: [85708.476710] CPU 3: hi: 0, btch: 1
usd: 0
Jun 20 08:54:12 klaatu kernel: [85708.476714] CPU 4: hi: 0, btch: 1
usd: 0
Jun 20 08:54:12 klaatu kernel: [85708.476718] CPU 5: hi: 0, btch: 1
usd: 0
then this:
Jun 20 08:54:13 klaatu kernel: [85709.272960] atkbd serio2: Use 'setkeycodes 00
<keycode>' to make it known.
Jun 20 08:54:13 klaatu kernel: [85709.285061] atkbd serio2: Unknown key pressed
(raw set 2, code 0x8 on isa0060/serio2).
Jun 20 08:54:13 klaatu kernel: [85709.285069] atkbd serio2: Use 'setkeycodes 08
<keycode>' to make it known.
Jun 20 08:54:13 klaatu kernel: [85709.286069] SysRq : Terminate All Tasks
Jun 20 08:54:13 klaatu kernel: [85709.286615] init: tty5 main process (1288)
killed by TERM signal
Jun 20 08:54:13 klaatu kernel: [85709.286675] init: tty5 main process ended,
respawning
Jun 20 08:54:13 klaatu kernel: [85709.287125] atkbd serio2: Unknown key pressed
(raw set 2, code 0x0 on isa0060/serio2).
Jun 20 08:54:13 klaatu kernel: [85709.287131] atkbd serio2: Use 'setkeycodes 00
<keycode>' to make it known.
Jun 20 08:54:13 klaatu kernel: [85709.335915] atkbd serio2: Unknown key pressed
(raw set 2, code 0x18 on isa0060/serio2).
Jun 20 08:54:13 klaatu kernel: [85709.335922] atkbd serio2: Use 'setkeycodes 18
<keycode>' to make it known.
Jun 20 08:54:13 klaatu kernel: [85709.336968] atkbd serio2: Unknown key pressed
(raw set 2, code 0x15c on isa0060/serio2).
Jun 20 08:54:13 klaatu kernel: [85709.336974] atkbd serio2: Use 'setkeycodes 5c
<keycode>' to make it known.
Jun 20 08:54:13 klaatu kernel: [85709.340922] atkbd serio2: Unknown key pressed
(raw set 2, code 0x0 on isa0060/serio2).
Jun 20 08:54:13 klaatu kernel: [85709.340929] atkbd serio2: Use 'setkeycodes 00
<keycode>' to make it known.
Jun 20 08:54:13 klaatu kernel: [85709.363329] atkbd serio2: Unknown key pressed
(raw set 2, code 0x8 on isa0060/serio2).
Jun 20 08:54:13 klaatu kernel: [85709.363336] atkbd serio2: Use 'setkeycodes 08
<keycode>' to make it known.
Jun 20 08:54:13 klaatu kernel: [85709.365812] atkbd serio2: Unknown key pressed
(raw set 2, code 0x0 on isa0060/serio2).
Leading to my system dying.
I think I finally know what the root issue here is.
A while back, I filed bug #984265 (Keyboard becomes all but unusable
after Precise runs for so long) because after a while, the atkbd driver
would start spitting out messages like the ones above, eventually
leading to random keypresses appearing in whatever window/terminal had
focus at the moment.
Looking at this new set of messages, I think that what is happening is
that atkbd is going nuts and during it's spouting of random keypresses,
it's hitting on SysRq key combinations causing the system to appear to
just crash and burn. Looking at the various SysRq messages that appear
amidst the atkbd flood, it looks like it does all this.
And to bring it all together, I finally caught THIS:
Jun 20 08:54:15 klaatu kernel: [85711.160291] atkbd serio2: Use 'setkeycodes 08
<keycode>' to make it known.
Jun 20 08:54:15 klaatu kernel: [85711.161727] SysRq : Emergency Thaw of all
frozen filesystems
Jun 20 08:54:15 klaatu kernel: [85711.161745] Emergency Thaw on sda6
Jun 20 08:54:15 klaatu kernel: [85711.161749] Emergency Thaw on sda6
Jun 20 08:54:15 klaatu kernel: [85711.161752] Emergency Thaw on sda6
Jun 20 08:54:15 klaatu kernel: [85711.161755] Emergency Thaw on sda6
so the Emergency Thaw messages seem to be triggered by the atkbd problem
stumbling across the SysRq combo to force an Emergency Thaw on all
filesystems. this then triggers a flood of millions of Emergency Thaw
messages, which flood syslog, fill RAM and eventually bring the system
down.
** Changed in: linux (Ubuntu)
Status: Expired => Confirmed
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/984273
Title:
Kernel went nuts, dumping thousands of Emergency Thaw messages,
malformed klog messages and stack traces into syslog
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/984273/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
