Daniel Raj wrote: > Hi Jeff, > > > *messages :* > Dec 19 11:42:26 cluster kernel: LustreError: > 23330:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous > similar messages > Dec 19 11:42:26 cluster kernel: LustreError: Skipped 3 previous > similar messages > Dec 19 11:44:05 cluster kernel: LustreError: 137-5: UUID > 'cluster-ost8_UUID' is not available for connect (no target) > Dec 19 11:44:05 cluster kernel: LustreError: > 23292:0:(ldlm_lib.c:1892:target_send_reply_msg()) @@@ processing error > (-19) r...@ffff810283dee000 x1353488904620989/t0 o8-><?>@<?>:0/0 lens > 368/0 e 0 to 0 dl 1292739345 ref 1 fl Interpret:/0/0 rc -19/0 > Dec 19 11:44:05 cluster kernel: LustreError: > 23292:0:(ldlm_lib.c:1892:target_send_reply_msg()) Skipped 3 previous > similar messages > Dec 19 11:44:05 cluster kernel: LustreError: Skipped 3 previous > similar messages
Looks like here the system rebooted on its own. Kernel crashes are not always able to be sent to syslog, so you would need the serial console output. You might also want to check the IPMI SEL (system event log) to see if there were any hardware errors recorded for your node. > Dec 19 11:48:11 cluster syslogd 1.4.1: restart. > Dec 19 11:48:11 cluster kernel: klogd 1.4.1, log source = /proc/kmsg > started. > Dec 19 11:48:11 cluster kernel: Linux version > 2.6.18-164.11.1.el5_lustre.1.8.3 (lbu...@x86-build-1) (gcc version > 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Fri Apr 9 18:00:39 MDT 2010 > Dec 19 11:48:11 cluster kernel: Command line: ro root=LABEL=/ rhgb > quiet irqpoll maxcpus=1 reset_devices memmap=exactmap memmap=6...@0k > memmap=52...@16384k memmap=1251...@22300k elfcorehdr=147440K > memmap=32K#3144016K > Dec 19 11:48:11 cluster kernel: BIOS-provided physical RAM map: > Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000000010000 - > 000000000009ec00 (usable) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 000000000009ec00 - > 00000000000a0000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000000100000 - > 00000000bfe54000 (usable) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe54000 - > 00000000bfe5c000 (ACPI data) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe5c000 - > 00000000bfe5d000 (usable) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000bfe5d000 - > 00000000c0000000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000e0000000 - > 00000000f0000000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000fec00000 - > 00000000fed00000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000fee00000 - > 00000000fee10000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 00000000ffc00000 - > 0000000100000000 (reserved) > Dec 19 11:48:11 cluster kernel: BIOS-e820: 0000000100000000 - > 000000043ffff000 (usable) Here someone initiated a reboot: > Dec 19 11:51:25 cluster init: Switching to runlevel: 6 > Dec 19 11:51:27 cluster rpc.statd[5886]: Caught signal 15, > un-registering and exiting. > Dec 19 11:51:27 cluster multipathd: --------shut down------- > Dec 19 11:51:27 cluster auditd[5740]: The audit daemon is exiting. > Dec 19 11:51:27 cluster kernel: audit(1292739687.831:17): audit_pid=0 > old=5740 by auid=4294967295 > Dec 19 11:51:27 cluster kernel: Kernel logging (proc) stopped. > Dec 19 11:51:27 cluster kernel: Kernel log daemon terminating. > Dec 19 11:51:29 cluster exiting on signal 15 > Dec 19 11:55:48 cluster syslogd 1.4.1: restart. > Dec 19 11:55:48 cluster kernel: klogd 1.4.1, log source = /proc/kmsg > started. > Dec 19 11:55:48 cluster kernel: Linux version > 2.6.18-164.11.1.el5_lustre.1.8.3 (lbu...@x86-build-1) (gcc version > 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Fri Apr 9 18:00:39 MDT 2010 > Dec 19 11:55:48 cluster kernel: Command line: ro root=LABEL=/ rhgb > quiet crashkernel=1...@16m > Dec 19 11:55:48 cluster kernel: BIOS-provided physical RAM map: _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
