I've got a system with corrupted files. I can't figure out who the
culprit is, DRBD or reiserFS. Here's my configuration:
- Two nodes running Red Hat 6.1
- DRBD Version: 58. Running over eth1 (10/100 ethernet).
- ReiserFS version 3.5.24
- Heartbeat 0.4.9. Running over eth1 and eth2 (10/100 ethernet).
The two systems are called c1220 and c1220b.
c1220b was reset and heartbeat on c1220 became primary, DRBD ran fsck
and mounted the reiserFS. At least one of the files in this
filesystem was corrupt.
Between the point that heartbeat started running datadisk and fsck
was run, DRBD reported the network disconnect. Could this be a clue?
Another odd thing, before the failover when c1220 was coming up, DRBD
reported this:
Aug 27 08:22:15 c1220 kernel: eth1: media is 100Mb/s.
Aug 27 08:22:15 c1220 kernel: eth2: media is 100Mb/s.
Aug 27 08:22:15 c1220 kernel: drbd: module initialised. Version: 58
Aug 27 08:22:15 c1220 kernel: eth0: media is 100Mb/s.
Aug 27 08:22:15 c1220 kernel: drbd0: user provided size = 2048728 KB
Aug 27 08:22:15 c1220 kernel: drbd : vmallocing 64022 B for bitmap. @d086a01c
Aug 27 08:22:15 c1220 kernel: drbd0: Connection established.
Aug 27 08:22:15 c1220 kernel: drbd0: size=2048728 KB / blksize=4096 B
Aug 27 08:22:15 c1220 kernel: drbd0: magic?? m: 942551345 c: 11826 l: 11825
Aug 27 08:22:15 c1220 kernel: drbd0: Connection lost.(pc=0,uc=0)
Aug 27 08:22:15 c1220 kernel: drbd0: Connection established.
Aug 27 08:22:15 c1220 kernel: drbd0: size=2048728 KB / blksize=4096 B
Aug 27 08:22:15 c1220 kernel: drbd0: sock_sendmsg returned -32
Aug 27 08:22:15 c1220 kernel: drbd0: sock_recvmsg returned -32
Aug 27 08:22:15 c1220 kernel: drbd0: Connection lost.(pc=0,uc=0)
Aug 27 08:22:15 c1220 kernel: drbd0: Connection established.
Aug 27 08:22:15 c1220 kernel: drbd0: size=2048728 KB / blksize=4096 B
Aug 27 08:22:13 c1220 adc_version: PID=831
The connection lost/established/lost/established is not reflected in
the peer's log.
Below are the logs for c1220 when the corruption occured:
Aug 27 08:24:48 c1220 /etc/ha.d/resource.d/PreBasLcd: status - stopped
Aug 27 08:24:48 c1220 heartbeat: info: Acquiring resource group: c1220
PreBasLcd datadisk BasLDAP IPaddr::192.168.221.226/24 basctlr PostBasLcd
Aug 27 08:24:48 c1220 heartbeat: info: Running /etc/ha.d/resource.d/PreBasLcd
start
Aug 27 08:24:48 c1220 /etc/ha.d/resource.d/PreBasLcd: start
Aug 27 08:24:48 c1220 heartbeat: info: Running /etc/ha.d/resource.d/datadisk
start
Aug 27 08:24:50 c1220 drbd: succeeded
Aug 27 08:24:51 c1220 kernel: drbd0: Connection lost.(pc=0,uc=0)
Aug 27 08:24:51 c1220 kernel: drbd0: user provided size = 2048728 KB
Aug 27 08:24:51 c1220 drbd: succeeded
Aug 27 08:24:51 c1220 datadisk: The node can become primary without waiting
Aug 27 08:24:54 c1220 drbdc: succeeded
Aug 27 08:24:55 c1220 drbdc: succeeded
Aug 27 08:24:57 c1220 kernel: Checking ReiserFS transaction log (device
2b:00) ...
Aug 27 08:25:03 c1220 kernel: Replayed 71 transactions in 6 seconds
Aug 27 08:25:03 c1220 kernel: Using tea hash to sort names
Aug 27 08:25:03 c1220 kernel: ReiserFS core development sponsored by SuSE
Labs (suse.com)
Aug 27 08:25:03 c1220 kernel: Journaling sponsored by MP3.com
Aug 27 08:25:03 c1220 kernel: ReiserFS version 3.5.24
Aug 27 08:25:03 c1220 drbdc: succeeded
Anyone have any thoughts on this?
Thanks.
--
Tony Willoughby [EMAIL PROTECTED]
"When I die I would like to be born again as me."
- Hugh Hefner, speaking at Oxford.
