See below: >>> From: [email protected] [mailto:[email protected]] On Behalf Of FDS | Forensik Data Services Sent: Friday, September 06, 2013 9:06 AM To: [email protected] Subject: [DRBD-user] generation identifiers
Hi there, I am involved in the aftermath of a split brain event (drbd 8.3.8). log files have given us several entries for drbd_sync_handshakes: 1. Feb 28 07:10:04 san02 kernel: [ 1435.507176] block drbd1: self 384ACFA6DDE7F305:6EC706A11FB6AAEB:AD2868A6C65F20AA:D5016EA9D8E57175 bits:28397184 flags:0 2. Feb 28 07:10:04 san02 kernel: [ 1435.507472] block drbd0: self 8FFC8087A5A8ACDF:128A2EF7246101ED:00C719D52E8C2482:4EEC506B23021307 bits:225135 flags:0 3. Feb 28 07:11:36 san01 kernel: [4427691.562720] block drbd1: self 16DA13A883B368DA:6EC706A11FB6AAEA:AD2868A6C65F20AA:D5016EA9D8E57175 bits:3072 flags:0 4. Feb 28 07:11:36 san01 kernel: [4427691.562923] block drbd0: self 2DBF62FFC79BF9F4:128A2EF7246101EC:00C719D52E8C2483:4EEC506B23021307 bits:53722300 flags:0 5. Feb 28 07:17:46 san02 kernel: [ 1897.442652] block drbd0: self 8FFC8087A5A8ACDE:8A4BA5176FB14BAD:128A2EF7246101ED:00C719D52E8C2482 bits:21858 flags:0 6. Feb 28 07:17:46 san02 kernel: [ 1897.443239] block drbd1: self 384ACFA6DDE7F304:E9797096C25A015C:6EC706A11FB6AAEB:AD2868A6C65F20AA bits:24262845 flags:0 7. Feb 28 07:19:19 san01 kernel: [4428153.502129] block drbd0: self 8A4BA5176FB14BAC:0000000000000000:00C719D52E8C2483:4EEC506B23021307 bits:49588595 flags:0 8. Feb 28 07:19:19 san01 kernel: [4428153.502363] block drbd1: self E9797096C25A015C:0000000000000000:AD2868A6C65F20AA:D5016EA9D8E57175 bits:0 flags:0 The Cluster consists of 2 nodes (san01 und san02) with two block devices (drbd01 und drbd02) running in dual primary with OCFS2. Analyzing Generation Identifiers oft he first handshake: · Current UUIDs (1.-4.) differ due to split brainsituation. · BitMap UUIDs (1.-4.) are nearly identical. Only the last digit differs from A-D. First Historical UUIDs (1.-4.) with drbd1 are identical but differ for block drbd0. Again only the last digit (2 vs 3). · Second Historical UUID (1.-4.) for both block-Devices are identical meaning both devices have the same data set, or? Questions: · Reason and meaning for differing last digit UUIDs are unknown. No information in the doc. >>> I don't know · How can I see when the data sets were sync for the last time? >>> Careful analysis of the logs · Meaning of values for flags fe 0 or 2? >>> I don't know · Is it possible to decide which resource (drbd0 or drbd1) was primary by comparing UUIDs? >>> You had dual primary - they were both primary. You'll have to look at the >>> data itself. For example if one file on one of the standalone drbd0 exists >>> with a newer date, that file is likely more valid than the other file. >>> Depending on how and why you had dual primary without proper fencing in the >>> first place, it may be trivial to determine which to keep and which to >>> "discard-my-data". If they are dual primary solely to provide live >>> migration - AND - no live migrations were taking place at the time of the >>> split brain event, you are fairly safe using the resource (or disk image) >>> to which a VM was pointed at the time of the "event". If the file systems >>> were shared by multiple hosts, you are stuck with file by file analysis - >>> and there are no promises that the newest file will contain all the data. >>> You may need to do a row by row analysis depending on your applications. >>> Again, without knowing the sharing and application logic in detail about >>> your specific systems, split-brain analysis is impossible. >>> hth and good luck! >>> (Get fencing working!!!!!, Get a current drbd installed!!) >>> Dan Aprecciating your help. -- Mit freundlichen Grüßen Ing. Mag. Horst Greifeneder (fds) FDS | Forensik Data Services. Better secure than sorry! Schenkelbachweg 32. A - 4600 W e l s. tel. 07242. 777 15. fax. 07242. 777 16. mailto:[email protected]
_______________________________________________ drbd-user mailing list [email protected] http://lists.linbit.com/mailman/listinfo/drbd-user
