sorry, accidentally posted to wrong list, please ignore. (unless you know the answers :P )
On 17 September 2015 at 13:56, Lee Musgrave <l...@sclinternet.co.uk> wrote: > > Hi, > > i'm a little confused by this, and i don't want to do anything with these > systems until i've got some clarity. > > i'm using drbd 8.3.13 on ubuntu 12.04.3, ( i know i should upgrade, but > that isn't an option right now). > > the metadata partition is on /dev/sda2, the os is installed on /dev/sda1 > > > on node1 cat /proc/drbd shows: > > > 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- > ns:40 nr:0 dw:12 dr:1104 al:2 bm:3 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 > 1: cs:Connected ro:Primary/Secondary ds:Diskless/UpToDate C r----- > ns:2015462262 nr:516042356 dw:1353605609 dr:1812380379 al:67433064 > bm:3 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 > 2: cs:Connected ro:Primary/Secondary ds:Diskless/UpToDate C r----- > ns:536493384 nr:80 dw:502334308 dr:1646088 al:28554 bm:0 lo:0 pe:0 > ua:0 ap:0 ep:1 wo:f oos:0 > 3: cs:Connected ro:Primary/Secondary ds:Diskless/UpToDate C r----- > ns:201451152 nr:0 dw:131716884 dr:84892 al:211 bm:0 lo:0 pe:0 ua:0 > ap:0 ep:1 wo:f oos:0 > 4: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- > ns:3202744 nr:0 dw:3202744 dr:349700 al:57 bm:0 lo:0 pe:0 ua:0 ap:0 > ep:1 wo:f oos:0 > > > on node2 : > > 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----- > ns:0 nr:40 dw:40 dr:0 al:0 bm:3 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 > 1: cs:Connected ro:Secondary/Primary ds:UpToDate/Diskless C r----- > ns:516043328 nr:2015478830 dw:2015478830 dr:516043328 al:33272078 > bm:3 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:289286616 > 2: cs:Connected ro:Secondary/Primary ds:UpToDate/Diskless C r----- > ns:80 nr:536493908 dw:536493908 dr:80 al:536 bm:0 lo:0 pe:0 ua:0 ap:0 > ep:1 wo:f oos:1825092 > 3: cs:Connected ro:Secondary/Primary ds:UpToDate/Diskless C r----- > ns:0 nr:201451968 dw:201451968 dr:0 al:298 bm:0 lo:0 pe:0 ua:0 ap:0 > ep:1 wo:f oos:91228 > 4: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----- > ns:0 nr:3202744 dw:3202744 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 > wo:f oos:0 > > > 0 is on /dev/sdb1, 1 is on /dev/sdb2, 2 is on /dev/sdc1, 3 is on > /dev/sdc2, 4 is on /dev/sdb3 > > /dev/sdb and /dev/sdc are 2 separate raid 10 arrays, each of 4 disks. > > > i don't believe there is any problem with the raid arrays, or their > composite disks. on node 1, the filesystem is currently mounted readonly, > so i believe the problem is the os disk, which also has the metadata > partition on it. > > does this seem the most likely to you? > > what's confusing me, is it's the node 2 partitions showing as out-of-sync, > how? surely it's the diskless partitions that are oos? is it keeping > everything in memory? are changes actually getting written to the disks on > node2? > > as i said, i believe the problem to be access to the metadata, since all 5 > drbd partitions share the same metadata partition, what would be the > recommended recovery method, i don't even want to try remounting / as rw > until i've got a bit more information, right now things are still working, > although in a degraded state, and it's not live, but i want to treat it as > live so i know i can recover from the same situation when it is in > production, so downtime or data loss needs to be avoided if at all possible. > > > thanks > lee. >
