Excuse the top post: You seem to think that the mount of /dev/drbd0 on /home on alpha has something to do with the fdisk output for /dev/sda6. It doesn't. You probably have done some of the necessary steps out of order.
Yes, drbd appears OK, but the size of /dev/drbd0 on both servers will be identical. You don't display its size on either server. After drbd is running and the resources up to date (they are), then you make the fs on the drbd device and mount it. Since you think the size of /home is 547286323, maybe your system does too (IE, /home was mounted before you made the file system on /dev/drbd0). What is in your drbd conf? Is the drbd resource on /dev/sda or another disk? The very first error is write beyond the end of the device, so something was done out of order. Dan in Atlanta -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Keith Christian Sent: Thursday, August 05, 2010 12:30 PM To: [email protected] Subject: [DRBD-user] Buffer I/O and lost page write errors On an otherwise apparently healthy DRBD server pair, (named alpha_server and beta_server in the captures below) I see the following errors in /var/log/messages when data is rsync'ed from a different machine to this drbd pair. The first indication of a problem is that the rsync process on the machine sending to this drbd pair reports a Read-only file system. I don't know if "read only" is a misnomer from rsync's perspective (the mount command shows /home mounted as rw (Line 68 below.) rsync: failed to set times on "/home/user/userdata1000": Read-only file system (30) The machine sending the data has around 200gb in use in /home and the targets have more than twice that in space available, so it it's not a space problem. The DRBD partitions are of the same size but the /home partition is slightly different in size on alpha and beta, alpha's is 547286323 blocks (Line 53) and beta's is 32 blocks larger, 547286355 blocks (Line 82.) As I understand it, this shouldn't be a problem. Any suggestions for further steps to diagnose/fix appreciated. 1 Aug 4 00:07:22 alpha_server kernel: drbd0: rw=1, want=1094572648, limit=1094572647 2 Aug 4 00:07:22 alpha_server kernel: Buffer I/O error on device drbd0, logical block 136821580 3 Aug 4 00:07:22 alpha_server kernel: lost page write due to I/O error on drbd0 4 Aug 4 00:07:22 alpha_server kernel: drbd0: rw=1, want=1094572656, limit=1094572647 5 Aug 4 00:07:22 alpha_server kernel: Buffer I/O error on device drbd0, logical block 136821581 6 Aug 4 00:07:22 alpha_server kernel: lost page write due to I/O error on drbd0 7 Aug 4 00:07:22 alpha_server kernel: drbd0: rw=1, want=1094572664, limit=1094572647 8 Aug 4 00:07:22 alpha_server kernel: Buffer I/O error on device drbd0, logical block 136821582 9 Aug 4 00:07:22 alpha_server kernel: lost page write due to I/O error on drbd0 10 Aug 4 00:07:23 alpha_server kernel: drbd0: rw=1, want=1094572672, limit=1094572647 11 Aug 4 00:07:23 alpha_server kernel: Buffer I/O error on device drbd0, logical block 136821583 12 Aug 4 00:07:23 alpha_server kernel: lost page write due to I/O error on drbd0 13 Aug 4 00:07:23 alpha_server kernel: drbd0: rw=1, want=1094572680, limit=1094572647 14 Aug 4 00:07:23 alpha_server kernel: Buffer I/O error on device drbd0, logical block 136821584 15 Aug 4 00:07:23 alpha_server kernel: lost page write due to I/O error on drbd0 16 Aug 4 00:07:23 alpha_server kernel: drbd0: rw=1, want=1094572688, limit=1094572647 17 Aug 4 00:07:23 alpha_server kernel: Buffer I/O error on device drbd0, logical block 136821585 18 Aug 4 00:07:23 alpha_server kernel: lost page write due to I/O error on drbd0 19 Aug 4 00:07:23 alpha_server kernel: drbd0: rw=1, want=1094572696, limit=1094572647 20 Aug 4 00:07:23 alpha_server kernel: Buffer I/O error on device drbd0, logical block 136821586 21 Aug 4 00:07:23 alpha_server kernel: lost page write due to I/O error on drbd0 22 Aug 4 00:07:23 alpha_server kernel: drbd0: rw=1, want=1094572704, limit=1094572647 23 Aug 4 00:07:23 alpha_server kernel: Buffer I/O error on device drbd0, logical block 136821587 24 Aug 4 00:07:23 alpha_server kernel: lost page write due to I/O error on drbd0 25 Aug 4 00:07:23 alpha_server kernel: Aborting journal on device drbd0. 26 Aug 4 00:07:23 alpha_server kernel: EXT3-fs error (device drbd0): ext3_journal_start_sb: Detected aborted journal 27 28 29 30 31 cat /proc/drbd (alpha_server) 32 version: 8.3.2 (api:88/proto:86-90) 33 GIT-hash: dd7985327f146f33b86d4bff5ca8c94234ce840e build by [email protected], 2009-08-29 14:02:24 34 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---- 35 ns:613059302 nr:8307628 dw:78246580 dr:630584548 al:64211 bm:46024 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 36 37 38 39 40 41 fdisk -l (alpha_server) 42 43 Disk /dev/sda: 584.6 GB, 584652423168 bytes 44 255 heads, 63 sectors/track, 71079 cylinders 45 Units = cylinders of 16065 * 512 = 8225280 bytes 46 47 Device Boot Start End Blocks Id System 48 /dev/sda1 * 1 13 104391 83 Linux 49 /dev/sda2 14 2563 20482875 83 Linux 50 /dev/sda3 2564 2818 2048287+ 82 Linux swap / Solaris 51 /dev/sda4 2819 71079 548306482+ 5 Extended 52 /dev/sda5 2819 2945 1020096 83 Linux 53 /dev/sda6 2946 71079 547286323+ 83 Linux 54 55 56 57 58 59 mount (alpha_server) 60 /dev/sda2 on / type ext3 (rw) 61 proc on /proc type proc (rw) 62 sysfs on /sys type sysfs (rw) 63 devpts on /dev/pts type devpts (rw,gid=5,mode=620) 64 /dev/sda1 on /boot type ext3 (rw) 65 tmpfs on /dev/shm type tmpfs (rw) 66 none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) 67 sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) 68 /dev/drbd0 on /home type ext3 (rw) 69 70 71 72 73 74 fdisk -l (beta_server) 75 76 Disk /dev/sda: 1348.4 GB, 1348485513216 bytes 77 255 heads, 63 sectors/track, 163944 cylinders 78 Units = cylinders of 16065 * 512 = 8225280 bytes 79 80 Device Boot Start End Blocks Id System 81 /dev/sda1 * 1 13 104391 83 Linux 82 /dev/sda2 14 68147 547286355 83 Linux 83 /dev/sda3 68148 70697 20482875 83 Linux 84 /dev/sda4 70698 163944 749006527+ 5 Extended 85 /dev/sda5 70698 70952 2048256 82 Linux swap / Solaris 86 /dev/sda6 70953 71079 1020096 83 Linux 87 /dev/sda7 71080 163944 745938081 83 Linux 88 89 90 91 92 93 mount (beta_server) 94 /dev/sda3 on / type ext3 (rw) 95 proc on /proc type proc (rw) 96 sysfs on /sys type sysfs (rw) 97 devpts on /dev/pts type devpts (rw,gid=5,mode=620) 98 /dev/sda7 on /spare type ext3 (rw) 99 /dev/sda1 on /boot type ext3 (rw) 100 tmpfs on /dev/shm type tmpfs (rw) 101 none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) 102 103 104 105 106 107 cat /proc/drbd (beta_server) 108 version: 8.3.2 (api:88/proto:86-90) 109 GIT-hash: dd7985327f146f33b86d4bff5ca8c94234ce840e build by [email protected], 2009-08-29 14:02:24 110 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r---- 111 ns:0 nr:547286323 dw:547286323 dr:0 al:0 bm:33404 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 112 =====END OF LISTING===== _______________________________________________ drbd-user mailing list [email protected] http://lists.linbit.com/mailman/listinfo/drbd-user _______________________________________________ drbd-user mailing list [email protected] http://lists.linbit.com/mailman/listinfo/drbd-user
