On 06/20/2012 10:59 AM, Phil Frost wrote:
On 06/20/2012 10:43 AM, Shaun Thomas wrote:
"Unfortunately device mapper (LVM) might not support barriers."
Further, the fsync() latency I observed running ext4 on MD and LVM
(but not DRBD devices) empirically suggest that barriers are working
for DRBD's underlying block device, at least given my understanding
that working barriers are required for a fsync() that really does
flush to non-volatile storage.
Just to check my sanity further, I just did a test of a DRBD device
directly on a partition of a SATA drive. No LVM, no MD; just a plain
SATA drive and DRBD. I reached the same conclusion, DRBD is not
observing fsync(), O_SYNC, etc., and it's not for lack of support on the
underlying device. Can anyone reproduce?
pfrost@storage02:/mnt/synctest$ cat /etc/drbd.d/test.res
resource test {
device minor 1;
net {
shared-secret
BTawM41lfw8L0RTKBXhOiGK4lWb6dZTqJGNaGJJwF0pCfNhasdfhB5qjBjgZL4O;
}
on storage01 {
disk /dev/sda2;
meta-disk internal;
address 10.0.0.7:7790;
}
on storage02 {
disk /dev/sda2;
meta-disk internal;
address 10.0.0.9:7790;
}
}
pfrost@storage02:~$ sudo drbdadm -- --overwrite-data-of-peer primary test
pfrost@storage02:~$ sudo mkfs -t ext4 /dev/drbd/by-res/test
pfrost@storage02:~$ sudo mount /dev/drbd/by-res/test -o barrier=1 -t
ext4 /mnt/synctest/
pfrost@storage02:~$ cd /mnt/synctest/
pfrost@storage02:/mnt/synctest$ sudo ~/test_fsync -f ./test_fsync
Simple write timing:
write 0.002742
Compare fsync times on write() and non-write() descriptor:
If the times are similar, fsync() can sync data written
on a different descriptor.
write, fsync, close 0.196210
write, close, fsync 0.175143
Compare one o_sync write to two:
one 16k o_sync write 0.175284
two 8k o_sync writes 0.300499
Compare file sync methods with one 8k write:
(o_dsync unavailable)
open o_sync, write 0.157953
write, fdatasync 0.167023
write, fsync 0.169357
Compare file sync methods with two 8k writes:
(o_dsync unavailable)
open o_sync, write 0.300676
write, fdatasync 0.182892
write, fsync 0.184106
pfrost@storage02:/mnt/synctest$ cd ..
pfrost@storage02:/mnt$ sudo umount synctest/
pfrost@storage02:/mnt$ sudo drbdadm down test
pfrost@storage02:/mnt$ sudo mkfs -t ext4 /dev/sda2
pfrost@storage02:/mnt$ cd /mnt/synctest/
pfrost@storage02:/mnt/synctest$ sudo ~/test_fsync -f ./test_fsync
Simple write timing:
write 0.002765
Compare fsync times on write() and non-write() descriptor:
If the times are similar, fsync() can sync data written
on a different descriptor.
write, fsync, close 8.561847
write, close, fsync 8.490528
Compare one o_sync write to two:
one 16k o_sync write 8.365526
two 8k o_sync writes 20.089010
Compare file sync methods with one 8k write:
(o_dsync unavailable)
open o_sync, write 8.390429
write, fdatasync 8.407142
write, fsync 8.540648
Compare file sync methods with two 8k writes:
(o_dsync unavailable)
open o_sync, write 16.739327
write, fdatasync 8.423901
write, fsync 8.515552
_______________________________________________
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user
