On 06/10/2010 11:39 AM, Andreas Kurz wrote:
On Thursday 10 June 2010 17:19:53 Michael Joyner wrote:
better, but still way below box2box bandwidth.... is fs type or fact
using lvm a factor? Here are my test results, will try w/o LVM next.
I can really recommend "Part V. Optimizing DRBD performance" in the DRBD
Users Guide ... or invest in a "DRBD Healthcheck" offered by Linbit ;-)
Regards,
Andreas
I split it up into 3 volumes, to prevent split brain issue with LVM. :)
I retest with EXT4, same horrible results.
I then tested with XFS, much, much better results:
Migrated VMs over to it. They all seem to be running much better with
XFS and not EXT4.
As soon as I can get back into the COLO, I will reroute cabling to setup
a RR bond between the two boxes directly to see if that helps.
Thanks for all the help.
=== *** === XFS Timing Test:
Large files test:
rsync box2box /var/tmp:
sent 6.96G bytes received 164 bytes 47.16M bytes/sec
rsync local2/data w/ peer:
sent 6.96G bytes received 164 bytes 41.53M bytes/sec
rsync local2/data no peer:
sent 6.96G bytes received 164 bytes 114.99M bytes/sec
Iozone test (no peer):
real 0m4.983s
real 0m4.922s
real 0m4.920s
real 0m4.915s
real 0m4.917s
Iozone test (w/ peer);
real 0m29.644s
real 0m28.056s
real 0m27.634s
real 0m28.203s
real 0m28.209s
On 06/10/2010 04:05 AM, Andreas Kurz wrote:
.... LVM on your system supports barriers --> drbd supports barriers -->
barriers are used to enforce write-after-write dependencies per default
... use drbd config-param 'no-disk-barrier'
=== new layout ==============================
using /dev/sdb as /dev/drbd0, non-spanned vg.
=== new drbd.conf ============================
global {
usage-count yes;
}
common {
protocol C;
syncer { rate 700000K; }
net {
sndbuf-size 0;
after-sb-0pri discard-least-changes;
after-sb-1pri discard-secondary;
}
startup {
degr-wfc-timeout 30000;
}
disk {
no-disk-flushes;
no-md-flushes;
no-disk-barrier;
}
}
resource sdb {
on san-node-1.ewc.edu {
device /dev/drbd0;
disk /dev/sdb;
address 192.168.75.201:7789;
meta-disk internal;
}
on san-node-2.ewc.edu {
device /dev/drbd0;
disk /dev/sdb;
address 192.168.75.202:7789;
meta-disk internal;
}
}
*********
=== *** mount /dev/vgdrbd0/nfs1 /data *** ===
=== iozone -a -e -g 4096 with peer down:
real 0m1.757s, real 0m1.738s, real 0m1.753s, real
0m1.762s, real 0m1.745s
=== with peer up:
real 0m6.739s, real 0m6.758s, real 0m6.619s, real 0m6.653s,
real 0m6.636s
=== big files test =============================
=== DRBD === rm -rfv /data/ISOS; rsync -a --verbose --human-readable
--progress ~mjoyner/ISOS /data/ISOS
building file list ...
8 files to consider
created directory /data/ISOS
ISOS/
ISOS/ISOS/
ISOS/ISOS/.lck-5a00b81000000000
84 100% 0.00kB/s 0:00:00 (xfer#1, to-check=5/8)
ISOS/ISOS/.lck-6500b81000000000
84 100% 82.03kB/s 0:00:00 (xfer#2, to-check=4/8)
ISOS/ISOS/.lck-7700b81000000000
84 100% 82.03kB/s 0:00:00 (xfer#3, to-check=3/8)
ISOS/ISOS/SW_DVD5_NTRL_SQL_Svr_2008_SP1_English_X15-51857.ISO
943.07M 100% 189.42MB/s 0:00:04 (xfer#4, to-check=2/8)
ISOS/ISOS/SW_DVD5_SQL_Svr_Enterprise_Edtn_2008_English_MLF_X14-89207.ISO
3.26G 100% 189.81MB/s 0:00:16 (xfer#5, to-check=1/8)
ISOS/ISOS/SW_DVD5_Windows_Svr_2008w_SP2_English__x64_DC_EE_SE_X15-41371.ISO
2.76G 100% 162.03MB/s 0:00:16 (xfer#6, to-check=0/8)
#1) sent 6.96G bytes received 164 bytes 190.60M bytes/sec
#2) sent 6.96G bytes received 164 bytes 244.10M bytes/sec
#3) sent 6.96G bytes received 164 bytes 244.10M bytes/sec
=== LOCAL FS === rm -rfv /var/tmp/ISOS; rsync -a --verbose
--human-readable --progress ~mjoyner/ISOS /var/tmp/ISOS
#1) sent 6.96G bytes received 164 bytes 167.63M bytes/sec
#2) sent 6.96G bytes received 164 bytes 185.51M bytes/sec
#3) sent 6.96G bytes received 164 bytes 220.85M bytes/sec
=== time umount /data
real 2m11.090s
user 0m0.000s
sys 1m49.566s
*********
=== *** mount -o sync,data=journal /dev/vgdrbd0/nfs1 /data *** ===
=== iozone test, no peer
real 0m9.483s real 0m9.583s real 0m9.583s real 0m9.559s
real 0m9.491s
=== izone test, w/ peer up.
real 0m41.558s real 0m41.386s real 0m41.456s real 0m41.448s
=== BIG FILES TEST
no peer) sent 6.96G bytes received 164 bytes 74.40M bytes/sec
w/ peer) sent 6.96G bytes received 164 bytes 29.79M bytes/sec
rsync/var/tmp2/var/tmp) sent 6.96G bytes received 164 bytes 47.49M
bytes/sec
=== time umount /data
real 0m1.078s
user 0m0.000s
sys 0m1.075s
_______________________________________________
drbd-user mailing list
[email protected]
http://lists.linbit.com/mailman/listinfo/drbd-user
--
Michael Joyner ᏩᏯ
System Administrator/Edward Waters College
1658 Kings Road, Jacksonville, FL 32209
904-470-8170 (V) / 904-470-8170 (F)
ᏩᏙ
========================================================================
HOW TO SUBMIT A WORK REQUEST TO INFORMATION TECHNOLOGY
------------------------------------------------------------------------
Via the web: http://otrs.ewc.edu/otrs/customer.pl
Via email: [email protected]
* It is imperative that you submit each problem as a new item.
* Please do NOT combine multiple problem reports in a single workorder.
* PLEASE DO NOT EMAIL SUPPORT STAFF REQUESTS DIRECTLY.
* PLEASE USE THE JOB TRACKING SYSTEM.
========================================================================
_______________________________________________
drbd-user mailing list
[email protected]
http://lists.linbit.com/mailman/listinfo/drbd-user
