Hi all,
I have been doing some tests with two proxmox servers in preparation of
a deployment in a client, and I thought it could be useful to share them.
Please don't take this scenario as a good implementation of Proxmox nor
a good repeatable benchmark; it's just a setup influenced by other
(external) factors and not a totally new setup both on client and for
the tests (we're reusing existing hardware). Its objective was to make a
rough comparison of disk performance.
Test scenario:
- Server A: Dell T610, 1xIntel Xeon E5520, 32 GB RAM, H200 RAID
controller, 2x500GB SATA, 2x1TB SATA, 2xSSD Intel 320 300G
- Server B: IBM System x3400 7976, 1xIntel Xeon E5410, 12 GB RAM, IBM
ServeRaid 8k 256 MB BBC, 2x500GB SATA, 2x1TB SATA
- Network: back-to-back 1G ethernet connection between both servers mtu
9000, additional 1G ethernet for general network access (ssh,
webinterface, VMs) in each server. Onboard NIC on Server A (Broadcom),
intel pro/1000 on Server B.
Servers have working VMs running on them. There's low load and tests
have been performed more than once and only the best is reported;
there's plenty of RAM and CPU free on both servers. Disks used in the
tests are not being used neither by VMs nor the base Proxmox system.
Test results:
[SERVER B]
-- 450 GB partition on 2x500 HW RAID1, XFS
CPU BOGOMIPS: 18620.04
REGEX/SECOND: 913801
HD SIZE: 442.16 GB (/dev/sdb3)
BUFFERED READS: 147.15 MB/sec
AVERAGE SEEK TIME: 12.27 ms
FSYNCS/SECOND: 4386.37
DNS EXT: 95.34 ms
-- 450 GB partition on 2x500 HW RAID1, EXT4
CPU BOGOMIPS: 18620.04
REGEX/SECOND: 894205
HD SIZE: 435.44 GB (/dev/sdb3)
BUFFERED READS: 161.24 MB/sec
AVERAGE SEEK TIME: 11.82 ms
FSYNCS/SECOND: 1797.41
DNS EXT: 229.17 ms
[SERVER A]
-- 1xSSD EXT4 (non-raid on H200 port) noop scheduler
CPU BOGOMIPS: 36176.88
REGEX/SECOND: 761723
HD SIZE: 275.08 GB (/dev/sdc)
BUFFERED READS: 211.47 MB/sec
AVERAGE SEEK TIME: 0.23 ms
FSYNCS/SECOND: 1404.39
DNS EXT: 151.66 ms
-- 1xSSD XFS (non-raid on H200 port) noop scheduler
CPU BOGOMIPS: 36176.88
REGEX/SECOND: 758984
HD SIZE: 279.32 GB (/dev/sdc)
BUFFERED READS: 174.78 MB/sec
AVERAGE SEEK TIME: 0.60 ms
FSYNCS/SECOND: 2819.07
DNS EXT: 464.62 ms
(SATA drives on Server A for completeness, not used on final tests)
- partition on H200 RAID1 2x500GB SATA, XFS
CPU BOGOMIPS: 36176.88
REGEX/SECOND: 805542
HD SIZE: 22.36 GB (/dev/sda2)
BUFFERED READS: 111.19 MB/sec
AVERAGE SEEK TIME: 6.70 ms
FSYNCS/SECOND: 117.84
DNS EXT: 519.26 ms
- partition on H200 RAID1 2x500GB SATA, EXT4
Seems I lost this data, buffered reads where around 135 MB/s and fsync/s
was about 65.
FINAL TEST SETUP
- SERVER B is iSCSI target, exports 450GB partition on RAID 2x500GB as a lun
- SERVER A is iSCSI initiator, iSCSI uses back-to-back 1G ethernet mtu9000
[SERVER A]
-- 100GB LV on iSCSI disk, EXT4
CPU BOGOMIPS: 36176.88
REGEX/SECOND: 754021
HD SIZE: 98.43 GB (/dev/mapper/grupo_iscsi-tests)
BUFFERED READS: 102.46 MB/sec
AVERAGE SEEK TIME: 10.08 ms
FSYNCS/SECOND: 821.93
DNS EXT: 383.01 ms
-- 275GB MDRAID1 (SSD + iSCSI) + bitmap + --write-mostly, EXT4
CPU BOGOMIPS: 36176.88
REGEX/SECOND: 799245
HD SIZE: 275.08 GB (/dev/md0)
BUFFERED READS: 206.95 MB/sec
AVERAGE SEEK TIME: 0.23 ms
FSYNCS/SECOND: 763.87
DNS EXT: 119.05 ms
-- 275GB MDRAID1 (SSD + iSCSI) + bitmap + --write-mostly +
--write-behind, EXT4
CPU BOGOMIPS: 36176.88
REGEX/SECOND: 780966
HD SIZE: 275.08 GB (/dev/md0)
BUFFERED READS: 207.35 MB/sec
AVERAGE SEEK TIME: 0.24 ms
FSYNCS/SECOND: 1147.99
DNS EXT: 145.67 ms
-- 275GB MDRAID1 (SSD + iSCSI) + bitmap + --write-mostly +
--write-behind, XFS
CPU BOGOMIPS: 36176.88
REGEX/SECOND: 790823
HD SIZE: 279.32 GB (/dev/md0)
BUFFERED READS: 152.35 MB/sec
AVERAGE SEEK TIME: 0.60 ms
FSYNCS/SECOND: 2650.10
DNS EXT: 122.49 ms
Where nothing is noted, defaults where used (ext4/xfs mount options,
write-behind max operations, ...). EXT4 filesystems where all created
with -m 0 (0% reserved)
Any comment or thought is appreciated :)
Cheers
Eneko
--
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Telf. 943575997
943493611
Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
www.binovo.es
_______________________________________________
pve-user mailing list
pve-user@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
