Re: Extremely slow write speeds to disk : FreeBSD 7.1 and Dell SAS 6/iR Adapter
Dieter wrote: But the issue is that writing to disk is extremely slow...i.e. 14Mb per second If I install any other linux distribution, like CentOS, write speek is the way it should be..it averages around 190Mb+ per second I was hoping if it were at all possible for you to assist me in with a later RAID Controller Driver for the SAS 6/iR Adapter Try: hw.mpt.enable_sata_wc=1 in /boot/loader.conf and reboot. Assuming you are using SATA drives, the write cache is disabled by default. This reenables it. The mpt manpage holds more information. Just keep in mind that turning the disk's write cache on puts your data at risk. :-( The correct solution is NCQ. Just wondering, if you have a battery on your RAID array, is this a problem? Wouldn't the cache get written anyway in case of a crash? Regards, Sebastiaan smime.p7s Description: S/MIME Cryptographic Signature
Re: FreeBSD 7.1 disk performance issue on ESXi 3.5
Hi, ivo...@gmail.com wrote: As for the original thread topic: I've communicated with the OP and it appears his method of benchmarking had an error so the problems that appear in his post are bogus. It is not quite true that the "method" is bogus, there just seems to be a huge difference between a soft updates vs non-soft-updates disk. These are the results I get now: dbench -D -t 60 1 on / (ufs, local): Throughput 13.4561 MB/sec 1 procs on /tmp (ufs, local, soft-updates): Throughput 92.299 MB/sec 1 procs However, whether it is caching or not, Linux gets 350 MB/s using 1 process and even 650 MB/s using 2. As I understand it, this shouldn't be possible on the physical disks, but still, the *virtual* disk seems to get this performance. When I benchmark the linux vs the freebsd using Unixbench 4.1/5.1 (I tried both) I also get ***HUGE*** differences: System: test-fbsd.vpn1.sebster.com: FreeBSD Benchmark Run: Tue Feb 10 2009 06:25:49 - 06:54:08 2 CPUs in system; running 1 parallel copy of tests System Benchmarks Index Values BASELINE RESULTINDEX Dhrystone 2 using register variables 116700.0 14144383.9 1212.0 Double-Precision Whetstone 55.0 3238.7588.9 Execl Throughput 43.0630.0146.5 File Copy 1024 bufsize 2000 maxblocks 3960.0 28793.2 72.7 File Copy 256 bufsize 500 maxblocks1655.0 33410.0201.9 File Copy 4096 bufsize 8000 maxblocks 5800.0 33536.8 57.8 Pipe Throughput 12440.01146784.7921.9 Pipe-based Context Switching 4000.0 36203.6 90.5 Process Creation126.0783.3 62.2 Shell Scripts (1 concurrent) 42.4645.1152.2 Shell Scripts (8 concurrent) 6.0115.4192.3 System Call Overhead 15000.0 939647.5626.4 System Benchmarks Index Score 212.4 System: test-ubuntu: GNU/Linux Benchmark Run: Mon Feb 09 2009 15:15:06 - 15:43:20 2 CPUs in system; running 1 parallel copy of tests System Benchmarks Index Values BASELINE RESULTINDEX Dhrystone 2 using register variables 116700.0 18610575.3 1594.7 Double-Precision Whetstone 55.0 2990.1543.7 Execl Throughput 43.0 1058.6246.2 File Copy 1024 bufsize 2000 maxblocks 3960.0 468973.2 1184.3 File Copy 256 bufsize 500 maxblocks1655.0 132022.2797.7 File Copy 4096 bufsize 8000 maxblocks 5800.0 921448.5 1588.7 Pipe Throughput 12440.01132933.6910.7 Pipe-based Context Switching 4000.0 93429.0233.6 Process Creation126.0 1744.3138.4 Shell Scripts (1 concurrent) 42.4 2566.9605.4 Shell Scripts (8 concurrent) 6.0518.4864.0 System Call Overhead 15000.01935577.0 1290.4 System Benchmarks Index Score 656.1 Here the disk intensive test (file copy) and context switch/process creation test do terrible. For all my personal servers this is not an issue for me at all. But for a big high traffic web site I'm building, I'm afraid I'm going to have to go for Linux. :-( Regards, Sebastiaan ___ freebsd-performance@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to "freebsd-performance-unsubscr...@freebsd.org" smime.p7s Description: S/MIME Cryptographic Signature
Re: FreeBSD 7.1 disk performance issue on ESXi 3.5
Sebastiaan van Erk wrote: (However, just to give you an idea I attached the basic 5.1.2 unixbench outputs (the CPU info for FreeBSD is "fake", since unixbench does a cat /proc/cpuinfo, so I removed the /proc/ part and copied the output under linux to the "procinfo" file.) Of course I forgot to attach them... :-( Here they are. Regards, Sebastiaan gmake all gmake[1]: Entering directory `/root/tmp/unixbench-5.1.2' Checking distribution of files ./pgms exists ./src exists ./testdir exists ./tmp exists ./results exists gmake[1]: Leaving directory `/root/tmp/unixbench-5.1.2' ## ## # ## # ## ## ## ## ## # # # # ## # ## # ## ## ## # # # #### # # # # # ## ## # # # ##### # # # # # ## ## # ## # # # ## # # ## ## ## ## # ## # ## ## ## Version 5.1.2 Based on the Byte Magazine Unix Benchmark Multi-CPU version Version 5 revisions by Ian Smith, Sunnyvale, CA, USA December 22, 2007 johantheghost at yahoo period com 1 x Dhrystone 2 using register variables 1 2 3 4 5 6 7 8 9 10 1 x Double-Precision Whetstone 1 2 3 4 5 6 7 8 9 10 1 x Execl Throughput 1 2 3 1 x File Copy 1024 bufsize 2000 maxblocks 1 2 3 1 x File Copy 256 bufsize 500 maxblocks 1 2 3 1 x File Copy 4096 bufsize 8000 maxblocks 1 2 3 1 x Pipe Throughput 1 2 3 4 5 6 7 8 9 10 1 x Pipe-based Context Switching 1 2 3 4 5 6 7 8 9 10 1 x Process Creation 1 2 3 1 x System Call Overhead 1 2 3 4 5 6 7 8 9 10 1 x Shell Scripts (1 concurrent) 1 2 3 1 x Shell Scripts (8 concurrent) 1 2 3 2 x Dhrystone 2 using register variables 1 2 3 4 5 6 7 8 9 10 2 x Double-Precision Whetstone 1 2 3 4 5 6 7 8 9 10 2 x Execl Throughput 1 2 3 2 x File Copy 1024 bufsize 2000 maxblocks 1 2 3 2 x File Copy 256 bufsize 500 maxblocks 1 2 3 2 x File Copy 4096 bufsize 8000 maxblocks 1 2 3 2 x Pipe Throughput 1 2 3 4 5 6 7 8 9 10 2 x Pipe-based Context Switching 1 2 3 4 5 6 7 8 9 10 2 x Process Creation 1 2 3 2 x System Call Overhead 1 2 3 4 5 6 7 8 9 10 2 x Shell Scripts (1 concurrent) 1 2 3 2 x Shell Scripts (8 concurrent) 1 2 3 BYTE UNIX Benchmarks (Version 5.1.2) System: test-fbsd.vpn1.sebster.com: FreeBSD OS: FreeBSD -- 7.1-RELEASE -- FreeBSD 7.1-RELEASE #1: Mon Feb 9 18:26:19 CET 2009 r...@test-fbsd.vpn1.sebster.com:/usr/obj/usr/src/sys/VMWARE Machine: amd64 (VMWARE) Language: en_US.utf8 (charmap=, collate=) CPU 0: Intel(R) Xeon(R) CPU E5420 @ 2.50GHz (4999.9 bogomips) x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET CPU 1: Intel(R) Xeon(R) CPU E5420 @ 2.50GHz (5000.8 bogomips) x86-64, MMX, Physical Address Ext, SYSENTER/SYSEXIT, SYSCALL/SYSRET 6:25AM up 6:54, 1 user, load averages: 0.08, 0.02, 0.01; runlevel Benchmark Run: Tue Feb 10 2009 06:25:49 - 06:54:08 2 CPUs in system; running 1 parallel copy of tests Dhrystone 2 using register variables 14144383.9 lps (10.0 s, 7 samples) Double-Precision Whetstone 3238.7 MWIPS (9.9 s, 7 samples) Execl Throughput630.0 lps (29.9 s, 2 samples) File Copy 1024 bufsize 2000 maxblocks 28793.2 KBps (30.0 s, 2 samples) File Copy 256 bufsize 500 maxblocks 33410.0 KBps (30.0 s, 2 samples) File Copy 4096 bufsize 8000 maxblocks 33536.8 KBps (30.1 s, 2 samples) Pipe Throughput 1146784.7 lps (10.0 s, 7 samples) Pipe-based Context Switching 36203.6 lps (10.0 s, 7 samples) Process Creation783.3 lps (30.0 s, 2 samples) Shell Scripts (1 concurrent)645.1 lpm (60.1 s, 2 samples) Shell Scripts (8 concurrent)115.4 lpm (60.1 s, 2 samples) System Call Overhead 939647.5 lps (10.0 s, 7 samples) System Benchmarks Index Values BASELINE RESULTINDEX Dhrystone 2 using register variables 116700.0 14144383.9 1212.0 Double-Precision Whetstone 55.0 3238.7588.9 Execl Throughput 43.0630.0146.5 File Copy 1024 bufsize 2000 maxblocks 3960.0 28793.2 72.7 File Copy 256 bufsize 500 maxblocks1655.0 33410.0201.9 File Copy 4096 bufsize 8000 maxblocks 5800.0 33536.8 57.8 Pipe Throughput 12440.01146784.7921.9 Pipe-based Context Switching
Re: FreeBSD 7.1 disk performance issue on ESXi 3.5
Ivan Voras wrote: Hi, Thanks for the reply. Sebastiaan van Erk wrote: Hi, [snip] 1 2 4 freebsd 12.0009 13.6348 12.9402 (MB/s) linux 376.145 651.314 634.649 (MB/s) Both virtual machines run dbench 3.04 and the results are extremely stable over repeated runs. VMWare has many optimizations for Linux that are not used with FreeBSD. VMI, for example, makes the Linux guest paravirtualized, and then there are special drivers for networking, its vmotion driver (this one probably doesn't contribute to performance much), etc. and Linux is in any case much better tested and supported. VMI/paravirtualization is not enabled for this Linux host. Neither is VMotion. Networking is performing extremely well (see also below). If VMWare allows, you may try changing the type of the controller (I don't know about ESXi but VMWare Server supports LSI or Buslogic SCSI emulation) or switch to ATA emulation and try again. I tried this, and it has no significant effect. Just for completeness, here's the relevant output of dmesg: bt0: port 0x1060-0x107f mem 0xf481-0xf481001f irq 17 at device 16.0 on pci0 bt0: BT-958 FW Rev. 5.07B Ultra Wide SCSI Host Adapter, SCSI ID 7, 192 CCBs bt0: [GIANT-LOCKED] bt0: [ITHREAD] da0 at bt0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 40.000MB/s transfers (20.000MHz DT, offset 15, 16bit) da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) The transfer rate for dbench 1 is 15.0118 MB/s. A generic optimization is to reduce kern.hz to something like 50 but it probably won't help your disk performance. I already had this (not 50, but 100), but this doesn't do anything for the disk performance. As for unixbench, you need to examine and compare each microbenchmark result individually before drawing a conclusion. Yes, I realize that. However the dbench result is my first priority, and when (if) that is fixed, I'll run the unixbench again and see what my next priority is. (However, just to give you an idea I attached the basic 5.1.2 unixbench outputs (the CPU info for FreeBSD is "fake", since unixbench does a cat /proc/cpuinfo, so I removed the /proc/ part and copied the output under linux to the "procinfo" file.) Finally, I also ran some network benchmarks such as netio, and tested VM to VM communication on *different* ESXi machines connected via Gigabit ethernet, and it achieved more than 100MB/s throughput. Since CPU speed + Network IO are doing just fine, I'm guessing this is a pure disk (driver?) related issue. However, to go into production with FreeBSD I *must* be able to fix it. Note also the discrepency: 12 MB/s vs 350 MB/s on disk access! My lousy home machine (FreeBSD) is even 5 times faster at 60 MB/s. This machine has extremely fast disks in a RAID10 configuration. Any ideas are welcome! Regards, Sebastiaan smime.p7s Description: S/MIME Cryptographic Signature
FreeBSD 7.1 disk performance issue on ESXi 3.5
Hi, I want to deploy a production FreeBSD web site (database cluster, apache cluster, ip failover using carp, etc.), however I'm experiencing painful disk I/O throughput problems which currently does not make the above project viable. I've done some rudimentary benchmarking of two identically configured virtual machines (2 VCPUs, 512MB memory, 8GB disk) and installed one with FreeBSD 7.1-amd64 and one with Linux Ubuntu 8.10-amd64. These are the results I'm getting with dbench : 1 2 4 freebsd 12.0009 13.6348 12.9402 (MB/s) linux 376.145 651.314 634.649 (MB/s) Both virtual machines run dbench 3.04 and the results are extremely stable over repeated runs. The virtual hardware detected by the FreeBSD machine is as follows: mpt0: port 0x1080-0x10ff mem 0xf481-0xf4810fff irq 17 at device 16.0 on pci0 mpt0: [ITHREAD] mpt0: MPI Version=1.2.0.0 And: da0 at mpt0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 3.300MB/s transfers da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) I've also run unixbench (4.1 and 5.1.2) and the performance of the FreeBSD machine is horrible compared to Linux on many of the tests, though my first guess is that it all comes back down the disk performance (on the CPU-only tests the results are about the same). Online when I see logs of da0 specs via google, they more look more like this (much higher transfer rate, and SCSI-n, n>2): da0: Fixed Direct Access SCSI-5 device da0: 300.000MB/s transfers Does anybody know how I can get proper performance for the drive under ESXi? Regards, Sebastiaan smime.p7s Description: S/MIME Cryptographic Signature
