Re: Tests of RAID adapters
On Mon, Apr 25, 2011 at 07:38:57PM +0200, Sascha Wildner wrote: On 4/25/2011 9:12, Francois Tigeot wrote: LSI SAS 3081E-R --- The RAID1 volume created in the BIOS of the card was visible but there were some timeout error messages from the start: I couldn't install DragonFly, newfs_hammer hung at 90% completion. I've put an update for mpt(4) here: http://leaf.dragonflybsd.org/~swildner/0001-mpt-4-Sync-with-FreeBSD.patch Thanks for updating the driver. With this new version, things are much better :) I've used a similar setup as before, with the following changes: - host system is running DragonFly 2.10 + updated mps driver - host system is a 4-core Xeon E5506 - the drives are 2x WD5003ABYX (500 GB, 7200 RPM, 64MB cache) Observations: 1. no timeouts on startup mpt0: LSILogic SAS/SATA Adapter port 0xe000-0xe0ff mem 0xfbbf-0xfbbf,0xfbbec000-0xfbbe irq 16 at device 0.0 on pci3 mpt0: MPI Version=1.5.20.0 mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 ) mpt0: 1 Active Volume (2 Max) mpt0: 2 Hidden Drive Members (14 Max) mpt0:vol0(mpt0:0:0): Settings ( Hot-Plug-Spares ) mpt0:vol0(mpt0:0:0): Using Spare Pool: 0 mpt0:vol0(mpt0:0:0): 2 Members: (mpt0:1:2:0): Primary Online (mpt0:1:1:0): Secondary Online mpt0:vol0(mpt0:0:0): RAID-1 - Optimal mpt0:vol0(mpt0:0:0): Status ( Enabled ) (mpt0:vol0:1): Physical (mpt0:0:1:0), Pass-thru (mpt0:1:0:0) (mpt0:vol0:1): Online (mpt0:vol0:0): Physical (mpt0:0:2:0), Pass-thru (mpt0:1:1:0) (mpt0:vol0:0): Online da0 at mpt0 bus 0 target 0 lun 0 da0: LSILOGIC Logical Volume 3000 Fixed Direct Access SCSI-2 device da0: 300.000MB/s transfers da0: 476837MB (976562176 512 byte sectors: 255H 63S/T 60788C) no B_DEVMAGIC (bootdev=0) 2. newfs_hammer completes succesfully, although performance is bad $ iostat da0 2 tty da0 cpu tin tout KB/t tps MB/s us ni sy in id 1 245 0.000 0.00 0 0 2 0 98 0 40 16.00 118 1.85 0 0 0 0 100 0 643 16.00 116 1.81 0 0 0 0 100 3. blogbench also runs without crashing $ blogbench -d /mnt/bench [...] Final score for writes: 1407 Final score for reads :244892 $ iostat da0 2 tty da0 cpu tin tout KB/t tps MB/s us ni sy in id 0 22 15.99 3045 47.55 11 0 89 0 0 0 11 15.86 865 13.39 11 0 89 0 0 0 11 15.98 1886 29.44 11 0 89 0 0 0 23 16.00 2974 46.46 10 0 90 0 1 For information, this is the blogbench score of a disk of the same model plugged on one of the AHCI SATA ports: Final score for writes: 1307 Final score for reads :215165 -- Francois Tigeot
Re: Tests of RAID adapters
On Wed, Apr 27, 2011 at 02:18:49PM +0200, Francois Tigeot wrote: On Mon, Apr 25, 2011 at 07:38:57PM +0200, Sascha Wildner wrote: On 4/25/2011 9:12, Francois Tigeot wrote: LSI SAS 3081E-R --- [...] I've put an update for mpt(4) here: http://leaf.dragonflybsd.org/~swildner/0001-mpt-4-Sync-with-FreeBSD.patch [...] 2. newfs_hammer completes succesfully, although performance is bad The bad performance was caused by disabled write-cache on the disks. MPT SAS controllers disable disk cache for writes by default and do not allow the user to enable it in their BIOS setup. The disks I used can read/write linearly at about 135 MB/s. On startup, linear read speed on the array is 133 MB/s, and write speed is an abysmal 13.50 MB/s (no, it's not a typo) Fortunately, it's possible to enable write-cache again with a sysctl: # sysctl -w hw.mpt0.vol_member_wce=On After typing that command, linear speeds became 136 MB/s for reads and 132 MB/s for writes. -- Francois Tigeot
Tests of RAID adapters
Hi all, During last week, I've done some systematic hardware tests to determine which recent RAID cards work fine with DragonFly. I've bought an Areca ARC-1880-i adapter and thanks to a local server-class hardware distributor, I've been able to test a grand total of 11 different SAS or SATA RAID adapters. The tests were done with a DragonFly 2.9/x86_64 snapshot from 2011-04-19. Besides the various RAID adapters, the following hardware was used: - 1x Supermicro X7SPA-H mainboard with Atom D510 CPU - 1x 2GB DDR2 667 SODIMM - 2x Samsung SATA 3Gb/s 160GB hard drives - 1x SATA DVD-ROM Drive plugged on one of the motherboard's SATA ports For the tests, all RAID volumes were configured as a mirror with the two Samsung disks (RAID 1) List of RAID adapters: - Areca ARC-1200 (SATA only) - Areca ARC-1210 (SATA only) - Areca ARC-1222 - Areca ARC-1680-ix - Areca ARC-1880-i - LSI MegaRAID SAS 8204ELP - LSI SAS 3081E-R - LSI SAS 9211-8i - 3Ware 9690SA-4i - Adaptec AAR-1120SA - Adaptec 3405 Areca ARC-1200 -- http://www.areca.com.tw/products/pciex1_2p_internal.htm This is a SATA card only The card was immediately recognized by the DragonFly snapshot. There were no problems to install the system and to run the new installation from the RAID volume. dmesg extract: arcmsr0: Areca SATA Host Adapter RAID controller Areca ARC-1210 -- http://www.areca.com.tw/products/pcie.htm This is a SATA card only This card was quite similar to the ARC-1200; as before, all went well. Areca ARC-1222 -- http://www.areca.com.tw/products/pcietosas01.htm The adapter was recognized out of the box: arcmsr0: Areca SAS 3G Host Adapter RAID contreoller (RAID6 capable) mem ... irq16 at device 0.0 ARECA RAID ADAPTER0: Driver Version 1.20.00.19 2010-11-11 ARECA RAID ADAPTER0: FIRMWARE version v1.48 2010-07-08 A RAID volume not beeing fully initialized is not detected. Once the initialization is complete or if background initialization is selected in the adapter BIOS setup, the volume is correctly recognized: da0 at arcmsr0 bus0 target0 lun0 da0: Areca ARC-1212-VOL#000 R001 Fixed Direct Access SCSI-5 device da0: Serial Number 41811d2909406830 da0: 166.666MB/s transfers (83.333 MHz, offset 32, 16bit) da0: Command queuing Enabled da0: 152587MB (312499200 512 byte sectors: 255H 63 S/t 19452C) A linear read of the first 1.25 GB of the drive gave a speed of 127.24 MB/s, twice that of the previous models, which lead me to believe the reading was parallelized on both drives. Areca ARC-1680-ix http://www.areca.com.tw/products/pcietosas1680series.htm As before, all went well. The adapter recognized the volume created on the ARC-1222 and could boot from it. Areca ARC-1880-i http://www.areca.com.tw/products/1880.htm A complete installation and boot from the new system went fine. dmesg extract: arcmsr0: Areca SAS 6G Host Adapter RAID Controller (RAID6 capable) port 0xc800-0xc8ff mem 0xfe94-0xfe97,0xfe93-0xfe93 irq 11 at device 0.0 on pci1 ARECA RAID ADAPTER0: Driver Version 1.20.00.19 2010-11-11 ARECA RAID ADAPTER0: FIRMWARE VERSION V1.49 2010-12-10 da0 at arcmsr0 bus 0 target 0 lun 0 da0: Areca ARC-1880-VOL#000 R001 Fixed Direct Access SCSI-5 device da0: Serial Number 4187860270766765 da0: 166.666MB/s transfers (83.333MHz, offset 32, 16bit) da0: Command Queueing Enabled da0: 152587MB (312499200 512 byte sectors: 255H 63S/T 19452C) LSI MegaRAID SAS 8204ELP http://www.lsi.com/storage_home/products_home/internal_raid/megaraid_sas/megaraid_sas_8204elp/ Even though a RAID volume had been created in the BIOS of the card, it was not visible from DragonFly The individual disks were visible as da0 and da1. LSI SAS 3081E-R --- http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3081er/ The RAID1 volume created in the BIOS of the card was visible but there were some timeout error messages from the start: dmesg extract: mpt0: LSI Logic SAS/SATA Adapter mpt0: MPI Version = 1.5.20.0 mpt0: Capabilities: (RAID-0 RAID-1E RAID-1) mpt0: 1 Active Volume (2 Max) mpt0: 2 Hidden Drive Members (14 Max) ... mpt0: mpt_wait_req(4) timed out mpt0: read_cfg_header timed out mpt0: vol0(mpt0:0:0) mpt_refresh_raid_vol: Failed to read RAID vol Hdr(0) mpt0: vol0(mpt0:0:0): Settings() mpt0: vol0(mpt0:0:0): 0 Members: mpt0: vol0(mpt0:0:0): RAID0-Optimal (mpt0:0:1): Physical(mpt0:0:1:0),Pass-thru (mpt0:1:0:0) (mpt0:0:1): Online (mpt0:0:2): Physical(mpt0:0:2:0),Pass-thru (mpt0:1:1:0) (mpt0:0:2): Online ... da0 at mpt0 bus 0 target 0 lun 0 da0: LSILOGIC Logical Volume 3000 Fixed DirectAccess SCSI-2 device da0: 300.000MB/s transfers da0: 151634MB (310546432 512 byte sectors: 255H 63S/T 19330C) I couldn't install DragonFly, newfs_hammer hung at 90% completion. The disks were making strange noises, as if they were
Re: Tests of RAID adapters
25 apr 2011 kl. 09.12 skrev Francois Tigeot: Hi all, During last week, I've done some systematic hardware tests to determine which recent RAID cards work fine with DragonFly. ... Conclusions --- I'm very impressed by Areca. All their cards worked flawlessly out of the box. The Areca people were very cooperative, they tested and reviewed the initial port of the arcmsr driver (from FreeBSD). Many of their adapters include an ethernet port which can be used for supervision, removing the need to install a special low-level utility in the host OS. I only had access to one 3Ware adapter, but it also worked out of the box and I have reports of different recent models also working flawlessly. There are some LSI models known to work flawlessly. Unfortunately, they seem to be old products which are not sold anymore. With all the troubles I had getting Adaptec cards to work reliably before DragonFly was even booted, I cannot seriously consider to purchase products of this brand. If you want a RAID adapter to use with DragonFly, Areca and 3Ware are the two best choices of the moment, my first choice beeing Areca. Tests like these are awesome. Thank you! max
Re: Tests of RAID adapters
On 4/25/2011 9:12, Francois Tigeot wrote: LSI SAS 3081E-R --- http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3081er/ The RAID1 volume created in the BIOS of the card was visible but there were some timeout error messages from the start: dmesg extract: mpt0:LSI Logic SAS/SATA Adapter mpt0: MPI Version = 1.5.20.0 mpt0: Capabilities: (RAID-0 RAID-1E RAID-1) mpt0: 1 Active Volume (2 Max) mpt0: 2 Hidden Drive Members (14 Max) ... mpt0: mpt_wait_req(4) timed out mpt0: read_cfg_header timed out mpt0: vol0(mpt0:0:0) mpt_refresh_raid_vol: Failed to read RAID vol Hdr(0) mpt0: vol0(mpt0:0:0): Settings() mpt0: vol0(mpt0:0:0): 0 Members: mpt0: vol0(mpt0:0:0): RAID0-Optimal (mpt0:0:1): Physical(mpt0:0:1:0),Pass-thru (mpt0:1:0:0) (mpt0:0:1): Online (mpt0:0:2): Physical(mpt0:0:2:0),Pass-thru (mpt0:1:1:0) (mpt0:0:2): Online ... da0 at mpt0 bus 0 target 0 lun 0 da0:LSILOGIC Logical Volume 3000 Fixed DirectAccess SCSI-2 device da0: 300.000MB/s transfers da0: 151634MB (310546432 512 byte sectors: 255H 63S/T 19330C) I couldn't install DragonFly, newfs_hammer hung at 90% completion. The disks were making strange noises, as if they were constantly seeking I've put an update for mpt(4) here: http://leaf.dragonflybsd.org/~swildner/0001-mpt-4-Sync-with-FreeBSD.patch Please test it when you get the chance. Sasca
Re: Tests of RAID adapters
Am 25.04.2011 09:12, schrieb Francois Tigeot: [...] Conclusions --- If you want a RAID adapter to use with DragonFly, Areca and 3Ware are the two best choices of the moment, my first choice beeing Areca. [...} This is very interesting. Thanks a lot for the effort Jan -- professional: http://www.oscar-consult.de private: http://neslonek.homeunix.org/drupal/
