I'm seeing a 10-fold performance hit when using an LVM2 logical volume that sits on top of a RAID0 stripe. Using dd to read directly from the stripe (i.e. a large sequential read) I get speeds over 600MB/s. Reading from the logical volume using the same method only gives around 57MB/s. I am new to LVM and I need to for the snapshots. Would anyone suggest where to start looking for the problem?
The server runs the amd64 version of Lenny. Most packages (including lvm2) are stock from Lenny, but we had to upgrade the kernel to the one from lenny-backports (2.6.32). There are ten RAID1 triplets: md0 through md9 (that's 30 physical disks arranged into ten 3-way mirrors), connected over iSCSI from six targets. The ten triplets are then striped together into a RAID0 stripe /dev/md10. I don't think we have any issues with the MD layers, because each of them seems to perform fairly well; it's when we add LVM into the soup the speeds start getting slow. test4:~# uname -a Linux test4 2.6.32-bpo.4-amd64 #1 SMP Thu Apr 8 10:20:24 UTC 2010 x86_64 GNU/Linux test4:~# dd of=/dev/null bs=8K count=2500000 if=/dev/md10 2500000+0 records in 2500000+0 records out 20480000000 bytes (20 GB) copied, 33.4619 s, 612 MB/s test4:~# dd of=/dev/null bs=8K count=2500000 if=/dev/vg0/lvol0 2500000+0 records in 2500000+0 records out 20480000000 bytes (20 GB) copied, 354.951 s, 57.7 MB/s I used the following commands to create the volume group: pvcreate /dev/md10 vgcreate vg0 /dev/md10 lvcreate -l 102389 vg0 Here's what LVM reports of its devices: test4:~# pvdisplay --- Physical volume --- PV Name /dev/md10 VG Name vg0 PV Size 399.96 GB / not usable 4.00 MB Allocatable yes (but full) PE Size (KByte) 4096 Total PE 102389 Free PE 0 Allocated PE 102389 PV UUID ocIGdd-cqcy-GNQl-jxRo-FHmW-THMi-fqofbd test4:~# vgdisplay --- Volume group --- VG Name vg0 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 2 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 399.96 GB PE Size 4.00 MB Total PE 102389 Alloc PE / Size 102389 / 399.96 GB Free PE / Size 0 / 0 VG UUID o2TeAm-gPmZ-VvJc-OSfU-quvW-OB3a-y1pQaB test4:~# lvdisplay --- Logical volume --- LV Name /dev/vg0/lvol0 VG Name vg0 LV UUID Q3nA6w-0jgw-ImWY-IYJK-kvMJ-aybW-GAdoOs LV Write Access read/write LV Status available # open 0 LV Size 399.96 GB Current LE 102389 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:0 Many thanks in advance for any pointers! -- Arcady Genkin -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktiksmhwitdv1_iji72tak_1irx9dxpj2mccah...@mail.gmail.com