raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G)
>>local-highspeed is LogicalVolume on Raid-10 lvm volume ? classic lvm or thin lvm ? >>raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G) >>"do you have cache on your raid controller ?" I don't know what's means? Well, generally (for hdd mainly), hardware raid controller have memory to handle writes fast, then flush them to disk. what is your model ? >>fio command is : ./fio -filename=/dev/vda -direct=1 -iodepth 1 -thread >>-rw=randwrite -ioengine=psync -bs=16k -size=4G -numjobs=30 -runtime=100 >>-group_reporting -name=mytest So you are doing sync write, that mean that it should bypass all caches (including raid controller cache, disk cache) to be sure that data are correctly written to the disk platters. I'm surprised than you can reach iops=1344 with 6 SAS 10K in raid10. (should be 250 iops by disk , x3 because of raid10, so around 750iops). Now, some raid hardware controller, can handle the sync writes, without transmit them to disk platter (They need to have a batery, or persistent memory). Could explain why you have better performance with your san. can you do fio benchmark with ioengin : libaio , and increase iodepth to 32 ./fio -filename=/dev/vda -direct=1 -iodepth 32 -rw=randwrite -ioengine=libaiO -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest then compare with --sync=1 option ./fio -filename=/dev/vda -direct=1 -iodepth 32 -thread -rw=randwrite -ioengine=libaiO -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest ----- Mail original ----- De: "haoyun" <[email protected]> À: "proxmoxve" <[email protected]> Envoyé: Mercredi 15 Juin 2016 03:51:03 Objet: Re: [PVE-User] vm's disk io lower with local highspeed disk? fio command is : ./fio -filename=/dev/vda -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest vmconfig: balloon: 1024 bootdisk: virtio2 cores: 2 ide2: none,media=cdrom memory: 4096 name: tt net0: bridge=vmbr0,e1000=66:37:31:32:34:37 numa: 0 ostype: l26 smbios1: uuid=ce775a6f-6652-43c4-aa92-69ffd3d20013 sockets: 1 virtio0: local-highspeed:vm-255-disk-2,size=60G virtio2: local-highspeed:vm-255-disk-1,size=8G raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G) local-highspeed is LogicalVolume on Raid-10 "do you have cache on your raid controller ?" I don't know what's means? At 2016-06-14 17:59:17, "Alexandre DERUMIER" <[email protected]> wrote: >>>Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm >>>%util >>>vda 0.00 71.00 8.00 226.00 64.00 2368.00 10.39 1.37 5.85 4.24 99.10 >>>but I test disk random write is better with fio tools, >>>bw=21518KB/s iops=1344 > > >do you have cache on your raid controller ? > >can you send the fio job config? > >can you send vmconfig ? > >what is the local raid10 filesystem ? > > > > >----- Mail original ----- >De: "haoyun" <[email protected]> >À: "proxmoxve" <[email protected]> >Envoyé: Mardi 14 Juin 2016 07:53:29 >Objet: [PVE-User] vm's disk io lower with local highspeed disk? > >hello everyone~ > > >I have a problem, >I have a new host is Dell R330, >run pve4.2 os >disk with 2 raid,one is raid1,another is raid10(sas 10k 600G * 6) >raid1 as system disk,raid10 as vm images >my mysql slave run on this vm with raid10 local disk(lvm). >the question is this vm's disk io is very slow, >that's why? >Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm >%util >vda 0.00 71.00 8.00 226.00 64.00 2368.00 10.39 1.37 5.85 4.24 99.10 >but I test disk random write is better with fio tools, >bw=21518KB/s iops=1344 >same vm,I use iscsi remote storage better than it, >Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm >%util >vda 0.00 0.00 0.00 3.00 0.00 24.00 8.00 0.00 0.33 0.33 0.10 >that's why? >who can help me? >What information is needed？please tell me? >thanks! >_______________________________________________ >pve-user mailing list >[email protected] >http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user >_______________________________________________ >pve-user mailing list >[email protected] >http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user _______________________________________________ pve-user mailing list [email protected] http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user _______________________________________________ pve-user mailing list [email protected] http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
