On Fri, 14 Mar 2014 12:21:43 -0700 Chris Allen <ca.al...@gmail.com> wrote:
> 8k for me too is much better than 4k. With 4k I tend to hit my IOPS limit > easily, with not much throughput, and I get a lot of IO delay on VMs when > the SAN is fairly busy. Currently I'm leaning towards 16k, sparse, with > lz4 compression. If you go the sparse route then compression is a no > brainer as it accelerates performance on the underlying storage > considerably. Compression will lower your IOPS and data usage both are > good things for performance. ZFS performance drops as usage rises and gets > really ugly at around 90% capacity. Some people say it starts to drop with > as little as 10% used, but I have not tested this. With 16k block sizes > I'm getting good compression ratios - my best volume is 2.21x, my worst > 1.33x, and the average is 1.63x. So as you can see a lot of the time my > real block size on disk is going to be effectively smaller than 16k. The > tradeoff here is that compression ratios will go up with a larger block > size, but you'll have to do larger operations and thus more waste will > occur when the VM is doing small I/O. With a large block size on a busy > SAN your I/O is going to get fragmented before it hits the disk anyway, so > I think 16k is good balance. I only have 7200 RPM drives in my array, but > a ton of RAM and a big ZFS cache device, which is another reason I went > with 16k, to maximize what I get when I can get it. I think with 15k RPM > drives 8k block size might be better, as your IOPS limit will be roughly > double that of 7200 RPM. > nice catch. I haven't thought of this. I will experiment some more with sparse volumes. I already have lz4 activated on all my pools and can proof that performance actually increases using compression. Hint: If you haven't already tested this you should try making some performance tests using RAID10. 12 disks (4 vdevs with a 3 disk mirror gives excellent speed/IO with reasonable security (you can loose 2 disks in any vdev and still not loose any data from the pool) > Dedup did not work out well for me. Aside from the huge memory > consumption, it didn't save all that much space and to save the max space > you need to match the VM's filesystem cluster size to the ZVOL block size. > Which means 4k for ext4 and NTFS (unless you change it during a Windows > install). Also dedup really really slows down zpool scrubbing and possibly > rebuild. This is one of the main reasons I avoid it. I don't want scrubs > to take forever, when I'm paranoid of something potentially being wrong. > I don't recall anybody have provided a proof for anything good coming out of using dedub. > > > Regards write caching: Why not simply use sync > > directly on the volume? > > Good question. I don't know. > > If you use Omnios you should install napp-it. With napp-it administration of the Omnios storage cluster is a breeze. Changing caching policy is two clicks with a mouse. > I'm in the process of trying to run away from all things Oracle at my > company. We keep getting burned by them. It's so freakin' expensive, and > they hold you over a barrel with patches for both hardware and software. > We bought some very expensive hardware from them, and a management > controller for a blade chassis had major bugs to the point it was > practically unusable out of the box. Oracle would not under any > circumstance supply us with the new firmware unless we spent boatloads of > cash for a maintenance contract. We ended up doing this because we needed > the controller to work as advertised. This is what annoys me the most with > them - you buy a product and it doesn't do what is written on the box and > then you have to pay tons extra for it to do what they said it would do > when you bought it. I miss Sun... > Hehe, more or less the same story here. We stick to Oracle database and application servers though for mission critical data since having a comparable setup from MS puts huge demands on hardware. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael <at> rasmussen <dot> cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir <at> datanom <dot> net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir <at> miras <dot> org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: How should I know if it works? That's what beta testers are for. I only coded it. -- Attributed to Linus Torvalds, somewhere in a posting
signature.asc
Description: PGP signature
_______________________________________________ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
