[zfs-discuss] Question on 4k sectors
Hi. Is the problem with ZFS supporting 4k sectors or is the problem mixing 512 byte and 4k sector disks in one pool, or something else? I have seen alot of discussion on the 4k issue but I haven't understood what the actual problem ZFS has with 4k sectors is. It's getting harder and harder to find large disks with 512 byte sectors so what should we do? TIA... ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Server with 4 drives, how to configure ZFS?
Edward Ned Harvey opensolarisisdeadlongliveopensola...@nedharvey.com wrote: Well ... Slice all 4 drives into 13G and 60G. Use a mirror of 13G for the rpool. Use 4x 60G in some way (raidz, or stripe of mirrors) for tank Use a mirror of 13G appended to tank Hi Edward! Thanks for your post. I think I understand what you are saying but I don't know how to actually do most of that. If I am going to make a new install of Solaris 10 does it give me the option to slice and dice my disks and to issue zpool commands? Until now I have only used Solaris on Intel with boxes and used both complete drives as a mirror. Can you please tell me what are the steps to do your suggestion? I imagine I can slice the drives in the installer and then setup a 4 way root mirror (stupid but as you say not much choice) on the 13G section. Or maybe one root mirror on two slices and then have 13G aux storage left to mirror for something like /var/spool? What would you recommend? I didn't understand what you suggested about appending a 13G mirror to tank. Would that be something like RAID10 without actually being RAID10 so I could still boot from it? How would the system use it? In this setup that will install everything on the root mirror so I will have to move things around later? Like /var and /usr or whatever I don't want on the root mirror? And then I just make a RAID10 like Jim was saying with the other 4x60 slices? How should I move mountpoints that aren't separate ZFS filesystems? The only conclusion you can draw from that is: First take it as a given that you can't boot from a raidz volume. Given, you must have one mirror. Thanks, I will keep it in mind. Then you raidz all the remaining space that's capable of being put into a raidz... And what you have left is a pair of unused space, equal to the size of your boot volume. You either waste that space, or you mirror it and put it into your tank. So RAID10 sounds like the only reasonable choice since there are an even number of slices, I mean is RAIDZ1 even possible with 4 slices? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Server with 4 drives, how to configure ZFS?
Hello! I don't see the problem. Install the OS onto a mirrored partition, and configure all the remaining storage however you like - raid or mirror or watever. I didn't understand your point of view until I read the next paragraph. My personal preference, assuming 4 disks, since the OS is mostly reads and only a little bit of writes, is to create a 4-way mirrored 100G partition for the OS, and the remaining 900G of each disk (or whatever) becomes either a stripe of mirrors or raidz, as appropriate in your case, for the storagepool. Oh, you are talking about 1T drives and my servers are all 4x73G! So it's a fairly big deal since I have little storage to waste and still want to be able to survive losing one drive. I should have given the numbers at the beginning, sorry. Given this meager storage do you have any suggestions? Thank you. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Server with 4 drives, how to configure ZFS?
Hello Jim! I understood ZFS doesn't like slices but from your reply maybe I should reconsider. I have a few older servers with 4 bays x 73G. If I make a root mirror pool and swap on the other 2 as you suggest, then I would have about 63G x 4 left over. If so then I am back to wondering what to do about 4 drives. Is raidz1 worthwhile in this scenario? That is less redundancy that a mirror and much less than a 3 way mirror, isn't it? Is it even possible to do raidz2 on 4 slices? Or would 2, 2 way mirrors be better? I don't understand what RAID10 is, is it simply a stripe of two mirrors? Or would it be best to do a 3 way mirror and a hot spare? I would like to be able to tolerate losing one drive without loss of integrity. I will be doing new installs of Solaris 10. Is there an option in the installer for me to issue ZFS commands and set up pools or do I need to format the disks before installing and if so how do I do that? Thank you. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Is another drive worth anything? [Summary]
Many thanks to all who responded. I learned a lot from this thread! For now I have decided to make a 3 way mirror because of the read performance. I don't want to take a risk on an unmirrored drive. Instead of replying to everyone separately I am following the Sun Managers system since I read that newsgroup occasionalliy also. Here's a summary of the responses. Jim Klimov wrote: Well, you can use this drive as a separate scratch area, as a separate single-disk pool, without redundancy. You'd have a separate spindle for some dedicated tasks with data you're okay with losing. I thought about that and I really don't like losing data. I also don't generate much temporary data so I love ZFS because it makes mirroring easy. On my other systems where I don't have ZFS I run hourly backups from drive to drive. Consumer drives are pretty good these days but you never know when one will fail. I had a failure recently on a Linux box and although I didn't lose data because I back up hourly it's still annoying to deal with. If I hadn't had another good drive with that data on it I would have lost critical data. You can also make the rpool a three-way mirror which may increase read speeds if you have enough concurrentcy. And when one drive breaks, your rpool is still mirrored. I think that's the best suggestion. I didn't realize a 3 way mirror would help performance but you and several others said it does, so that's what I will do. Thanks for the suggestions, Jim. Roy pointed out a theoretical 50% read increase when adding the third drive. Thanks Roy! Edward Ned Harvey wrote: In my benchmarking, I found 2-way mirror reads 1.97x the speed of a single disk, and a 3-way mirror reads 2.91x a single disk. Always great having hard data to base a decision on! That helped me make my decision! Thanks Edward! Jim Klimov answered a question that came up based on comments that read performance was improved in a three way mirror: Writes in a mirror are deemed to be not faster than the slowest disk - all two or three drives must commit a block before it is considered written (in sync write mode), likewise for TXG sync but with some optimization by caching and write-coalescing. Thanks Jim! Good to know. Edward Ned Harvey pointed out If you make it a 3-way mirror, your write performance will be unaffected, but your read performance will increase 50% over a 2-way mirror. All 3 drives can read different data simultaneously for the net effect of 3x a single disk read performance. Bob clarified the theoretical benefit of adding a third drive to a mirror by saying I think that a read performance increase of (at most) 33.3% is more correct. You might obtain (at most) 50% over one disk by mirroring it. Zfs makes a random selection of which disk to read from in a mirror set so the improvement is not truely linear. Thanks guys, that makes sense. Daniel Carosone suggested keeping the extra drive around in case of a failure and in the meantime using an SSD in the 3rd SATA slot. He pointed out a few other options that could help with performance besides creating a 3 way mirror when he wrote: Namely, leave the third drive on the shelf as a cold spare, and use the third sata connector for an ssd, as L2ARC, ZIL or even possibly both (which will affect selection of which device to use). That's not an option for me right now but I am planning to revisit SSD again when the consumer drives are reliable enough and don't have wear issues. Right now overall integrity and long service life are more important than absolute performance on this box, although since I have the integrity with the ZFS mirror I could add an SSD but I really don't want to deal with another failure as long as I don't have to. I do want additional performance if I can afford it, but not at the expense of possible data loss. Daniel also wrote: L2ARC is likely to improve read latency (on average) even more than a third submirror. ZIL will be unmirrored, but may improve writes at an acceptable risk for development system. If this risk is acceptable, you may wish to consider whether setting sync=disabled is also acceptable at least for certain datasets. I don't know what L2ARC is, but I'll take a look on the net. I did hear about ZIL but don't understand it fully, but I figured spending 500G on ZIL would be unwise. By that I mean I understand ZIL doesn't require much storage but if I don't have an identical drive I can't add a drive or slice with less storage than the other drives in a mirror to that mirror, so I would be forced to waste a lot of storage to implement ZIL. Finally, if you're considering spending money, can you increase the RAM instead? If so, do that first. This mobo is maxed out at 4G, it's a socket 775 I bought a couple of years ago. I have always seen the benefits to more RAM and I agree with you it helps more than people generally believe. Next time I buy a new box I am hoping to go with 8 to 16G although on