On Sat, Feb 21, 2009 at 7:17 PM, Jeremy Chadwick <[email protected]> wrote: > On Sun, Feb 22, 2009 at 11:59:57AM +1100, Dmitri Nikulin wrote: >> On Sun, Feb 22, 2009 at 10:34 AM, Bill Hacker <[email protected]> wrote: >> > Hopefully more 'good stuff' will be ported out of Solaris before it hits >> > the >> > 'too costly vs the alternatives' wall and is orphaned. >> >> Btrfs has been merged into mainline Linux now, and although it's >> pretty far behind ZFS in completeness at the moment, it represents a >> far greater degree of flexibility and power. In a couple of years when >> it's stable and user friendly, high-end storage solutions will move >> back to Linux, after having given Sun a lot of contracts due >> specifically to ZFS. > > The fact that btrfs offers grow/shrink capability puts it ahead of ZFS > with regards to home users who desire a NAS. I can't stress this point > enough. ZFS's lack of this capability limits its scope. As it stands > now, if you replace a disk with a larger one, you have to go through > this extremely fun process to make use of the new space available: > > - Offload all of your data somewhere (read: not "zfs export"); rsync > is usually what people end up using -- if you have multiple ZFS > filesystems, this can take some time > - zpool destroy > - zpool create > - zfs create
According to the ZFS Admin manual, doing an online "replace" of the drive with a larger one, than a zpool export and zpool import is all that's needed to make use of the extra space. In theory, one can replace all the drives in the storage array one at a time, allowing the resilvers to complete each time, and then just export/import once, and have a massively larger pool to use. > And if you add a new disk to the system, it's impossible to add that > disk to the existing pool -- you can, of course, create an entirely > new zpool which uses that disk, but that has nothing to do with the > existing zpool. So you get to do the above dance. You can add vdevs to a pool at any time. Data will be striped across all vdevs in the pool. What you can't do is extend a raidz vdev. But you can add more raidz vdevs to a pool. If you create a pool with a 3-drive raidz vdev, you can later extend the pool by adding another 3-drive raidz vdev. Or by adding a mirrored vdev. Or by just adding a single drive (although then you lose the redundancy of the entire pool). I've done this several times when playing around with ZFS on FreeBSD 7.1 on a test system with 24-drives. Started with a 12-drive raidz2 vdev. Then addded a 6-drive raidz2. Then another 6-drive raidz2. Then played around with 3x 8-drive raidz2 vdevs. And a bunch of other setups, just to see what the limitations were. The only one is that you can't start with an X-drive raidz2 and later extend that single raidz2 vdev out to Y-drives, like you can with some hardware RAID controllers. > I'll also point out that ZFS on FreeBSD (at least 7.x) performs very > differently than on Solaris 10. We use Solaris 10 x86 + ZFS at my > workplace, and the overall usability of the system during heavy disk I/O > is much more refined (read: smooth) than on FreeBSD. It's interesting > to do something like "zpool iostat 1" on FreeBSD compared to Solaris 10; > FreeBSD will show massive write bursts (e.g. 0MB, 0MB, 0MB, 70MB, 0MB, > 0MB, 0MB, 67MB, etc.), while Solaris behaves more appropriately (50MB, > 60MB, 70MB, 40MB, etc.). "zpool scrub" is a great way to test this. Hrm, we haven't run into that, but we're mostly limited by network speeds in our setup. "zpool iostat" shows a fairly constant 2 MB read or write to each of the 24-drives in the servers. But that's all rsync usage, and limited by the ADSL links we use. -- Freddie Cash [email protected]
