Re: [zfs-discuss] raidz recovery
On Wed, Dec 15, 2010 at 3:29 PM, Gareth de Vaux z...@lordcow.org wrote: On Mon 2010-12-13 (16:41), Marion Hakanson wrote: After you clear the errors, do another scrub before trying anything else. Once you get a complete scrub with no new errors (and no checksum errors), you should be confident that the damaged drive has been fully re-integrated into the pool. Ok I did a scrub after zero'ing, and the array came back clean, apparently, but same final result - the array faults as soon as I 'offline' a different vdev. The zero'ing is just a pretend-the-errors-aren't-there directive, and the scrub seems to be listening to that. What I need in this situation is a way to prompt ad6 to resilver from scratch. I think scrub doesn't replace all superblocks or other stuff not in the active dataset but rather some drive labels. have you tried zpool replace? like remove ad6, fill with zeroes, replace, command zpool replace tank ad6. That should simulate drive failure and replacement with a new disk. -- - Tuomas ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Mixing different disk sizes in a pool?
Another question: all those disks are on Dell MD1000 JBODs (11 of them) and we have 12 SAS ports on three LSI 9200-16e HBAs. Is there any point connecting each JBOD on a separate port or is it ok cascading them in groups of three? Is there a bandwidth limit we'll be hitting doing that? Thanks -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] A few questions
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Alexander Lesle at Dezember, 17 2010, 17:48 Lanky Doodle wrote in [1]: By single drive mirrors, I assume, in a 14 disk setup, you mean 7 sets of 2 disk mirrors - I am thinking of traditional RAID1 here. Or do you mean 1 massive mirror with all 14 disks? Edward means a set of two-way-mirrors. Correct. mirror disk0 disk1 mirror disk2 disk3 mirror disk4 disk5 ... You would normally call this a stripe of mirrors. Even though the ZFS concept of striping is more advanced than traditional raid striping... We still call this a ZFS stripe for lack of any other term. A ZFS stripe has all the good characteristics of raid concatenation and striping, without any of the bad characteristics. It can utilize bandwidth on multiple disks when it wants to, or use a single device when it wants to for small blocks. It can dynamically add randomly sized devices, and it can be done one-at-a-time. Gaining everything good of traditional raid stripe or concatenation, without any of the negatives of traditional raid stripe and concatenation. At Sol11 Express Oracle announced that at TestInstall you can set RootPool to mirror during installation. At the moment I try it out in a VM but I didnt find this option. :-( Actually, even in solaris 10, I habitually install the root filesystem onto a ZFS mirror. You just select 2 disks, and it's automatically a mirror. zpool create lankyserver mirror vdev1 vdev2 mirror vdev3 vdev4 When you need more space you can expand a bundle of two disks to your lankyserver. Each pair with the same capacity is effective. zpool add lankyserver mirror vdev5 vdev6 mirror vdev7 vdev8 ... Correct. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] A few questions
From: Bob Friesenhahn [mailto:bfrie...@simple.dallas.tx.us] Sent: Friday, December 17, 2010 9:16 PM While I agree that smaller vdevs are more reliable, your statement about the failure being more likely be in the same vdev if you have only 2 vdev's to be a rather useless statement. The probability of vdev failure does not have anything to do with the number of vdevs. However, the probability of vdev failure increases tremendously if there is only one vdev and there is a second disk failure. I'm not sure you got what I meant. I'll rephrase and see if it's more clear: Correct, the number of vdev's doesn't affect the probability of a failure in a specific vdev, but the number of disks in a vdev does. Lanky said he was considering 2x7disk raidz, versus 3x5disk raidz. So when I said he's more likely to have a 2nd disk fail in the same vdev if he only has 2 vdev's ... That was meant to be taken in context, not as a generalization about pools in general. Consider a single disk. Let P be the probability of the disk failing, within 1 day. If you have 5 disks in a raidz vdev, and one fails, there are 4 remaining. If resilver will last 8 days, then the probability of a 2nd disk failing is 4*8*P = 32P If you have 7 disks in a raidz vdev, and one fails, there are 6 remaining. If a resilver will last 12 days, then the probability of a 2nd disk failing is 6*12*P = 72P ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] copy complete zpool via zfs send/recv
Am 18.12.10 05:44, schrieb Edward Ned Harvey: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Stephan Budach Now, I want to use zfs send -R t...@movetank | zfs recv targetTank/... which would place all zfs fs one level down below targetTank. Overwriting targetTank is not an option, since the zfs fs musn't exist prior to zfs recv. You lost me in that one. If there is a zfs filesystem at the recipient... and you don't want to receive into a subdirectory of it ... then you have to overwrite it. And I don't see any reason not to overwrite it. You say the destination filesystem mustn't exist prior to receive... But did you know of the -F option? It forces the overwrite, if you want to receive and overwrite an existing filesystem. What am I missing? Or did I just answer it? Ehh... well... you answered it... sort of. ;) I think I simply didn't dare to overwrite the root zfs on the destination zpool with -F, but of course you're right, that this is the way to go. Thanks, budy ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] a single nfs file system shared out twice with different permissions
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Geoff Nordli I am trying to configure a system where I have two different NFS shares which point to the same directory. The idea is if you come in via one path, you will have read-only access and can't delete any files, if you come in the 2nd path, then you will have read/write access. I think you can do this client-side. mkdir /foo1 mkdir /foo2 mount nfsserver:/exports/bar /foo1 mount -o ro nfsserver:/exports/bar /foo2 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] copy complete zpool via zfs send/recv
From: Stephan Budach [mailto:stephan.bud...@jvm.de] Ehh. well. you answered it. sort of. ;) I think I simply didn't dare to overwrite the root zfs on the destination zpool with -F, but of course you're right, that this is the way to go. What are you calling the root zfs on the destination? You're not trying to overwrite / are you? That would ... admittedly ... not be so straightforward. But I don't think it's impossible. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] copy complete zpool via zfs send/recv
Am 18.12.10 15:14, schrieb Edward Ned Harvey: From: Stephan Budach [mailto:stephan.bud...@jvm.de] Ehh. well. you answered it. sort of. ;) I think I simply didn't dare to overwrite the root zfs on the destination zpool with -F, but of course you're right, that this is the way to go. What are you calling the root zfs on the destination? You're not trying to overwrite / are you? That would ... admittedly ... not be so straightforward. But I don't think it's impossible. The root zfs, to me, is the fs that gets created once you create the zpool. So, if I create the zpool tank, I also get the zfs fs tank, no? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] copy complete zpool via zfs send/recv
2010/12/18 Stephan Budach stephan.bud...@jvm.de Am 18.12.10 15:14, schrieb Edward Ned Harvey: From: Stephan Budach [mailto:stephan.bud...@jvm.de stephan.bud...@jvm.de] Ehh. well. you answered it. sort of. ;) I think I simply didn't dare to overwrite the root zfs on the destination zpool with -F, but of course you're right, that this is the way to go. What are you calling the root zfs on the destination? You're not trying to overwrite / are you? That would ... admittedly ... not be so straightforward. But I don't think it's impossible. The root zfs, to me, is the fs that gets created once you create the zpool. So, if I create the zpool tank, I also get the zfs fs tank, no? Yes, but zfs receive can put received data only to another pool. You cannot zfs receive to RAW disk ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] raidz recovery
On Sat 2010-12-18 (14:55), Tuomas Leikola wrote: have you tried zpool replace? like remove ad6, fill with zeroes, replace, command zpool replace tank ad6. That should simulate drive failure and replacement with a new disk. 'replace' requires a different disk to replace with. How do you remove ad6? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] A few questions
On the subject of where to install ZFS, I was planning to use either Compact Flash or USB drive (both of which would be mounted internally); using up 2 of the drive bays for a mirrored install is possibly a waste of physical space, considering it's a) a home media server and b) the config can be backed up to a protected ZFS pool - if the CF or USB drive failed I would just replace and restore the config. Can you have an equivalent of a global hot spare in ZFS. If I did go down the mirror route (mirror disk0 disk1 mirror disk2 disk3 mirror disk4 disk5 etc) all the way up to 14 disks that would leave the 15th disk spare. Now this is getting really complex, but can you have server failover in ZFS, much like DFS-R in Windows - you point clients to a clustered ZFS namespace so if a complete server failed nothing is interrupted. I am still undecided as to mirror vs RAID Z. I am going to be ripping uncompressed Blu-Rays so space is vital. I use RAID DP in NetApp kit at work and I'm guessing RAID Z2 is the equivalent? I have 5TB space at the moment so going to the expense of mirroring for only 2TB extra doesn't seem much of a pay off. Maybe a compromise of 2x 7-disk RAID Z1 with global hotspare is the way to go? Put it this way, I currently use Windows Home Server, which has no true disk failure protection, so any of ZFS's redundancy schemes is going to be a step up; is there an equivalent system in ZFS where if 1 disk fails you only lose that disks data, like unRAID? Thanks everyone for your input so far :) -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Mixing different disk sizes in a pool?
On Sat, Dec 18, 2010 at 7:26 AM, Ian D rewar...@hotmail.com wrote: Another question: all those disks are on Dell MD1000 JBODs (11 of them) and we have 12 SAS ports on three LSI 9200-16e HBAs. Is there any point connecting each JBOD on a separate port or is it ok cascading them in groups of three? Is there a bandwidth limit we'll be hitting doing that? Thanks It's fine to cascade them. SAS is all point-to-point. I strongly doubt you'll hit a bandwidth constraint on the backend, especially if you have the shelves multipathed, but if that's a concern you will get more peak bandwidth putting them on separate ports. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] a single nfs file system shared out twice with different permissions
-Original Message- From: Edward Ned Harvey [mailto:opensolarisisdeadlongliveopensola...@nedharvey.com] Sent: Saturday, December 18, 2010 6:13 AM To: 'Geoff Nordli'; zfs-discuss@opensolaris.org Subject: RE: [zfs-discuss] a single nfs file system shared out twice with different permissions From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Geoff Nordli I am trying to configure a system where I have two different NFS shares which point to the same directory. The idea is if you come in via one path, you will have read-only access and can't delete any files, if you come in the 2nd path, then you will have read/write access. I think you can do this client-side. mkdir /foo1 mkdir /foo2 mount nfsserver:/exports/bar /foo1 mount -o ro nfsserver:/exports/bar /foo2 Thanks Edward. The client side solution works great. Happy holidays!! Geoff ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Mixing different disk sizes in a pool?
On 12/18/10 10:49 AM, Ian D wrote: I have 159x 15K RPM SAS drives I want to build a ZFS appliance with. 75x 145G 60x 300G 24x 600G The box has 4 CPUs, 256G of RAM, 14x 100G SLC SSDs for the cache and a mirrored pair of 4G DDRDrive X1s for the SLOG. My plan is to mirror all these drives and keep some hot spares. My question is: should I create three pools (one for each size of drives) and share the cache and slog among them or should I create a single pool with them all? The answer really depends on what you want to do with pool(s). You'll have to provide more information. -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS ... open source moving forward?
On 12/16/10 10:24 AM -0500 Linder, Doug wrote: Tim Cook wrote: Claiming you'd start paying for Solaris if they gave you ZFS for free in Linux is absolutely ridiculous. *Start* paying? You clearly have NO idea what it costs to run Solaris in a production environment with support. In my experience, it's less than RedHat. Also TCO is less since Solaris offers more to begin with. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS ... open source moving forward?
On 12/16/10 9:11 AM -0500 Linder, Doug wrote: The only thing I'll add is that I, as I said, I really don't care at all about licenses. Then you have no room to complain or even suggest a specific license! When it comes to licenses, to me (and, I suspect, the vast majority of other OSS users), GPL is synonymous with open source. Is that correct? No. Am I aware that plenty of other licenses exist? Yes. Is the issue important? Sure. Agreed. Do I have time or interest to worry about niggly little details? No. Well the problem with licenses is that they are decidedly NOT niggly little details. You should consider re-evaluating what you have time or interest for, if you care about the things you say (such as maximum and flexible use of the products you are using). All I want is to be able to use the best technology in the ways that are most useful to me without artificial restrictions. Anything that advances that, I'm for. CDDL is close to that, much closer than GPL. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS ... open source moving forward?
On 12/16/10 11:32 AM +0100 Joerg Schilling wrote: Note that while there existist numerous papers from lawyers that consistently explain which parts of the GPLv2 are violating US law and thus are void, Can you elaborate? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS ... open source moving forward?
Frank Cusack frank+lists/z...@linetwo.net wrote: On 12/16/10 11:32 AM +0100 Joerg Schilling wrote: Note that while there existist numerous papers from lawyers that consistently explain which parts of the GPLv2 are violating US law and thus are void, Can you elaborate? See: http://www.osscc.net/en/gpl.html for a list. e.g. the papers from Lawrence Rosen, Tom Gordon and Lothar Determan. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Mixing different disk sizes in a pool?
The answer really depends on what you want to do with pool(s). You'll have to provide more information. Get the maximum of very random IOPS I get can out of those drives for database usage. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Mixing different disk sizes in a pool?
On Sat, Dec 18, 2010 at 4:24 PM, Ian D rewar...@hotmail.com wrote: The answer really depends on what you want to do with pool(s). You'll have to provide more information. Get the maximum of very random IOPS I get can out of those drives for database usage. -- Random IOPS won't max out the SAS link. You'll be fine stacking them. But again, if you have the ports available, and already have the cables, it won't hurt anything to use them. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] A few questions
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Lanky Doodle On the subject of where to install ZFS, I was planning to use either Compact Flash or USB drive (both of which would be mounted internally); using up 2 of the drive bays for a mirrored install is possibly a waste of physical space, considering it's a) a home media server and b) the config can be backed up to a protected ZFS pool - if the CF or USB drive failed I would just replace and restore the config. All of the above is correct. One thing you should keep in mind however: If your unmirrored rpool (usb fob) fails... Although yes you can restore assuming you have been sufficiently backing it up ... You will suffer an ungraceful halt. Maybe you can live with that. Can you have an equivalent of a global hot spare in ZFS. If I did go down the mirror route (mirror disk0 disk1 mirror disk2 disk3 mirror disk4 disk5 etc) all the way up to 14 disks that would leave the 15th disk spare. Check the zpool man page for spare, but I know you can have spares assigned to a vdev, and I'm pretty sure you can assign any given spare to multiples, effectively making it a global hotspare. So yes is the answer. Now this is getting really complex, but can you have server failover in ZFS, much like DFS-R in Windows - you point clients to a clustered ZFS namespace so if a complete server failed nothing is interrupted. If that's somehow possible, it's something I don't know. I don't believe you can do that with ZFS. I am still undecided as to mirror vs RAID Z. I am going to be ripping uncompressed Blu-Rays so space is vital. For both read and write, raidz works extremely well for sequential operations. It sounds like you're probably going to be doing mostly sequential operations, so raidz should perform very well for you. A lot of people will avoid raidzN because it doesn't perform very well for random reads, so they opt for mirrors instead. But in your case, no so much. In your case, the only reason I can think to avoid raidz would be if you're worrying about resilver times. That's a valid concern, but you can linearly choose any number of disks you want ... You could make raidz using 3-disks each... It's just a compromise between the mirror and the larger raidz vdev. I use RAID DP in NetApp kit at work and I'm guessing RAID Z2 is the equivalent? Yup, raid-dp and raidz2 are conceptually pretty much the same. Put it this way, I currently use Windows Home Server, which has no true disk failure protection, so any of ZFS's redundancy schemes is going to be a step up; is there an equivalent system in ZFS where if 1 disk fails you only lose that disks data, like unRAID? No. Not unless you make that many separate volumes. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS ... open source moving forward?
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Frank Cusack Claiming you'd start paying for Solaris if they gave you ZFS for free in Linux is absolutely ridiculous. *Start* paying? You clearly have NO idea what it costs to run Solaris in a production environment with support. In my experience, it's less than RedHat. Also TCO is less since Solaris offers more to begin with. Guys... The discussion of whether or not ZFS is open source moving forward has long since been concluded. Of course, please feel free to discuss anything you like, but maybe you want to start a new thread to argue about whether solaris is better than redhat, or GPL is legally significant and blah blah blah, and so forth? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
