Re: [zfs-discuss] Mount External USB cdrom on zfs
iman habibi wrote: Dear support when i connect my external usb dvdrom to the sparc machine which has installed solaris 10u6 based zfs file system,,it return this error: Your ZFS question is? DVDs uue the HSFS filesystem. One good place for general Solaris questions is comp.unix.solaris. -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Mount External USB cdrom on zfs
On Tue, Jan 27, 2009 at 9:49 AM, iman habibi iman.hab...@gmail.com wrote: Dear support when i connect my external usb dvdrom to the sparc machine which has installed solaris 10u6 based zfs file system,,it return this error: bash-3.00# mount /dev/dsk/c1t0d0s0 /dvd/ Jan 27 11:08:41 global ufs: NOTICE: mount: not a UFS magic number (0x0) mount: /dev/dsk/c1t0d0s0 is not this fstype On Solaris, by default mount assumes that the file system type to be mounted is UFS. Basically, when mounting anything other than UFS, you need to specify what it is. The two exceptions are: a) When the vfstab can give information about what file system type to expect, or b) When using zfs mount (which only mounts zfs file systems) So essentially you need to specify the file system type on the mount command, like this: mount -F hsfs -r /dev/dsk/c1t0d0s0 /dvd/ The -r is for read-only. You can also (optionally) add a line to your /etc/vfstab file, like this: /dev/dsk/c1t0d0s0 - /dvd hsfs - no ro With this in place you can then mount the disk using: mount /dvd (It will learn the device, read-only flag, and the file system type from /etc/vfstab automatically) Of course I am wondering why you don't use the auto-mounter. There are of course other things you could do. You could change the default file system type in /etc/default/fs, but that is not recommended. You could write a little script to mount disks. etc etc etc. For more info, read man mount and man vfstab Cheers _Johan -- Any sufficiently advanced technology is indistinguishable from magic. Arthur C. Clarke My blog: http://initialprogramload.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] freeze/ thaw zfs file system
Hi All, Can we freeze and thaw a zfs file-system either from user land (lockfs for ufs) or from the kernel space or through ioctl? Thanks in advance for your help. Best regards, ajit ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] freeze/ thaw zfs file system
ajit jain wrote: Hi All, Can we freeze and thaw a zfs file-system either from user land (lockfs for ufs) or from the kernel space or through ioctl? Can you step back a level and explain what you're trying to achieve? Freezing UFS is to get round ufs-specific issues which don't apply to ZFS, so you maybe asking the wrong question. -- Andrew ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Problems using ZFS on Smart Array P400
I am having trouble getting ZFS to behave as I would expect. I am using the HP driver (cpqary3) for the Smart Array P400 (in a HP Proliant DL385 G2) with 10k 2.5 146GB SAS drives. The drives appear correctly, however due to the controller not offering JBOD functionality I had to configure each drive as a RAID0 logical drive. Everything appears to work fine, the drives are detected and I created a mirror for the OS to install to and an additional raidz2 array with the remaining 6 discs. But when I remove a disc and then reinsert it I cannot get ZFS to accept it back into the array see bellow for the details. I thought it might be a problem with using the whole discs eg: c1t*d0 so I created a single partition on each and used that, but had the same results. The module seem to detect the drive has been reinserted successfully but the OS doesn't seem to want to write to it. Any help would be most appreciated as I would much prefer to use ZFS's software capabilities rather than the hardware card in the machine. When rebooting the system the Array BIOS also displays some interesting behavior. ### BIOS Output 1792-Slot 1 Drive Array - Valid Data Found in the Array Accelerator Data will automatically be written to the drive array 1779-Slot 1 Drive Array - Replacement drive(s) detected OR previously failed drives(s) no appear to be operational POrt 2I: Box1: Bay3 Logical drives(s) disabled due to possible data loss. Select F1 to continue with logical drive(s) disabled Select F2 to accept data loss and to re-enable logical drive(s) Terminal output bash-3.00# zpool status test pool: test state: DEGRADED status: One or more devices could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Attach the missing device and online it using 'zpool online'. see: http://www.sun.com/msg/ZFS-8000-2Q scrub: resilver completed after 0h0m with 0 errors on Tue Jan 27 03:30:16 2009 config: NAME STATE READ WRITE CKSUM test DEGRADED 0 0 0 raidz2 DEGRADED 0 0 0 c1t2d0p0 ONLINE 0 0 0 c1t3d0p0 ONLINE 0 0 0 c1t4d0p0 ONLINE 0 0 0 c1t5d0p0 UNAVAIL 0 0 0 cannot open c1t6d0p0 ONLINE 0 0 0 c1t8d0p0 ONLINE 0 0 0 errors: No known data errors bash-3.00# zpool online test c1t5d0p0 warning: device 'c1t5d0p0' onlined, but remains in faulted state use 'zpool replace' to replace devices that are no longer present bash-3.00# dmesg Jan 27 03:27:40 unknown cpqary3: [ID 823470 kern.notice] NOTICE: Smart Array P400 Controller Jan 27 03:27:40 unknown cpqary3: [ID 823470 kern.notice] Hot-plug drive inserted, Port: 2I Box: 1 Bay: 3 Jan 27 03:27:40 unknown cpqary3: [ID 479030 kern.notice] Configured Drive ? ... YES Jan 27 03:27:40 unknown cpqary3: [ID 10 kern.notice] Jan 27 03:27:40 unknown cpqary3: [ID 823470 kern.notice] NOTICE: Smart Array P400 Controller Jan 27 03:27:40 unknown cpqary3: [ID 834734 kern.notice] Media exchange detected, logical drive 6 Jan 27 03:27:40 unknown cpqary3: [ID 10 kern.notice] ... Jan 27 03:36:24 unknown scsi: [ID 107833 kern.warning] WARNING: /p...@38,0/pci1166,1...@10/pci103c,3...@0/s...@5,0 (sd6): Jan 27 03:36:24 unknown SYNCHRONIZE CACHE command failed (5) ... Jan 27 03:47:58 unknown scsi: [ID 107833 kern.warning] WARNING: /p...@38,0/pci1166,1...@10/pci103c,3...@0/s...@5,0 (sd6): Jan 27 03:47:58 unknown drive offline -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Problems using ZFS on Smart Array P400
On Tue, Jan 27, 2009 at 7:16 PM, Alex a...@pancentric.com wrote: I am using the HP driver (cpqary3) for the Smart Array P400 (in a HP Proliant DL385 G2) with 10k 2.5 146GB SAS drives. The drives appear correctly, however due to the controller not offering JBOD functionality I had to configure each drive as a RAID0 logical drive. Ouch. Short comment, it might not worth it. Seriously. Does the P400 have a battery-backed cache? If yes, it will be MUCH easier to simply let it handle RAID5/10 and use stripe config for zfs. HW controllers with battery-backed cache will reduce the possibility of raid5 write hole. Depending on what your goal is, it might be the best choice. But when I remove a disc and then reinsert it I cannot get ZFS to accept it back into the array see bellow for the details. I don't think it's zfs' fault. Any help would be most appreciated as I would much prefer to use ZFS's software capabilities rather than the hardware card in the machine. Yet you're stuck with a hardware that practically does not allow you to do just that. 1779-Slot 1 Drive Array - Replacement drive(s) detected OR previously failed drives(s) no appear to be operational POrt 2I: Box1: Bay3 Logical drives(s) disabled due to possible data loss. Select F1 to continue with logical drive(s) disabled Select F2 to accept data loss and to re-enable logical drive(s) From this output, the HW controller refuses to enable the removed disc, so that the OS (and thus zfs) can't see it yet. You can enable it from BIOS during booting (which kinda defeat the whole hot-swap thing), or you might be able to find a way (using ILO perhaps, or with some HP-supplied software installed on the OS that can talk to P400) to enable it without having to reboot. Regards, Fajar ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Problems using ZFS on Smart Array P400
I'm testing the same thing on a DL380 G5 with P400 controller. I set individual RAID 0 logical drives for each disk. I ended up with the same result upon drive removal. I'm looking into whether the hpacucli array command line utility will let me re-enable a logical drive from its interface. -- Edmund William White ewwh...@mac.com From: Alex a...@pancentric.com Date: Tue, 27 Jan 2009 04:16:56 -0800 (PST) To: zfs-discuss@opensolaris.org Subject: [zfs-discuss] Problems using ZFS on Smart Array P400 I am having trouble getting ZFS to behave as I would expect. I am using the HP driver (cpqary3) for the Smart Array P400 (in a HP Proliant DL385 G2) with 10k 2.5 146GB SAS drives. The drives appear correctly, however due to the controller not offering JBOD functionality I had to configure each drive as a RAID0 logical drive. Everything appears to work fine, the drives are detected and I created a mirror for the OS to install to and an additional raidz2 array with the remaining 6 discs. But when I remove a disc and then reinsert it I cannot get ZFS to accept it back into the array see bellow for the details. I thought it might be a problem with using the whole discs eg: c1t*d0 so I created a single partition on each and used that, but had the same results. The module seem to detect the drive has been reinserted successfully but the OS doesn't seem to want to write to it. Any help would be most appreciated as I would much prefer to use ZFS's software capabilities rather than the hardware card in the machine. When rebooting the system the Array BIOS also displays some interesting behavior. ### BIOS Output 1792-Slot 1 Drive Array - Valid Data Found in the Array Accelerator Data will automatically be written to the drive array 1779-Slot 1 Drive Array - Replacement drive(s) detected OR previously failed drives(s) no appear to be operational POrt 2I: Box1: Bay3 Logical drives(s) disabled due to possible data loss. Select F1 to continue with logical drive(s) disabled Select F2 to accept data loss and to re-enable logical drive(s) Terminal output bash-3.00# zpool status test pool: test state: DEGRADED status: One or more devices could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Attach the missing device and online it using 'zpool online'. see: http://www.sun.com/msg/ZFS-8000-2Q scrub: resilver completed after 0h0m with 0 errors on Tue Jan 27 03:30:16 2009 config: NAME STATE READ WRITE CKSUM test DEGRADED 0 0 0 raidz2 DEGRADED 0 0 0 c1t2d0p0 ONLINE 0 0 0 c1t3d0p0 ONLINE 0 0 0 c1t4d0p0 ONLINE 0 0 0 c1t5d0p0 UNAVAIL 0 0 0 cannot open c1t6d0p0 ONLINE 0 0 0 c1t8d0p0 ONLINE 0 0 0 errors: No known data errors bash-3.00# zpool online test c1t5d0p0 warning: device 'c1t5d0p0' onlined, but remains in faulted state use 'zpool replace' to replace devices that are no longer present bash-3.00# dmesg Jan 27 03:27:40 unknown cpqary3: [ID 823470 kern.notice] NOTICE: Smart Array P400 Controller Jan 27 03:27:40 unknown cpqary3: [ID 823470 kern.notice] Hot-plug drive inserted, Port: 2I Box: 1 Bay: 3 Jan 27 03:27:40 unknown cpqary3: [ID 479030 kern.notice] Configured Drive ? ... YES Jan 27 03:27:40 unknown cpqary3: [ID 10 kern.notice] Jan 27 03:27:40 unknown cpqary3: [ID 823470 kern.notice] NOTICE: Smart Array P400 Controller Jan 27 03:27:40 unknown cpqary3: [ID 834734 kern.notice] Media exchange detected, logical drive 6 Jan 27 03:27:40 unknown cpqary3: [ID 10 kern.notice] ... Jan 27 03:36:24 unknown scsi: [ID 107833 kern.warning] WARNING: /p...@38,0/pci1166,1...@10/pci103c,3...@0/s...@5,0 (sd6): Jan 27 03:36:24 unknown SYNCHRONIZE CACHE command failed (5) ... Jan 27 03:47:58 unknown scsi: [ID 107833 kern.warning] WARNING: /p...@38,0/pci1166,1...@10/pci103c,3...@0/s...@5,0 (sd6): Jan 27 03:47:58 unknown drive offline -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool status -x strangeness
I forgot the pool that's having problems was recreated recently so it's already at zfs version 3. I just did a 'zfs upgrade -a' for another pool, but some of those filesystems failed since they are busy and couldn't be unmounted. # zfs upgrade -a cannot unmount '/var/mysql': Device busy cannot unmount '/var/postfix': Device busy 6 filesystems upgraded 821 filesystems already at this version Ben You can upgrade live. 'zfs upgrade' with no arguments shows you the zfs version status of filesystems present without upgrading. On Jan 24, 2009, at 10:19 AM, Ben Miller mil...@eecis.udel.edu wrote: We haven't done 'zfs upgrade ...' any. I'll give that a try the next time the system can be taken down. Ben A little gotcha that I found in my 10u6 update process was that 'zpool upgrade [poolname]' is not the same as 'zfs upgrade [poolname]/[filesystem(s)]' What does 'zfs upgrade' say? I'm not saying this is the source of your problem, but it's a detail that seemed to affect stability for me. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] freeze/ thaw zfs file system
Hi Andrew, I am writing a filtering device which tracks the write to the file-system. I am doing it for ufs, vxfs and for zfs. Sometime for consistent point I need to freeze the file-system which flushes dirty block to the disk and block every IO on the top level. So, for ufs and vxfs I got lockfs and VX_FREEZE/VX_THAW, but with zfs I didn't get the luck. Thanks for your response. Best regards, ajit On Tue, Jan 27, 2009 at 5:34 PM, Andrew Gabriel agabr...@opensolaris.org wrote: ajit jain wrote: Hi All, Can we freeze and thaw a zfs file-system either from user land (lockfs for ufs) or from the kernel space or through ioctl? Can you step back a level and explain what you're trying to achieve? Freezing UFS is to get round ufs-specific issues which don't apply to ZFS, so you maybe asking the wrong question. -- Andrew ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Problems using ZFS on Smart Array P400
You need to step back and appreciate that the manner in which you are presenting Solaris with disks is the problem and not necessarily ZFS. As your storage system is incapable of JBOD operation, you have decided to present each disk as a 'simple' RAID0 volume. Whilst this looks like a 'pass-thru' access method to the disk and its contents, it is far from it. The HW RAID sub-system is creating a logical volume based on this single spindle (in exactly the same way it would be for multiple spindles, aka a stripe), metadata is recorded by the RAID system with regard to the make-up of said volume. The important issue here is that you have a non-redundant RAID (!) config, hence a single failure (in this case your single spindle failure) causes the RAID sub-system to declare the volume (and hence its operational status) as failed, this in turn is declared to the OS as a failed volume. At this juncture, intervention is normally necessary to re-destroy/re-create a volume (remember no redundancy--- so this is manual!) and hence re-present it to the OS (which will find a new UID for the volume and treat it as a new device). On occasions it may be possible to intervene and resurrect a volume by manually overriding the status of the RAID0 volume, but in many HW RAID systems this is not to be recommended. In short, you've got more abstractions (layers) in place than you need/ desire and that is fundamentally the cause of your problem ... either plump for a simpler array or swallow some loss of transparency in the ZFS layer and present redundant RAID sets from your array, but live with the consequences of increased admin and complexity and some loss of transparency/protection---but hopefully the RAID sub-system will be capable of automated recovery in most circumstances of simple failures. Craig On 27 Jan 2009, at 13:00, Edmund White wrote: I'm testing the same thing on a DL380 G5 with P400 controller. I set individual RAID 0 logical drives for each disk. I ended up with the same result upon drive removal. I'm looking into whether the hpacucli array command line utility will let me re-enable a logical drive from its interface. -- Edmund William White ewwh...@mac.com From: Alex a...@pancentric.com Date: Tue, 27 Jan 2009 04:16:56 -0800 (PST) To: zfs-discuss@opensolaris.org Subject: [zfs-discuss] Problems using ZFS on Smart Array P400 I am having trouble getting ZFS to behave as I would expect. I am using the HP driver (cpqary3) for the Smart Array P400 (in a HP Proliant DL385 G2) with 10k 2.5 146GB SAS drives. The drives appear correctly, however due to the controller not offering JBOD functionality I had to configure each drive as a RAID0 logical drive. Everything appears to work fine, the drives are detected and I created a mirror for the OS to install to and an additional raidz2 array with the remaining 6 discs. But when I remove a disc and then reinsert it I cannot get ZFS to accept it back into the array see bellow for the details. I thought it might be a problem with using the whole discs eg: c1t*d0 so I created a single partition on each and used that, but had the same results. The module seem to detect the drive has been reinserted successfully but the OS doesn't seem to want to write to it. Any help would be most appreciated as I would much prefer to use ZFS's software capabilities rather than the hardware card in the machine. When rebooting the system the Array BIOS also displays some interesting behavior. ### BIOS Output 1792-Slot 1 Drive Array - Valid Data Found in the Array Accelerator Data will automatically be written to the drive array 1779-Slot 1 Drive Array - Replacement drive(s) detected OR previously failed drives(s) no appear to be operational POrt 2I: Box1: Bay3 Logical drives(s) disabled due to possible data loss. Select F1 to continue with logical drive(s) disabled Select F2 to accept data loss and to re-enable logical drive(s) Terminal output bash-3.00# zpool status test pool: test state: DEGRADED status: One or more devices could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Attach the missing device and online it using 'zpool online'. see: http://www.sun.com/msg/ZFS-8000-2Q scrub: resilver completed after 0h0m with 0 errors on Tue Jan 27 03:30:16 2009 config: NAME STATE READ WRITE CKSUM test DEGRADED 0 0 0 raidz2 DEGRADED 0 0 0 c1t2d0p0 ONLINE 0 0 0 c1t3d0p0 ONLINE 0 0 0 c1t4d0p0 ONLINE 0 0 0 c1t5d0p0 UNAVAIL 0 0 0 cannot open c1t6d0p0 ONLINE 0 0 0 c1t8d0p0 ONLINE 0 0 0 errors: No known data errors bash-3.00# zpool online test c1t5d0p0 warning: device 'c1t5d0p0' onlined, but remains in faulted state use 'zpool replace' to replace devices that are no longer present bash-3.00# dmesg Jan 27 03:27:40 unknown cpqary3: [ID 823470 kern.notice] NOTICE: Smart
[zfs-discuss] Crazy Problem with
Hi Folks, call me a lernen ;-) I got a crazy Problem with zpool list and the size of my pool: created zpool create raidz2 hdd1 hdd2 hdd3 - each hdd is about 1GB. zpool list shows me a size of 2.95GB - shouldn't this bis online 1GB? After creating a file about 500MB - Capacity is shown as 50 % - The right value? Is this a known bug / feature? THX Henri -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Crazy Problem with
Henri Meddox wrote: Hi Folks, call me a lernen ;-) I got a crazy Problem with zpool list and the size of my pool: created zpool create raidz2 hdd1 hdd2 hdd3 - each hdd is about 1GB. zpool list shows me a size of 2.95GB - shouldn't this bis online 1GB? After creating a file about 500MB - Capacity is shown as 50 % - The right value? Is this a known bug / feature? You're lucky. Ben just wrote about it :-) http://www.cuddletech.com/blog/pivot/entry.php?id=1013 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Crazy Problem with
Mika Borner wrote: You're lucky. Ben just wrote about it :-) http://www.cuddletech.com/blog/pivot/entry.php?id=1013 Oops, should have read your message completly :-) Anyway you can lernen something from it... ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Problems using ZFS on Smart Array P400
Given that I have lots of ProLiant equipment, are there any recommended controllers that would work in this situation? Is this an issue unique to the Smart Array controllers? If I do choose to use some level of hardware RAID on the existing Smart Array P400, what's the best way to use it with ZFS (assume 8 disks with an emphasis on capacity)? -- Edmund William White ewwh...@mac.com From: Craig Morgan craig.mor...@sun.com Date: Tue, 27 Jan 2009 13:54:46 + To: Edmund White ewwh...@mac.com Cc: Alex a...@pancentric.com, zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] Problems using ZFS on Smart Array P400 You need to step back and appreciate that the manner in which you are presenting Solaris with disks is the problem and not necessarily ZFS. As your storage system is incapable of JBOD operation, you have decided to present each disk as a 'simple' RAID0 volume. Whilst this looks like a 'pass-thru' access method to the disk and its contents, it is far from it. The HW RAID sub-system is creating a logical volume based on this single spindle (in exactly the same way it would be for multiple spindles, aka a stripe), metadata is recorded by the RAID system with regard to the make-up of said volume. The important issue here is that you have a non-redundant RAID (!) config, hence a single failure (in this case your single spindle failure) causes the RAID sub-system to declare the volume (and hence its operational status) as failed, this in turn is declared to the OS as a failed volume. At this juncture, intervention is normally necessary to re-destroy/re-create a volume (remember no redundancy--- so this is manual!) and hence re-present it to the OS (which will find a new UID for the volume and treat it as a new device). On occasions it may be possible to intervene and resurrect a volume by manually overriding the status of the RAID0 volume, but in many HW RAID systems this is not to be recommended. In short, you've got more abstractions (layers) in place than you need/ desire and that is fundamentally the cause of your problem ... either plump for a simpler array or swallow some loss of transparency in the ZFS layer and present redundant RAID sets from your array, but live with the consequences of increased admin and complexity and some loss of transparency/protection---but hopefully the RAID sub-system will be capable of automated recovery in most circumstances of simple failures. Craig On 27 Jan 2009, at 13:00, Edmund White wrote: -- Craig Craig Morgan t: +44 (0)791 338 3190 f: +44 (0)870 705 1726 e: craig.mor...@sun.com ~ NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ~ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] how to fix zpool with corrupted disk?
On 26-Jan-09, at 8:15 PM, Miles Nordin wrote: js == Jakov Sosic jso...@gmail.com writes: tt == Toby Thain t...@telegraphics.com.au writes: js Yes but that will do the complete resilvering, and I just want js to fix the corrupted blocks... :) tt What you are asking for is impossible, since ZFS cannot know tt which blocks are corrupted without actually checking them yeah of course you have to read every (occupied) block, but he's still not asking for something completely nonsensical. What if the good drive has a latent sector error in one of the blocks that hasn't been scribbled over on the bad drive? scrub could heal the error if not for the ``too many errors'' fault, while 'zpool replace' could not heal it. Yes he's asking for a scrub. I was just pointing out that resilver is something else entirely :) --Toby ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ? Changing storage pool serial number
Hi I took a look at the archives and I have seen a few threads about using array block level snapshots with ZFS and how we face the old issue that we used to see with logical volumes and unique IDs (quite correctly) stopping the same volume being presented twice to the same server. IHAC wanting to do this with a ZFS storage pool and an LSI array. All works well when they present the array snapshot to hosts other than the one where the original zpool resides but (as expected) we cannot import the zpool on the array snapshot to the original host..though no error message is issues, it just does not work. So, I am assuming that this is because the zpools have the same serial number (happy to be corrected). My question is are there any RFEs that will enable a zpool's serial number to be changed (if that is what is required to make this work). It may be that this is an awful idea..in which case I am happy to hear that as well and will feed that back to the customer. Thanks Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ? Changing storage pool serial number
Hi Tim, I took a look at the archives and I have seen a few threads about using array block level snapshots with ZFS and how we face the old issue that we used to see with logical volumes and unique IDs (quite correctly) stopping the same volume being presented twice to the same server. IHAC wanting to do this with a ZFS storage pool and an LSI array. All works well when they present the array snapshot to hosts other than the one where the original zpool resides but (as expected) we cannot import the zpool on the array snapshot to the original host..though no error message is issues, it just does not work. So, I am assuming that this is because the zpools have the same serial number (happy to be corrected). My question is are there any RFEs that will enable a zpool's serial number to be changed (if that is what is required to make this work). http://bugs.opensolaris.org/view_bug.do?bug_id=5097228 It may be that this is an awful idea..in which case I am happy to hear that as well and will feed that back to the customer. For both controller-based host-based snapshots, replicas, even iSCSI LUs, this would be an awful [good] idea. :-) - Jim Thanks Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] freeze/ thaw zfs file system
ajit jain wrote: Hi Andrew, I am writing a filtering device which tracks the write to the file-system. I am doing it for ufs, vxfs and for zfs. Sometime for consistent point I need to freeze the file-system which flushes dirty block to the disk and block every IO on the top level. So, for ufs and vxfs I got lockfs and VX_FREEZE/VX_THAW, but with zfs I didn't get the luck. I see no need to freeze the file system, or maybe I would say, I see no benefit to freezing a file system. Perhaps your needs will be met by the snapshot feature which will also cause a sync(2). -- richard Thanks for your response. Best regards, ajit On Tue, Jan 27, 2009 at 5:34 PM, Andrew Gabriel agabr...@opensolaris.org wrote: ajit jain wrote: Hi All, Can we freeze and thaw a zfs file-system either from user land (lockfs for ufs) or from the kernel space or through ioctl? Can you step back a level and explain what you're trying to achieve? Freezing UFS is to get round ufs-specific issues which don't apply to ZFS, so you maybe asking the wrong question. -- Andrew ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Crazy Problem with
Henri Meddox wrote: Hi Folks, call me a lernen ;-) I got a crazy Problem with zpool list and the size of my pool: created zpool create raidz2 hdd1 hdd2 hdd3 - each hdd is about 1GB. zpool list shows me a size of 2.95GB - shouldn't this bis online 1GB? After creating a file about 500MB - Capacity is shown as 50 % - The right value? Is this a known bug / feature? [Didn't I answer this just the other day? I need to improve my search skills :-)] From the man page for zpool(1m) These space usage properties report actual physical space available to the storage pool. The physical space can be different from the total amount of space that any contained datasets can actually use. The amount of space used in a raidz configuration depends on the characteristics of the data being written. In addition, ZFS reserves some space for internal accounting that the zfs(1M) command takes into account, but the zpool command does not. For non-full pools of a reasonable size, these effects should be invisible. For small pools, or pools that are close to being completely full, these discrepancies may become more noticeable. From the man page for zfs(1m) The amount of space available to the dataset and all its children, assuming that there is no other activity in the pool. Because space is shared within a pool, availa- bility can be limited by any number of factors, includ- ing physical pool size, quotas, reservations, or other datasets within the pool. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Raidz1 faulted with single bad disk. Requesting assistan
Any ideas on this? It looks like a potential bug to me, or there is something that I'm not seeing. Thanks again! -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] freeze/ thaw zfs file system
On 27 Jan 2009, at 17:59, Richard Elling wrote: ajit jain wrote: Hi Andrew, I am writing a filtering device which tracks the write to the file-system. I am doing it for ufs, vxfs and for zfs. Sometime for consistent point I need to freeze the file-system which flushes dirty block to the disk and block every IO on the top level. So, for ufs and vxfs I got lockfs and VX_FREEZE/VX_THAW, but with zfs I didn't get the luck. I see no need to freeze the file system, or maybe I would say, I see no benefit to freezing a file system. Perhaps your needs will be met by the snapshot feature which will also cause a sync(2). It sounds like the OP could take advantage of the vscan service http://www.opensolaris.org/os/project/vscan/ Cheers, Chris ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS and Amanda
Hello all, There is some project here to integrate amanda on opensolaris, or some howto for integration with ZFS? Some use case (using the opensource version)? The amanda site there is a few instructions, but i think here we can create something more specific to OS. Thanks. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Import Issues Btw 101 104
Hi! I have a system with S10, b101, and b104 installed in the same partition on disk 1. On disks 1 and 2 in different partitions, I also created ZFS pools from S10 to be imported by b101 and b104. Pool 1 is mirrored. Pool 2 is not. About every three builds, I replace the oldest build with the latest available and switch to that as the default OS. Up through 101 everything was fine. I just installed 104, however, and when I do the zpool import, the mirrored pool is picked up just fine, but the non-mirrored pool shows up as corrupted. zpool import -D shows me that zpool on b104 thinks that the pool is on c1t1d0p2, whereas S10, b101, and fdisk agree that it's actually on c1t1d0p3. How do I convince zpool on b104 where my non-mirrored pool really is? I'm a bit afraid to do an import -f because that pool is my home directory, and I'd really rather not screw it up. And I don't see where the -f will change zpool's mind about where the pool actually lives. Maybe import -c from the default location? Where is the default location? Any thoughts or suggestions? Thanks! Daniel ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Raidz1 faulted with single bad disk. Requesting assistan
Do you know 7200.11 has firmware bugs? Go to seagate website to check. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] how to fix zpool with corrupted disk?
Jakov Sosic wrote: Be happy, the data is already fixed. The DEGRADED state is used when too many errors were found in a short period of time, which one would use as an idicator of a failing device. However, since the evice is not actually failed, it is of no practical use in your test case. Well yes, now I do get it! The thing is, after scrubbing, I rebooted the machine and dd-ed the /dev/urandom on the other device in the mirror pool. After that, I rebooted again, and checked the md5sum of a file that occupies entire drive. The md5 is same as at the beggining of the test, so that's it. Problem solved, it seems that scrub really solves the problem, but after it solves it I have to zpool clear so that errors go away :) Pool seems in degraded state altough it's fully corrected... -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Raidz1 faulted with single bad disk. Requesting assistance.
I guess you could try 'zpool import -f'. This is a pretty odd status, I think. I'm pretty sure raidz1 should survive a single disk failure. Perhaps a more knowledgeable list member can explain. On Sat, Jan 24, 2009 at 12:48 PM, Brad Hill b...@thosehills.com wrote: I've seen reports of a recent Seagate firmware update bricking drives again. What's the output of 'zpool import' from the LiveCD? It sounds like ore than 1 drive is dropping off. r...@opensolaris:~# zpool import pool: tank id: 16342816386332636568 state: FAULTED status: The pool was last accessed by another system. action: The pool cannot be imported due to damaged devices or data. The pool may be active on another system, but can be imported using the '-f' flag. see: http://www.sun.com/msg/ZFS-8000-EY config: tankFAULTED corrupted data raidz1DEGRADED c6t0d0 ONLINE c6t1d0 ONLINE c6t2d0 ONLINE c6t3d0 UNAVAIL cannot open c6t4d0 ONLINE pool: rpool id: 9891756864015178061 state: ONLINE status: The pool was last accessed by another system. action: The pool can be imported using its name or numeric identifier and the '-f' flag. see: http://www.sun.com/msg/ZFS-8000-EY config: rpool ONLINE c3d0s0ONLINE -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Unable to destory a pool
Can you share the output of 'uname -a' and the disk controller you are using? On Sun, Jan 25, 2009 at 6:24 PM, Ramesh Mudradi rameshm.ku...@gmail.com wrote: # zpool list NAME SIZE USED AVAILCAP HEALTH ALTROOT jira-app-zpool 272G 330K 272G 0% ONLINE - The following command hangs forever. If I reboot the box , zpool list shows online as I mentioned the output above. # zpool destroy -f jira-app-zpool How can get rid of this pool and any reference to it. bash-3.00# zpool status pool: jira-app-zpool state: UNAVAIL status: One or more devices are faultd in response to IO failures. action: Make sure the affected devices are connected, then run 'zpool clear'. see: http://www.sun.com/msg/ZFS-8000-HC scrub: none requested config: NAMESTATE READ WRITE CKSUM jira-app-zpool UNAVAIL 0 0 4 insufficient replicas c3t0d3FAULTED 0 0 4 experienced I/O failures errors: 2 data errors, use '-v' for a list bash-3.00# -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Can't add mirror half back to mirror after moving controllers
In the previous config I had two RAID0 hardware stripes on an LSI1068 that were then mirrored together with ZFS. I then got a PERC 6/i card (aka LSI1078) to stick in the box and so I moved the one stripe over to that (and had to re-create the stripe of course). The problem is that once the stripe is moved to the PERC it's a handful of sectors smaller. The tricky part is that this is my ZFS Root OS pool. What are my best options for getting it all moved over to the new vdev so I can rebuild the primary stripe on the PERC? Also, what's a good stripe size to configure on the PERC that would work best with ZFS? Thanks! -brian -- Coding in C is like sending a 3 year old to do groceries. You gotta tell them exactly what you want or you'll end up with a cupboard full of pop tarts and pancake mix. -- IRC User (http://www.bash.org/?841435) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Unusual CIFS write bursts
comment far below... Brent Jones wrote: On Mon, Jan 26, 2009 at 10:40 PM, Brent Jones br...@servuhome.net wrote: While doing some performance testing on a pair of X4540's running snv_105, I noticed some odd behavior while using CIFS. I am copying a 6TB database file (yes, a single file) over our GigE network to the X4540, then snapshotting that data to the secondary X4540. Writing said 6TB file can peak our gigabit network, with about 95-100MB/sec going over the wire (can't ask for any more, really). However, the disk IO on the X4540 appears unusual. I would expect the disks to be constantly writing 95-100MB/sec, but it appears it buffers about 1GB worth of data before committing to disk. This is in contrast to NFS write behavior, where as I write a 1GB file to the NFS server from an NFS client, traffic on the wire correlates concisely to the disk writes. For example, 60MB/sec on the wire via NFS will trigger 60MB/sec on disk. This is a single file on both cases. I wouldn't have a problem with this buffer, it seems to be a rolling 10-second buffer, if I am copying several small files at lower speeds, the disk buffer still seems to purge after roughly 10 seconds, not when a certain size is reached. The larger the amount of data that goes into the buffer is what causes a problem, writing 1GB to disk can cause the system to slow down substantially, all network traffic pauses or drops to mere kilobytes a second while it writes this buffer. I would like to see a smoother handling of this buffer, or a tuneable to make the buffer write more often or fill quicker. This is a 48TB unit, 64GB ram, and the arcstat perl script reports my ARC is 55GB in size, with near 0% miss on reads. Has anyone seen something similar, or know of any un-documented tuneables to reduce the effects of this? Here is 'zpool iostat' output, in 1 second intervals while this write storm occurs. # zpool iostat pdxfilu01 1 capacity operationsbandwidth pool used avail read write read write -- - - - - - - pdxfilu01 2.09T 36.0T 1 61 143K 7.30M pdxfilu01 2.09T 36.0T 0 0 0 0 pdxfilu01 2.09T 36.0T 0 0 0 0 pdxfilu01 2.09T 36.0T 0 0 0 0 pdxfilu01 2.09T 36.0T 0 60 0 7.55M pdxfilu01 2.09T 36.0T 0 1.70K 0 211M pdxfilu01 2.09T 36.0T 0 2.56K 0 323M pdxfilu01 2.09T 36.0T 0 2.97K 0 375M pdxfilu01 2.09T 36.0T 0 3.15K 0 399M pdxfilu01 2.09T 36.0T 0 2.22K 0 244M pdxfilu01 2.09T 36.0T 0 0 0 0 pdxfilu01 2.09T 36.0T 0 0 0 0 pdxfilu01 2.09T 36.0T 0 0 0 0 pdxfilu01 2.09T 36.0T 0 0 0 0 Here is my 'zpool status' output. # zpool status pool: pdxfilu01 state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM pdxfilu01 ONLINE 0 0 0 raidz1ONLINE 0 0 0 c5t0d0 ONLINE 0 0 0 c6t0d0 ONLINE 0 0 0 c7t0d0 ONLINE 0 0 0 c8t0d0 ONLINE 0 0 0 c9t0d0 ONLINE 0 0 0 raidz1ONLINE 0 0 0 c4t1d0 ONLINE 0 0 0 c6t1d0 ONLINE 0 0 0 c7t1d0 ONLINE 0 0 0 c8t1d0 ONLINE 0 0 0 c9t1d0 ONLINE 0 0 0 raidz1ONLINE 0 0 0 c4t2d0 ONLINE 0 0 0 c5t2d0 ONLINE 0 0 0 c7t2d0 ONLINE 0 0 0 c8t2d0 ONLINE 0 0 0 c9t2d0 ONLINE 0 0 0 raidz1ONLINE 0 0 0 c4t3d0 ONLINE 0 0 0 c5t3d0 ONLINE 0 0 0 c6t3d0 ONLINE 0 0 0 c8t3d0 ONLINE 0 0 0 c9t3d0 ONLINE 0 0 0 raidz1ONLINE 0 0 0 c4t4d0 ONLINE 0 0 0 c5t4d0 ONLINE 0 0 0 c6t4d0 ONLINE 0 0 0 c7t4d0 ONLINE 0 0 0 c9t4d0 ONLINE 0 0 0 raidz1ONLINE 0 0 0 c4t5d0 ONLINE 0 0 0 c5t5d0 ONLINE 0 0 0 c6t5d0 ONLINE 0 0 0 c7t5d0 ONLINE 0 0 0 c8t5d0 ONLINE 0 0 0 raidz1ONLINE 0 0 0 c4t6d0 ONLINE 0 0 0 c5t6d0 ONLINE 0 0 0 c6t6d0 ONLINE 0 0 0 c7t6d0
Re: [zfs-discuss] Unusual CIFS write bursts
On Tue, Jan 27, 2009 at 5:47 PM, Richard Elling richard.ell...@gmail.com wrote: comment far below... Brent Jones wrote: On Mon, Jan 26, 2009 at 10:40 PM, Brent Jones br...@servuhome.net wrote: -- Brent Jones br...@servuhome.net I found some insight to the behavior I found at this Sun blog by Roch Bourbonnais : http://blogs.sun.com/roch/date/20080514 Excerpt from the section that I seem to have encountered: The new code keeps track of the amount of data accepted in a TXG and the time it takes to sync. It dynamically adjusts that amount so that each TXG sync takes about 5 seconds (txg_time variable). It also clamps the limit to no more than 1/8th of physical memory. So, when I fill up that transaction group buffer, that is when I see that 4-5 second I/O burst of several hundred megabytes per second. He also documents that the buffer flush can, and does issue delays to the writing threads, which is why I'm seeing those momentary drops in throughput and sluggish system performance while that write buffer is flushed to disk. Yes, this tends to be more efficient. You can tune it by setting zfs_txg_synctime, which is 5 by default. It is rare that we've seen this be a win, which is why we don't mention it in the Evil Tuning Guide. Wish there was a better way to handle that, but at the speed I'm writing (and I'll be getting a 10GigE link soon), I don't see any other graceful methods of handling that much data in a buffer I think your workload might change dramatically when you get a faster pipe. So unless you really feel compelled to change it, I wouldn't suggest changing it. -- richard Loving these X4540's so far though... Are there any additional tuneables, such as opening a new txg buffer before the previous one is flushed? Or otherwise allow writes to continue without the tick delay? My workload will be pretty consistent, it is going to serve a few roles, which I hope to accomplish in the same units: - large scale backups - cifs share for window app servers - nfs server for unix app servers GigE quickly became the bottleneck, and I imagine 10GigE will add further stress to those write buffers. -- Brent Jones br...@servuhome.net ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] firewire card?
just installed s10_u6 with a root pool. i'm blown away. so now i want to attach my external storage via firewire. i could use usb2 but i prefer firewire as i won't need an external hub. what firewire cards are supported for x86? the HCL doesn't list any that i could find. i searched for any of the terms 'firewire', '1394', 'ohci', 'uhci', 'ehci'. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] firewire card?
On Tue, 27 Jan 2009, Frank Cusack wrote: what firewire cards are supported for x86? the HCL doesn't list any that i could find. i searched for any of the terms 'firewire', '1394', 'ohci', 'uhci', 'ehci'. The Sun Ultra-40 (recently discontinued) comes with dual 400Mbit firefire ports. See if you can figure out what hardware it uses. I would be surprised if it is not the same hardware that was used for SPARC systems like the Blade 2500 and Ultra-45, with a commercially available adaptor card still available as recent as a couple of years ago (when I last checked). The firewire support in the Ultra-40 is from the motherboard rather than an adaptor card. However, be aware that bugs causing problems with ZFS were reported against the Firewire support, and it is not clear if anyone got around to fixing them even though the fix was apparently simple. I am using USB2 with two external drives without problem. Bob == Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] firewire card?
On Tue, 27 Jan 2009 21:01:55 -0600 (CST) Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Tue, 27 Jan 2009, Frank Cusack wrote: what firewire cards are supported for x86? the HCL doesn't list any that i could find. i searched for any of the terms 'firewire', '1394', 'ohci', 'uhci', 'ehci'. The Sun Ultra-40 (recently discontinued) comes with dual 400Mbit firefire ports. See if you can figure out what hardware it uses. I would be surprised if it is not the same hardware that was used for SPARC systems like the Blade 2500 and Ultra-45, with a commercially available adaptor card still available as recent as a couple of years ago (when I last checked). The firewire support in the Ultra-40 is from the motherboard rather than an adaptor card. However, be aware that bugs causing problems with ZFS were reported against the Firewire support, and it is not clear if anyone got around to fixing them even though the fix was apparently simple. I am using USB2 with two external drives without problem. Hi Frank, This is what lspci reports on my u40m2 system: 01:08.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link) (prog-if 10 [OHCI]) Subsystem: Sun Microsystems Computer Corp.: Unknown device 6676 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium TAbort- TAbort- MAbort- SERR- PERR- Latency: 64 (500ns min, 1000ns max), cache line size 10 Interrupt: pin A routed to IRQ 11 Region 0: Memory at a0104800 (32-bit, non-prefetchable) Region 1: Memory at a010 (32-bit, non-prefetchable) Capabilities: [44] Power Management version 2 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-) Status: D0 PME-Enable- DSel=0 DScale=0 PME+ prtconf reports that the full pci vid/did information is pci104c,8023.108e.6676.0 As Bob mentioned, this is on the motherboard rather than a plugin card. James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Replacing HDD in x4500
I'm not an authority, but on my 'vanilla' filer, using the same controller chipset as the thumper, I've been in really good shape since moving to zfs boot in 10/08 and doing 'zpool upgrade' and 'zfs upgrade' to all my mirrors (3 3-way). I'd been having similar troubles to yours in the past. My system is pretty puny next to yours, but it's been reliable now for slightly over a month. On Tue, Jan 27, 2009 at 12:19 AM, Jorgen Lundman lund...@gmo.jp wrote: The vendor wanted to come in and replace an HDD in the 2nd X4500, as it was constantly busy, and since our x4500 has always died miserably in the past when a HDD dies, they wanted to replace it before the HDD actually died. The usual was done, HDD replaced, resilvering started and ran for about 50 minutes. Then the system hung, same as always, all ZFS related commands would just hang and do nothing. System is otherwise fine and completely idle. The vendor for some reason decided to fsck root-fs, not sure why as it is mounted with logging, and also decided it would be best to do so from a CDRom boot. Anyway, that was 12 hours ago and the x4500 is still down. I think they have it at single-user prompt resilvering again. (I also noticed they'd decided to break the mirror of the root disks for some very strange reason). It still shows: raidz1 DEGRADED 0 0 0 c0t1d0ONLINE 0 0 0 replacing UNAVAIL 0 0 0 insufficient replicas c1t1d0s0/o OFFLINE 0 0 0 c1t1d0 UNAVAIL 0 0 0 cannot open So I am pretty sure it'll hang again sometime soon. What is interesting though is that this is on x4500-02, and all our previous troubles mailed to the list was regarding our first x4500. The hardware is all different, but identical. Solaris 10 5/08. Anyway, I think they want to boot CDrom to fsck root again for some reason, but since customers have been without their mail for 12 hours, they can go a little longer, I guess. What I was really wondering, has there been any progress or patches regarding the system always hanging whenever a HDD dies (or is replaced it seems). It really is rather frustrating. Lund -- Jorgen Lundman | lund...@lundman.net Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work) Shibuya-ku, Tokyo| +81 (0)90-5578-8500 (cell) Japan| +81 (0)3 -3375-1767 (home) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Replacing HDD in x4500
Thanks for your reply, While the savecore is working its way up the chain to (hopefully) Sun, the vendor asked us not to use it, so we moved x4500-02 to use x4500-04 and x4500-05. But perhaps moving to Sol 10 10/08 on x4500-02 when fixed is the way to go. The savecore had the usual info, that everything is blocked waiting on locks: 601* threads trying to get a mutex (598 user, 3 kernel) longest sleeping 10 minutes 13.52 seconds earlier 115* threads trying to get an rwlock (115 user, 0 kernel) 1678 total threads in allthreads list (1231 user, 447 kernel) 10 thread_reapcnt 0 lwp_reapcnt 1688 nthread thread pri pctcpu idle PID wchan command 0xfe8000137c80 60 0.000 -9m44.88s 0 0xfe84d816cdc8 sched 0xfe800092cc80 60 0.000 -9m44.52s 0 0xc03c6538 sched 0xfe8527458b40 59 0.005 -1m41.38s 1217 0xb02339e0 /usr/lib/nfs/rquotad 0xfe8527b534e0 60 0.000 -5m4.79s 402 0xfe84d816cdc8 /usr/lib/nfs/lockd 0xfe852578f460 60 0.000 -4m59.79s 402 0xc0633fc8 /usr/lib/nfs/lockd 0xfe8532ad47a0 60 0.000 -10m4.40s 623 0xfe84bde48598 /usr/lib/nfs/nfsd 0xfe8532ad3d80 60 0.000 -10m9.10s 623 0xfe84d816ced8 /usr/lib/nfs/nfsd 0xfe8532ad3360 60 0.000 -10m3.77s 623 0xfe84d816cde0 /usr/lib/nfs/nfsd 0xfe85341e9100 60 0.000 -10m6.85s 623 0xfe84bde48428 /usr/lib/nfs/nfsd 0xfe85341e8a40 60 0.000 -10m4.76s 623 0xfe84d816ced8 /usr/lib/nfs/nfsd SolarisCAT(vmcore.0/10X) tlist sobj locks | grep nfsd | wc -l 680 scl_writer = 0xfe8000185c80 - locking thread thread 0xfe8000185c80 kernel thread: 0xfe8000185c80 PID: 0 cmd: sched t_wchan: 0xfbc8200a sobj: condition var (from genunix:bflush+0x4d) t_procp: 0xfbc22dc0(proc_sched) p_as: 0xfbc24a20(kas) zone: global t_stk: 0xfe8000185c80 sp: 0xfe8000185aa0 t_stkbase: 0xfe8000181000 t_pri: 99(SYS) pctcpu: 0.00 t_lwp: 0x0 psrset: 0 last CPU: 0 idle: 44943 ticks (7 minutes 29.43 seconds) start: Tue Jan 27 23:44:21 2009 age: 674 seconds (11 minutes 14 seconds) tstate: TS_SLEEP - awaiting an event tflg: T_TALLOCSTK - thread structure allocated from stk tpflg: none set tsched: TS_LOAD - thread is in memory TS_DONT_SWAP - thread/LWP should not be swapped pflag: SSYS - system resident process pc: 0xfb83616f unix:_resume_from_idle+0xf8 resume_return startpc: 0xeff889e0 zfs:spa_async_thread+0x0 unix:_resume_from_idle+0xf8 resume_return() unix:swtch+0x12a() genunix:cv_wait+0x68() genunix:bflush+0x4d() genunix:ldi_close+0xbe() zfs:vdev_disk_close+0x6a() zfs:vdev_close+0x13() zfs:vdev_raidz_close+0x26() zfs:vdev_close+0x13() zfs:vdev_reopen+0x1d() zfs:spa_async_reopen+0x5f() zfs:spa_async_thread+0xc8() unix:thread_start+0x8() -- end of kernel thread's stack -- Blake wrote: I'm not an authority, but on my 'vanilla' filer, using the same controller chipset as the thumper, I've been in really good shape since moving to zfs boot in 10/08 and doing 'zpool upgrade' and 'zfs upgrade' to all my mirrors (3 3-way). I'd been having similar troubles to yours in the past. My system is pretty puny next to yours, but it's been reliable now for slightly over a month. On Tue, Jan 27, 2009 at 12:19 AM, Jorgen Lundman lund...@gmo.jp wrote: The vendor wanted to come in and replace an HDD in the 2nd X4500, as it was constantly busy, and since our x4500 has always died miserably in the past when a HDD dies, they wanted to replace it before the HDD actually died. The usual was done, HDD replaced, resilvering started and ran for about 50 minutes. Then the system hung, same as always, all ZFS related commands would just hang and do nothing. System is otherwise fine and completely idle. The vendor for some reason decided to fsck root-fs, not sure why as it is mounted with logging, and also decided it would be best to do so from a CDRom boot. Anyway, that was 12 hours ago and the x4500 is still down. I think they have it at single-user prompt resilvering again. (I also noticed they'd decided to break the mirror of the root disks for some very strange reason). It still shows: raidz1 DEGRADED 0 0 0 c0t1d0ONLINE 0 0 0 replacing UNAVAIL 0 0 0 insufficient replicas c1t1d0s0/o OFFLINE 0 0 0 c1t1d0 UNAVAIL 0 0 0 cannot open So I am pretty sure it'll hang again sometime soon. What is interesting though is that this is on x4500-02, and all our previous troubles mailed to the list was regarding our first x4500. The hardware is all different, but identical. Solaris 10 5/08. Anyway, I think they want to boot CDrom to
Re: [zfs-discuss] zpool status -x strangeness
What does 'zpool status -xv' show? On Tue, Jan 27, 2009 at 8:01 AM, Ben Miller mil...@eecis.udel.edu wrote: I forgot the pool that's having problems was recreated recently so it's already at zfs version 3. I just did a 'zfs upgrade -a' for another pool, but some of those filesystems failed since they are busy and couldn't be unmounted. # zfs upgrade -a cannot unmount '/var/mysql': Device busy cannot unmount '/var/postfix': Device busy 6 filesystems upgraded 821 filesystems already at this version Ben You can upgrade live. 'zfs upgrade' with no arguments shows you the zfs version status of filesystems present without upgrading. On Jan 24, 2009, at 10:19 AM, Ben Miller mil...@eecis.udel.edu wrote: We haven't done 'zfs upgrade ...' any. I'll give that a try the next time the system can be taken down. Ben A little gotcha that I found in my 10u6 update process was that 'zpool upgrade [poolname]' is not the same as 'zfs upgrade [poolname]/[filesystem(s)]' What does 'zfs upgrade' say? I'm not saying this is the source of your problem, but it's a detail that seemed to affect stability for me. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Raidz1 faulted with single bad disk. Requesting
r...@opensolaris:~# zpool import -f tank internal error: Bad exchange descriptor Abort (core dumped) Hoping someone has seen that before... the Google is seriously letting me down on that one. I guess you could try 'zpool import -f'. This is a pretty odd status, I think. I'm pretty sure raidz1 should survive a single disk failure. Perhaps a more knowledgeable list member can explain. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Replacing HDD in x4500
On Tue, Jan 27, 2009 at 9:28 PM, Jorgen Lundman lund...@gmo.jp wrote: Thanks for your reply, While the savecore is working its way up the chain to (hopefully) Sun, the vendor asked us not to use it, so we moved x4500-02 to use x4500-04 and x4500-05. But perhaps moving to Sol 10 10/08 on x4500-02 when fixed is the way to go. The savecore had the usual info, that everything is blocked waiting on locks: I assume you've changed the failmode to continue already? http://prefetch.net/blog/index.php/2008/03/01/configuring-zfs-to-gracefully-deal-with-failures/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] firewire card?
Frank Cusack wrote: just installed s10_u6 with a root pool. i'm blown away. so now i want to attach my external storage via firewire. I was able to use this cheap thing with good initial results: http://www.newegg.com/Product/Product.aspx?Item=N82E16815124002 However, I ran into a frequent hang[1] and declined to put it in the HWCL. The hang appears to be a bug in the scsa1394 kernel module, and are not yet fixed. I've moved the disks our of their Firewire case and inside a machine with SATA-to-IDE converters (ick). [1] http://bugs.opensolaris.org/view_bug.do?bug_id=6539587 Rob T ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Raidz1 faulted with single bad disk. Requesting
This is outside the scope of my knowledge/experience. Maybe there is now a core file you can examine? That might help you at least see what's going on? On Tue, Jan 27, 2009 at 10:32 PM, Brad Hill b...@thosehills.com wrote: r...@opensolaris:~# zpool import -f tank internal error: Bad exchange descriptor Abort (core dumped) Hoping someone has seen that before... the Google is seriously letting me down on that one. I guess you could try 'zpool import -f'. This is a pretty odd status, I think. I'm pretty sure raidz1 should survive a single disk failure. Perhaps a more knowledgeable list member can explain. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] firewire card?
thanks for all the feedback. i guess i'll stick with usb2. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Replacing HDD in x4500
I assume you've changed the failmode to continue already? http://prefetch.net/blog/index.php/2008/03/01/configuring-zfs-to-gracefully-deal-with-failures/ This appears to be new to 10/08, so that is another vote to upgrade. Also interesting that the default is wait, since it almost behaves like it. Not sure why it would block zpool, zfs and df commands as well though? Lund -- Jorgen Lundman | lund...@lundman.net Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work) Shibuya-ku, Tokyo| +81 (0)90-5578-8500 (cell) Japan| +81 (0)3 -3375-1767 (home) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Raidz1 faulted with single bad disk. Requesting assistan
I do, thank you. The disk that went out sounds like it had a head crash or some such - loud clicking shortly after spin-up then it spins down and gives me nothing. BIOS doesn't even detect it properly to do a firmware update. Do you know 7200.11 has firmware bugs? Go to seagate website to check. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Raidz1 faulted with single bad disk. Requesting
Just a thought, but have you physically disconnected the bad disk? It's not unheard of for a bad disk to cause problems with others. Failing that, it's the corrupted data bit that's worrying me, it sounds like you may have other corruption on the pool (always a risk with single parity raid), but I'm worried that it's not giving you any more details as to what's wrong. Also, what version of OpenSolaris are you running? Could you maybe try booting off a CD of the latest build? There are often improvements in the way ZFS copes with errors, so it's worth a try. I don't think it's likely to help, but I wouldn't discount it. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] mount race condition?
i was wondering if you have a zfs filesystem that mounts in a subdir in another zfs filesystem, is there any problem with zfs finding them in the wrong order and then failing to mount correctly? say you have pool1/data which mounts on /data and pool2/foo which mounts on /data/subdir/foo, what if at boot time, pool2 is imported first, what happens? /data would exist but /data/subdir wouldn't exist since pool1/data hasn't been mounted yet. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
