[zfs-discuss] Zpool problems
Hi, I'm using zfs version 6 on mac os x 10.5 using the old macosforge pkg. When I'm writing files to the fs they are appearing as 1kb files and if I do zpool status or scrub or anything the command is just hanging. However I can still read the zpool ok, just write is having problems and any diagnostics. Any ideas how I can get more information or what my symptoms are resemblent of? I'm considering using the freebsd ppc port (as i have a powermac) for better zfs support. Any thoughts would be great on why I'm having these problems. Thanks ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send | verify | receive
Edward Ned Harvey wrote: I use the excellent pbzip2 zfs send ... | tee (md5sum) | pbzip2 | ssh remote ... Utilizes those 8 cores quite well :) This (pbzip2) sounds promising, and it must be better than what I wrote. ;-) But I don't understand the syntax you've got above, using tee, redirecting to something in parens. I haven't been able to do this yet on my own system. Can you please give me an example to simultaneously generate md5sum and gzip? This is how I currently do it: cat somefile | multipipe md5sum somefile.md5sum gzip somefile.gz End result is: somefile somefile.md5sum somefile.gz So that would be cat somefile | tee (md5sum somefile.md5sum) | gzip somefile.gz ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [zfs-code] Transaction consistency of ZFS
On Sun, Dec 6, 2009 at 8:11 PM, Anurag Agarwal anu...@kqinfotech.com wrote: Hi, My reading of write code of ZFS (zfs_write in zfs_vnops.c), is that all the writes in zfs are logged in the ZIL. And if that indeed is the case, then IIRC, there is some upper limit (1MB?) on writes that go to ZIL, with larger ones executed directly. Yet again, this is an outsider's impression, not the architect's () statement. Regards, Andrey yes, ZFS does guarantee the sequential consistency, even when there are power outage or server crash. You might loose some writes if ZIL has not committed to disk. But that would not change the sequential consistency guarantee. There is no need to do a fsync or open the file with O_SYNC. It should work as it is. I have not done any experiments to verify this, so please take my observation with pinch of salt. Any ZFS developers to verify or refute this. Regards, Anurag. On Sun, Dec 6, 2009 at 8:12 AM, nxyyt schumi@gmail.com wrote: This question is forwarded from ZFS-discussion. Hope any developer can throw some light on it. I'm a newbie to ZFS. I have a special question against the COW transaction of ZFS. Does ZFS keeps the sequential consistency of the same file when it meets power outage or server crash? Assume following scenario: My application has only a single thread and it appends the data to the file continuously. Suppose at time t1, it append a buf named A to the file. At time t2, which is later than t1, it appends a buf named B to the file. If the server crashes after t2, is it possible the buf B is flushed back to the disk but buf A is not? My application appends the file only without truncation or overwrite.Does ZFS keep the consistency that the data written to a file in sequential order or casual order be flushed to disk in the same order? If the uncommitted writer operation to a single file always binding with the same opening transaction group and all transaction group is committed in sequential order, I think the answer should be YES. In other words, [b]whether there is only one opening transaction group at any time and the transaction group is committed in order for a single pool?[/b] Hope anybody can help me clarify it. Thank you very much! -- This message posted from opensolaris.org ___ zfs-code mailing list zfs-c...@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-code -- Anurag Agarwal CEO, Founder KQ Infotech, Pune www.kqinfotech.com 9881254401 Coordinator Akshar Bharati www.aksharbharati.org Spreading joy through reading ___ zfs-code mailing list zfs-c...@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-code ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
on 06/12/2009 19:40 Volker A. Brandt said the following: I wanted to add a disk to the tank pool to create a mirror. I accidentally used zpool add ? instead of zpool attach ? and now the disk is added. Is there a way to remove the disk without loosing data? Been there, done that -- at a customer site while showing off ZFS. :-) Currently, you cannot remove a simple device. Depending on your Solaris version, you can remove things like hot spares and cache devices, but not simple vdevs. Backup the pool and recreate it in the correct way. Sorry for the bad news -- Volker Yep. My 2 cents -- 'add' and 'attach' are so similar the words that I think that ZFS tools UI designers (if any) should re-consider naming of these commands. Or 'add' command should always be interactive and ask for at least two confirmations that a user knows what he is doing and why. Perhaps, it should include a ZFS micro-exam too. Jokes aside, this is too easy to make a mistake with the consequences that are too hard to correct. Anyone disagrees? -- Andriy Gapon ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] mpt errors on snv 101b
Hi, I would like to add , yet another, mpt timeout report. Suddently the system started to get slow, noticeable due to the fact that some linux clients where complaining about nfs server timeout, and after some time i saw alot of reset bus messages in the /var/adm/messsages file. I quickly took a look to the JBOD chassis, and one of the disks had a fixed light, and after the physical removal of this disk, the system re-started to respond and the resilver process kicked in, due to a spare disk took the place of the disconnected disk, as seen with the zpool status -v : zpool status -v DATAPOOL04 pool: DATAPOOL04 state: DEGRADED status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: http://www.sun.com/msg/ZFS-8000-9P scrub: resilver in progress for 1h40m, 8.26% done, 18h32m to go config: NAME STATE READ WRITE CKSUM DATAPOOL04 DEGRADED 0 0 0 raidz1 DEGRADED 0 0 0 c5t27d0ONLINE 0 0 0 105M resilvered c5t29d0ONLINE 0 0 0 105M resilvered c5t30d0ONLINE 0 0 0 105M resilvered spare DEGRADED 0 0 0 c5t31d0 REMOVED 0 423K 0 c5t28d0 ONLINE 0 0 0 9.83G resilvered c5t32d0ONLINE 0 0 0 105M resilvered spares c5t28d0 INUSE currently in use errors: No known data errors At this moment the system is doing the resilvering, but the messages regarding disk/disk controller still appear in the log. Could this messages appear due to the fact that the resilver process is a heavy one, or more disks are probably affected? In cases such as this one, what's the best procedure to do? * shutdown server and JBOD , including power off/power on and see how it goes * replace HBA/disk ? * other ? Thanks for the time, and if any other information is required (even ssh access can be granted) please feel free to ask it. Best regards, Bruno Sousa System specs : * OpenSolaris snv_101b, with two Dual-Core AMD, and 16 GB Ram * LSI Logic SAS1068E, revision B3 , MPT Rev 105, Firmware Rev 011a * 24 disks are attached to this HBA, the disks are Seagate Sata 1TB Enterprise class (ATA-ST31000340NS-SN06-931.51GB ) * the LSI HBA is connect with 1 SFF 8087 connector cable (SAS 846EL1 BP 1-Port Internal Cascading Cable) to a Supermicro Chassis SC 846 with a SAS / SATA Expander Backplane with single LSI SASX36 Expander Chip /var/adm/messages content Dec 7 13:57:12 san01 scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci10de,3...@a/pci1000,3...@0/s...@17,0 (sd18): Dec 7 13:57:12 san01 Error for Command: write(10) Error Level: Retryable Dec 7 13:57:12 san01 scsi: [ID 107833 kern.notice]Requested Block: 48696432 Error Block: 48696432 Dec 7 13:57:12 san01 scsi: [ID 107833 kern.notice]Vendor: ATASerial Number: Dec 7 13:57:12 san01 scsi: [ID 107833 kern.notice]Sense Key: Unit_Attention Dec 7 13:57:12 san01 scsi: [ID 107833 kern.notice]ASC: 0x29 (power on, reset, or bus reset occurred), ASCQ: 0x0, FRU: 0x0 Dec 7 13:57:15 san01 scsi: [ID 243001 kern.warning] WARNING: /p...@0,0/pci10de,3...@a/pci1000,3...@0 (mpt0): Dec 7 13:57:15 san01 mpt_handle_event_sync: IOCStatus=0x8000, IOCLogInfo=0x31123000 Dec 7 13:57:15 san01 scsi: [ID 243001 kern.warning] WARNING: /p...@0,0/pci10de,3...@a/pci1000,3...@0 (mpt0): Dec 7 13:57:15 san01 mpt_handle_event: IOCStatus=0x8000, IOCLogInfo=0x31123000 Dec 7 13:57:45 san01 Log info 0x31123000 received for target 21. Dec 7 13:57:45 san01 scsi_status=0x0, ioc_status=0x804b, scsi_state=0xc Dec 7 13:57:45 san01 scsi: [ID 365881 kern.notice] /p...@0,0/pci10de,3...@a/pci1000,3...@0 (mpt0): Dec 7 13:57:45 san01 Log info 0x31123000 received for target 21. Dec 7 13:57:45 san01 scsi_status=0x0, ioc_status=0x804b, scsi_state=0xc Dec 7 13:57:45 san01 scsi: [ID 365881 kern.notice] /p...@0,0/pci10de,3...@a/pci1000,3...@0 (mpt0): Dec 7 13:57:45 san01 Log info 0x31123000 received for target 21. Dec 7 13:57:45 san01 scsi_status=0x0, ioc_status=0x804b, scsi_state=0xc Dec 7 13:57:45 san01 scsi: [ID 365881 kern.notice] /p...@0,0/pci10de,3...@a/pci1000,3...@0 (mpt0): Dec 7 13:57:45 san01 Log info 0x31123000 received for target 21. Dec 7 13:57:45 san01 scsi_status=0x0, ioc_status=0x804b, scsi_state=0xc Dec 7 13:57:45 san01 scsi: [ID 365881 kern.notice] /p...@0,0/pci10de,3...@a/pci1000,3...@0 (mpt0): Dec 7 13:57:45 san01 Log info 0x31123000 received for
[zfs-discuss] SMC for ZFS administration in OpenSolaris 2009.06?
Hi, I this available for the current OpenSolaris? it would be great to have some graphical administration interface. rgds. xavier -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zpool problems
On 6 Dec 2009, at 16:14, Michael Armstrong wrote: Hi, I'm using zfs version 6 on mac os x 10.5 using the old macosforge pkg. When I'm writing files to the fs they are appearing as 1kb files and if I do zpool status or scrub or anything the command is just hanging. However I can still read the zpool ok, just write is having problems and any diagnostics. Any ideas how I can get more information or what my symptoms are resemblent of? I'm considering using the freebsd ppc port (as i have a powermac) for better zfs support. Any thoughts would be great on why I'm having these problems. You may be better off talking to the folks at https://groups.google.com/group/zfs-macos who are actively using and working on the Mac port of ZFS. Cheers, Chris ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zpool problems
Hi Michael, Whenever I see commands hanging, I would first rule out any hardware issues. I'm not sure how to do that on a OS X. Cindy On 12/06/09 09:14, Michael Armstrong wrote: Hi, I'm using zfs version 6 on mac os x 10.5 using the old macosforge pkg. When I'm writing files to the fs they are appearing as 1kb files and if I do zpool status or scrub or anything the command is just hanging. However I can still read the zpool ok, just write is having problems and any diagnostics. Any ideas how I can get more information or what my symptoms are resemblent of? I'm considering using the freebsd ppc port (as i have a powermac) for better zfs support. Any thoughts would be great on why I'm having these problems. Thanks ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Using iSCSI on ZFS with non-native FS - How to backup.
It does 'just work', however you may have some file and/or file system corruption if the snapshot was taken at the moment that your mac is updating some files. So use the time slider function and take a lot of snaps. :) -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Expected ZFS behavior?
Hi all, First, kudos to all the ZFS folks for a killer technology. We use several Sun 7000 series boxes at work and love the features. I recently decided to build an Opensolaris server for home. I just put the box together over the weekend. It is using an LSI 1068E based HBA (Supermicro FWIW) and 8 2TB WD drives in a single raidz2 pool. It is a clean install of snv_128a with the only changes from vanilla being to install the CIFS server packages and create and share a CIFS share. I started copying over all the data from my existing workstation. When copying files (mostly multi-gigabyte DV video files), network throughput drops to zero for ~1/2 second every 8-15 seconds. This throughput drop corresponds to drive activity on the Opensolaris box. The ZFS pool drives show no activity except every 8-15 seconds. As best as I can guess, the Opensolaris box is caching traffic and batching it to disk every so often. I guess I didn't expect disk writes to interrupt network traffic. Is this correct? One other item to note, the pool is currently degraded as one of the drives was apparently damaged during shipping and died almost immediately after I created my pool. I completely removed this drive to RMA it. I'd be happy to provide any info needed. Thanks in advance. Richard Bruce -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SMC for ZFS administration in OpenSolaris 2009.06?
On 12/07/09 09:37, Cindy Swearingen wrote: Hi Xavier, Neither the SMC interface nor the ZFS webconsole is available in OpenSolaris releases. The SMC cannot be used for ZFS administration in any Solaris release. I'm not sure what the replacement plans are but you might check with the experts on the indiana-discuss list. Thanks, Cindy On 12/07/09 08:44, Xavier Callejas wrote: Hi, I this available for the current OpenSolaris? it would be great to have some graphical administration interface. rgds. xavier ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Expected ZFS behavior?
On Mon, 7 Dec 2009, Richard Bruce wrote: I started copying over all the data from my existing workstation. When copying files (mostly multi-gigabyte DV video files), network throughput drops to zero for ~1/2 second every 8-15 seconds. This throughput drop corresponds to drive activity on the Opensolaris box. The ZFS pool drives show no activity except every 8-15 seconds. As best as I can guess, the Opensolaris box is caching traffic and batching it to disk every so often. I guess I didn't expect disk writes to interrupt network traffic. Is this correct? This is expected behavior. From what has been posted here, these are the current buffering rules: up to 7/8ths of available memory up to 5 seconds worth of 100% write I/O time up to 30 seconds without a write and if you don't like it, you can use the zfs:zfs_arc_max tunable in /etc/system to set a maximum amount of memory to be used prior to a write. This may be useful on systems with a large amount of memory and which want to limit the maximum delay time due to committing the zfs transation group. There will still be interruptions, but the interruptions can be made briefer (and more often). Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Expected ZFS behavior?
On Mon, 7 Dec 2009, Bob Friesenhahn wrote: and if you don't like it, you can use the zfs:zfs_arc_max tunable in /etc/system to set a maximum amount of memory to be used prior to a write. Oops. Bad cut-n-paste. That should have been zfs:zfs_write_limit_override So I am currently using * Set ZFS maximum TXG group size to 393216 set zfs:zfs_write_limit_override = 0xea60 Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
I agree that zpool attach and add look similar in their syntax, but if you attempt to add a disk to a redundant config, you'll see an error message similar to the following: # zpool status export pool: export state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM export ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 c1t4d0 ONLINE 0 0 0 errors: No known data errors # zpool add export c1t6d0 invalid vdev specification use '-f' to override the following errors: mismatched replication level: pool uses mirror and new vdev is disk Doesn't the mismatched replication message help? Cindy On 12/06/09 14:13, Andriy Gapon wrote: on 06/12/2009 19:40 Volker A. Brandt said the following: I wanted to add a disk to the tank pool to create a mirror. I accidentally used zpool add ? instead of zpool attach ? and now the disk is added. Is there a way to remove the disk without loosing data? Been there, done that -- at a customer site while showing off ZFS. :-) Currently, you cannot remove a simple device. Depending on your Solaris version, you can remove things like hot spares and cache devices, but not simple vdevs. Backup the pool and recreate it in the correct way. Sorry for the bad news -- Volker Yep. My 2 cents -- 'add' and 'attach' are so similar the words that I think that ZFS tools UI designers (if any) should re-consider naming of these commands. Or 'add' command should always be interactive and ask for at least two confirmations that a user knows what he is doing and why. Perhaps, it should include a ZFS micro-exam too. Jokes aside, this is too easy to make a mistake with the consequences that are too hard to correct. Anyone disagrees? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
On Dec 7, 2009, at 10:42 AM, Cindy Swearingen wrote: I agree that zpool attach and add look similar in their syntax, but if you attempt to add a disk to a redundant config, you'll see an error message similar to the following: # zpool status export pool: export state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM export ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 c1t4d0 ONLINE 0 0 0 errors: No known data errors # zpool add export c1t6d0 invalid vdev specification use '-f' to override the following errors: mismatched replication level: pool uses mirror and new vdev is disk This was one of the first, great debates on ZFS-discuss http://www.opensolaris.org/jive/thread.jspa?messageID=15363#15363 I believe all possible observations are made in that thread. Of particular note, humans train themselves to use a -f (or for kill, -9 :-) as a not-optional part of the command string. It really doesn't matter what the command is named, you will train yourself to go to the forced option. If you want to protect yourself from yourself, begin with yourself. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS dedup clarification
Hy all, On Fri, Nov 27, 2009 at 11:08 AM, Chavdar Ivanov ci4...@gmail.com wrote: Hi, I BFUd successfully snv_128 over snv_125: --- # cat /etc/release Solaris Express Community Edition snv_125 X86 Copyright 2009 Sun Microsystems, Inc. All Rights Reserved. Use is subject to license terms. Assembled 05 October 2009 # uname -a SunOS cheeky 5.11 snv_128 i86pc i386 i86pc I'm asking when dedupe feature will be added on Solaris 10.x. I recently update my Solaris 10.x box on production with LiveUpgrade (the previous was Update 4): # cat /etc/release Solaris 10 10/09 s10s_u8wos_08a SPARC Copyright 2009 Sun Microsystems, Inc. All Rights Reserved. Use is subject to license terms. Assembled 16 September 2009 # uname -a SunOS rmims03 5.10 Generic_141444-09 sun4v sparc SUNW,SPARC-Enterprise-T5120 Thanks Cesare -- Ogden Nash - The trouble with a kitten is that when it grows up, it's always a cat. - http://www.brainyquote.com/quotes/authors/o/ogden_nash.html ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Problems while resilvering
Hi Matthias, I'm not sure I understand all the issues that are going on in this configuration, but I don't see that you used the zpool replace command to complete physical replacement of the failed disk, which would look like this: # zpool replace performance c1t3d0 Then run zpool clear to clear the pool errors. Thanks, Cindy On 12/06/09 15:29, Matthias Appel wrote: Hi, i have go a problem with a zfs pool which wont resilver correctly. The pool consists of two two-way mirrors. One of the disks reportet checksum errors and fell out of the pool. I replaced the faulty harddisk and switched it to available via cfgadm. The resilver started immediately but gave me checksum errors upon completeing like this: pool: performance state: DEGRADED status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scrub: resilver completed after 3h4m with 1 errors on Sun Dec 6 18:27:37 2009 config: NAME STATE READ WRITE CKSUM performance DEGRADED 0 0 3 mirror ONLINE 0 0 0 c1t0d0ONLINE 0 0 0 c1t1d0ONLINE 0 0 0 mirror DEGRADED 0 0 6 c1t2d0ONLINE 0 0 8 256K resilvered replacing DEGRADED 6 0 0 c1t3d0s0/o FAULTED 0 0 0 corrupted data c1t3d0 ONLINE 0 0 6 445G resilvered errors: Permanent errors have been detected in the following files: /performance/VIRUSWALL/Ubuntu-01.vmdk I deleted the file in question (I have a backup of the file) and did a zpool clear performance. Resilvering started again and after completion i got this: r...@storage:/performance/VIRUSWALL# zpool status -v pool: performance state: DEGRADED status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scrub: resilver in progress for 0h11m, 7.35% done, 2h27m to go config: NAME STATE READ WRITE CKSUM performance DEGRADED 0 0 0 mirror ONLINE 0 0 0 c1t0d0ONLINE 0 0 0 c1t1d0ONLINE 0 0 0 mirror DEGRADED 0 0 0 c1t2d0ONLINE 0 0 1 128K resilvered replacing DEGRADED 0 0 0 c1t3d0s0/o FAULTED 0 0 0 corrupted data c1t3d0 ONLINE 0 0 0 33.7G resilvered errors: Permanent errors have been detected in the following files: performance/VIRUSWALL:0x88 Then I tried to rollback a snapshot of the zfs in question and did a zpool clear again because I hoped zfs resilver will complete successfully but it did not. I tried to destroy performance/VIRUSWALL but it said the dataset is busy. I don't know why the dataset is busy because i unmounted all NFS mounts and leaved the directory in the ssh shell. I tired a zpool clear again and resilvering started again and now I get this: r...@storage:/performance/VIRUSWALL# zpool status -v pool: performance state: DEGRADED status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scrub: resilver in progress for 0h39m, 12.21% done, 4h43m to go config: NAME STATE READ WRITE CKSUM performance DEGRADED 0 0 0 mirror ONLINE 0 0 0 c1t0d0ONLINE 0 0 0 c1t1d0ONLINE 0 0 0 mirror DEGRADED 0 0 0 c1t2d0ONLINE 0 0 3 384K resilvered replacing DEGRADED 0 0 0 c1t3d0s0/o FAULTED 0 0 0 corrupted data c1t3d0 ONLINE 0 0 0 56.5G resilvered errors: Permanent errors have been detected in the following files: 0x24e:0x88 Can anybody tell me how to resilver the pool in question and to get rid of c1t3d0s0/o which ist the old/defective harddisk. I dont't mind to delete the files with errors, I only want to get a consistent zpool again. Please tell me what I did wrong. I had a problem with a defective harddisk before...I replaced the harddisks as usual (configured via
Re: [zfs-discuss] Expected ZFS behavior?
and if you don't like it, you can use the zfs:zfs_arc_max tunable in /etc/system to set a maximum amount of memory to be used prior to a write. Oops. Bad cut-n-paste. That should have been zfs:zfs_write_limit_override So I am currently using * Set ZFS maximum TXG group size to 393216 set zfs:zfs_write_limit_override = 0xea60 I have a DAS array with nvram so I enabled zfs_nocacheflush = 1 and it made a world of difference in performance. Does the LSI HBA have any nvram to make this tuning acceptable? Is this setting acceptable as I understood the Evil Tuning Guide? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] nodiratime support in ZFS?
I was catching up on old e-mail on this list, and came across a blog posting from Henrik Johansson: http://sparcv9.blogspot.com/2009/10/curious-case-of-strange-arc.html it tells of his woes with a fragmented /var/pkg/downloads combined with atime updates. I see the same problem on my servers, e.g. $ time du -s /var/pkg/download 1614308 /var/pkg/download real11m50.682s $ time du -s /var/pkg/download 1614308 /var/pkg/download real12m03.395s on this server, increasing arc_meta_limit wouldn't help, but I think a newer kernel would be more aggressive (this is 2008.11). arc_meta_used = 262 MB arc_meta_limit = 2812 MB arc_meta_max = 335 MB turning off atime helps: real 8m06.563s in this test case, running du(1), turning off atime altogether isn't really needed, it would suffice to turn off atime updates on directories. in Linux, this can be achieved with the mount option nodiratime. if ZFS had it, I guess it would be a new value for the atime property, nodir or somesuch. I quite often find it useful to have access to atime information to see if files have been read, for forensic purposes, for debugging, etc. so I am loath to turn it off. however, atime on directories can hardly ever be used for anything -- you have to take really good care not to trigger an update just checking the atime, and even if you do get a proper reading, there are so many tree traversing utilities that the information value is low. it is quite unlikely that any applications break in a nodiratime mode, and few people should have any qualms enabling it. Santa, are you listening? :-) -- Kjetil T. Homme Redpill Linpro AS - Changing the game ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How can we help fix MPT driver post build 129
To be fair, I think it's obvious that Sun people are looking into it and that users are willing to help diagnose and test. There were requests for particular data in those threads you linked to, have you sent yours? It might help them find a pattern in the errors. I understand the frustration that it hasn't been fixed in a couple builds that they have been aware of it, but it could be a very tricky problem. It also sounds like it's not reproducible on Sun hardware, so they have to get cards and such as well. It's also less urgent now that they have identified a workaround that works for most of us. While disabling MSIs is not optimal, it does help a lot. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Pool resize
Hi, I've read before regarding zpool size increase by replacing the vdevs. The initial pool was a raidz2 with 4 640GB disks. I've replaced each disk with 1TB size by taking it out, inserting the new disk, doing cfgadm -c configure on port and zpool replace bigpool c6tXd0 The problem is the zpool size is the same (2.33TB raw) as seen below: # zpool list bigpool NAME SIZE ALLOC FREECAP DEDUP HEALTH ALTROOT bigpool 2.33T 1.41T 942G60% 1.00x ONLINE - It should be ~ 3.8-3.9 TB, right? I've performed a zpool export/import, but to no avail. I'm running OpenSolaris 128a Here is the zpool status: # zpool status bigpool pool: bigpool state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM bigpool ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 c6t2d0 ONLINE 0 0 0 c6t3d0 ONLINE 0 0 0 c6t4d0 ONLINE 0 0 0 c6t5d0 ONLINE 0 0 0 errors: No known data errors and here are the disks: # format /dev/null Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c6t0d0 DEFAULT cyl 60798 alt 2 hd 255 sec 63 /p...@0,0/pci8086,3...@1f,2/d...@0,0 1. c6t1d0 DEFAULT cyl 60797 alt 2 hd 255 sec 63 /p...@0,0/pci8086,3...@1f,2/d...@1,0 2. c6t2d0 ATA-SAMSUNG HD103SJ-00E4-931.51GB /p...@0,0/pci8086,3...@1f,2/d...@2,0 3. c6t3d0 ATA-SAMSUNG HD103SJ-00E4-931.51GB /p...@0,0/pci8086,3...@1f,2/d...@3,0 4. c6t4d0 ATA-SAMSUNG HD103SJ-00E4-931.51GB /p...@0,0/pci8086,3...@1f,2/d...@4,0 5. c6t5d0 ATA-SAMSUNG HD103SJ-00E4-931.51GB /p...@0,0/pci8086,3...@1f,2/d...@5,0 Specify disk (enter its number): Is there something that I am missing? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Pool resize
Did you set autoexpand on? Conversely, did you try doing a 'zpool online bigpool disk' for each disk after the replace completed? On Mon, 7 Dec 2009, Alexandru Pirvulescu wrote: Hi, I've read before regarding zpool size increase by replacing the vdevs. The initial pool was a raidz2 with 4 640GB disks. I've replaced each disk with 1TB size by taking it out, inserting the new disk, doing cfgadm -c configure on port and zpool replace bigpool c6tXd0 The problem is the zpool size is the same (2.33TB raw) as seen below: # zpool list bigpool NAME SIZE ALLOC FREECAP DEDUP HEALTH ALTROOT bigpool 2.33T 1.41T 942G60% 1.00x ONLINE - It should be ~ 3.8-3.9 TB, right? I've performed a zpool export/import, but to no avail. I'm running OpenSolaris 128a Here is the zpool status: # zpool status bigpool pool: bigpool state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM bigpool ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 c6t2d0 ONLINE 0 0 0 c6t3d0 ONLINE 0 0 0 c6t4d0 ONLINE 0 0 0 c6t5d0 ONLINE 0 0 0 errors: No known data errors and here are the disks: # format /dev/null Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c6t0d0 DEFAULT cyl 60798 alt 2 hd 255 sec 63 /p...@0,0/pci8086,3...@1f,2/d...@0,0 1. c6t1d0 DEFAULT cyl 60797 alt 2 hd 255 sec 63 /p...@0,0/pci8086,3...@1f,2/d...@1,0 2. c6t2d0 ATA-SAMSUNG HD103SJ-00E4-931.51GB /p...@0,0/pci8086,3...@1f,2/d...@2,0 3. c6t3d0 ATA-SAMSUNG HD103SJ-00E4-931.51GB /p...@0,0/pci8086,3...@1f,2/d...@3,0 4. c6t4d0 ATA-SAMSUNG HD103SJ-00E4-931.51GB /p...@0,0/pci8086,3...@1f,2/d...@4,0 5. c6t5d0 ATA-SAMSUNG HD103SJ-00E4-931.51GB /p...@0,0/pci8086,3...@1f,2/d...@5,0 Specify disk (enter its number): Is there something that I am missing? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss Regards, markm ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Pool resize
On Mon, Dec 7, 2009 at 3:41 PM, Alexandru Pirvulescu sigx...@gmail.com wrote: I've read before regarding zpool size increase by replacing the vdevs. The initial pool was a raidz2 with 4 640GB disks. I've replaced each disk with 1TB size by taking it out, inserting the new disk, doing cfgadm -c configure on port and zpool replace bigpool c6tXd0 The problem is the zpool size is the same (2.33TB raw) as seen below: # zpool list bigpool NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT bigpool 2.33T 1.41T 942G 60% 1.00x ONLINE - It should be ~ 3.8-3.9 TB, right? An autoexpand property was added a few months ago for zpools. This needs to be turned on to enable the automatic vdev expansion. For example, # zpool set autoexpand=on bigpool Ed Plese ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
Andriy Gapon writes: on 06/12/2009 19:40 Volker A. Brandt said the following: I wanted to add a disk to the tank pool to create a mirror. I accidentally used zpool add ? instead of zpool attach ? and now the disk is added. Is there a way to remove the disk without loosing data? Been there, done that -- at a customer site while showing off ZFS. :-) [...] Yep. My 2 cents -- 'add' and 'attach' are so similar the words that I think that ZFS tools UI designers (if any) should re-consider naming of these commands. Or 'add' command should always be interactive and ask for at least two confirmations that a user knows what he is doing and why. Perhaps, it should include a ZFS micro-exam too. Jokes aside, this is too easy to make a mistake with the consequences that are too hard to correct. Anyone disagrees? I absolutely totally, like, fully agree, man! :-) There should be some interaction and maybe even a display of the new pool structure, followed for a confirmation prompt. This could be overridden with the already well-established -f flag. Regards -- Volker -- Volker A. Brandt Consulting and Support for Sun Solaris Brandt Brandt Computer GmbH WWW: http://www.bb-c.de/~vab/ Am Wiesenpfad 6, 53340 Meckenheim Email: v...@bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgröße: 45 Geschäftsführer: Rainer J. H. Brandt und Volker A. Brandt ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Pool resize
Thank you. That fixed the problem. All the tutorials on Internet didn't mention autoexpand. Again, thank you everybody for the quick reply and solving my problem. Alex On Dec 7, 2009, at 11:48 PM, Ed Plese wrote: On Mon, Dec 7, 2009 at 3:41 PM, Alexandru Pirvulescu sigx...@gmail.com wrote: I've read before regarding zpool size increase by replacing the vdevs. The initial pool was a raidz2 with 4 640GB disks. I've replaced each disk with 1TB size by taking it out, inserting the new disk, doing cfgadm -c configure on port and zpool replace bigpool c6tXd0 The problem is the zpool size is the same (2.33TB raw) as seen below: # zpool list bigpool NAME SIZE ALLOC FREECAP DEDUP HEALTH ALTROOT bigpool 2.33T 1.41T 942G60% 1.00x ONLINE - It should be ~ 3.8-3.9 TB, right? An autoexpand property was added a few months ago for zpools. This needs to be turned on to enable the automatic vdev expansion. For example, # zpool set autoexpand=on bigpool Ed Plese ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
Jokes aside, this is too easy to make a mistake with the consequences that are too hard to correct. Anyone disagrees? No, and this sums up the situation nicely, in that there are two parallel paths toward a resolution: - make the mistake harder to make (various ideas here) - make the consequences easier to correct (coming) We really can and should do both. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
but if you attempt to add a disk to a redundant config, you'll see an error message similar [..] Doesn't the mismatched replication message help? Not if you're trying to make a single disk pool redundant by adding .. er, attaching .. a mirror; then there won't be such a warning, however effective that warning might or might not be otherwise. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
On Dec 7, 2009, at 2:23 PM, Daniel Carosone wrote: but if you attempt to add a disk to a redundant config, you'll see an error message similar [..] Doesn't the mismatched replication message help? Not if you're trying to make a single disk pool redundant by adding .. er, attaching .. a mirror; then there won't be such a warning, however effective that warning might or might not be otherwise. Not a problem because you can then detach the vdev and add it. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
On Mon, Dec 7, 2009 at 12:42 PM, Cindy Swearingen cindy.swearin...@sun.com wrote: I agree that zpool attach and add look similar in their syntax, but if you attempt to add a disk to a redundant config, you'll see an error message similar to the following: # zpool status export pool: export state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM export ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 c1t4d0 ONLINE 0 0 0 errors: No known data errors # zpool add export c1t6d0 invalid vdev specification use '-f' to override the following errors: mismatched replication level: pool uses mirror and new vdev is disk Doesn't the mismatched replication message help? When adding a disk to a single-disk pool, this message isn't given and the add proceeds without any warning and without the need to force it: # cd /tmp # mkfile 256m f1 f2 # zpool create testpool /tmp/f1 # zpool add testpool /tmp/f2 # zpool status testpool pool: testpool state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM testpoolONLINE 0 0 0 /tmp/f1 ONLINE 0 0 0 /tmp/f2 ONLINE 0 0 0 errors: No known data errors Would it be beneficial to have a command line option to zpool that would only preview or do a dry-run through the changes, but instead just display what the pool would look like after the operation and leave the pool unchanged? For those that very rarely make pool changes, getting in the habit of always using an option like this might be a good way to ensure the change is really what is desired. Some information that might be nice to see would be the before and after versions of zpool list, the zpool status, and what command could be run to reverse the change, or a warning if the change is irreversible like the case with zpool add. Ed Plese ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
On Dec 7, 2009, at 11:23 PM, Daniel Carosone wrote: but if you attempt to add a disk to a redundant config, you'll see an error message similar [..] Doesn't the mismatched replication message help? Not if you're trying to make a single disk pool redundant by adding .. er, attaching .. a mirror; then there won't be such a warning, however effective that warning might or might not be otherwise. That's exactly where I made the mistake, a single disk pool and I wanted to attach another disk to build a mirror. I'm now using zfs send and receive to backup the disk to another zpool. Then I want to rename the old and the new zpools and mount the new zpool. Then I can destroy the old zpool, attach the free disks from the old zpool creating a mirror and in the end detach the extra disk from new mirrored zpool. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
On Dec 7, 2009, at 11:32 PM, Ed Plese wrote: On Mon, Dec 7, 2009 at 12:42 PM, Cindy Swearingen cindy.swearin...@sun.com wrote: I agree that zpool attach and add look similar in their syntax, but if you attempt to add a disk to a redundant config, you'll see an error message similar to the following: # zpool status export pool: export state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM export ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 c1t4d0 ONLINE 0 0 0 errors: No known data errors # zpool add export c1t6d0 invalid vdev specification use '-f' to override the following errors: mismatched replication level: pool uses mirror and new vdev is disk Doesn't the mismatched replication message help? When adding a disk to a single-disk pool, this message isn't given and the add proceeds without any warning and without the need to force it: # cd /tmp # mkfile 256m f1 f2 # zpool create testpool /tmp/f1 # zpool add testpool /tmp/f2 # zpool status testpool pool: testpool state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM testpoolONLINE 0 0 0 /tmp/f1 ONLINE 0 0 0 /tmp/f2 ONLINE 0 0 0 errors: No known data errors Would it be beneficial to have a command line option to zpool that would only preview or do a dry-run through the changes, but instead just display what the pool would look like after the operation and leave the pool unchanged? For those that very rarely make pool changes, getting in the habit of always using an option like this might be a good way to ensure the change is really what is desired. Some information that might be nice to see would be the before and after versions of zpool list, the zpool status, and what command could be run to reverse the change, or a warning if the change is irreversible like the case with zpool add. I guess the best solution is to make it easy to undo the changes. zfs remove should be just as easy. Ed Plese ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Pool resize
Hi Alex, The SXCE Admin Guide is generally up-to-date on docs.sun.com. The section that covers the autoreplace property and default behavior is here: http://docs.sun.com/app/docs/doc/817-2271/gazgd?a=view Thanks, Cindy On 12/07/09 14:50, Alexandru Pirvulescu wrote: Thank you. That fixed the problem. All the tutorials on Internet didn't mention autoexpand. Again, thank you everybody for the quick reply and solving my problem. Alex On Dec 7, 2009, at 11:48 PM, Ed Plese wrote: On Mon, Dec 7, 2009 at 3:41 PM, Alexandru Pirvulescu sigx...@gmail.com wrote: I've read before regarding zpool size increase by replacing the vdevs. The initial pool was a raidz2 with 4 640GB disks. I've replaced each disk with 1TB size by taking it out, inserting the new disk, doing cfgadm -c configure on port and zpool replace bigpool c6tXd0 The problem is the zpool size is the same (2.33TB raw) as seen below: # zpool list bigpool NAME SIZE ALLOC FREECAP DEDUP HEALTH ALTROOT bigpool 2.33T 1.41T 942G60% 1.00x ONLINE - It should be ~ 3.8-3.9 TB, right? An autoexpand property was added a few months ago for zpools. This needs to be turned on to enable the automatic vdev expansion. For example, # zpool set autoexpand=on bigpool Ed Plese ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
On Mon, Dec 7, 2009 at 4:32 PM, Ed Plese e...@edplese.com wrote: Would it be beneficial to have a command line option to zpool that would only preview or do a dry-run through the changes, but instead just display what the pool would look like after the operation and leave the pool unchanged? For those that very rarely make pool changes, getting in the habit of always using an option like this might be a good way to ensure the change is really what is desired. There I go requesting features that are already there. zpool add already has a -n option that will show the pool configuration that would be used, without actually making any changes. A couple other subcommands accept the -n option as well. Ed Plese ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Live Upgrade Solaris 10 UFS to ZFS boot pre-requisites?
I have a Solaris 10 U5 system massively patched so that it supports ZFS pool version 15 (similar to U8, kernel Generic_141445-09), live upgrade components have been updated to Solaris 10 U8 versions from the DVD, and GRUB has been updated to support redundant menus across the UFS boot environments. I have studied the Solaris 10 Live Upgrade manual (821-0438) and am unable to find any statement which requires/suggests that I live upgrade to U8 with UFS boot before live upgrading to ZFS boot but the page at http://docs.sun.com/app/docs/doc/819-5461/ggpdm?a=view recommends that this process should be used. The two documents do not seem to agree. Given that my system is essentially equivalent to U8, is there any reason to live upgrade to UFS U8 prior to ZFS U8 or can the more direct path be used? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Transaction consistency of ZFS
Because ZFS is transaction, (effectively preserves order), the rename trick will work. If you find the .filename delete create a new .filename and when finish writing rename it to filename. If filename exists you no all writes were completed. If you have a batch system which looks for the file it will not find it until it is renamed. Not that I am a of batch systems which use CPU poll for files existance. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recovering FAULTED zpool
On 2009-Nov-18 11:50:44 +1100, I wrote: I have a zpool on a JBOD SE3320 that I was using for data with Solaris 10 (the root/usr/var filesystems were all UFS). Unfortunately, we had a bit of a mixup with SCSI cabling and I believe that we created a SCSI target clash. The system was unloaded and nothing happened until I ran zpool status at which point things broke. After correcting all the cabling, Solaris panic'd before reaching single user. I wound up installing OpenSolaris snv_128a on some spare disks and this enabled me to recover the data. Thanks to Tim Haley and Victor Latushkin for their assistance. As a first attempt, 'zpool import -F data' said Destroy and re-create the pool from a backup source.. 'zpool import -nFX data' initially ran the system out of swap (I hadn't attached any swap and it only has 8GB RAM): WARNING: /etc/svc/volatile: File system full, swap space limit exceeded INIT: Couldn't write persistent state file `/etc/svc/volatile/init.state'. After rebooting and adding some swap (which didn't seem to ever get used), it did work (though it took several hours - unfortunately, I didn't record exactly how long): # zpool import -nFX data Would be able to return data to its state as of Thu Jan 01 10:00:00 1970. Would discard approximately 369 minutes of transactions. # zpool import -FX data Pool data returned to its state as of Thu Jan 01 10:00:00 1970. Discarded approximately 369 minutes of transactions. cannot share 'data/backup': share(1M) failed cannot share 'data/JumpStart': share(1M) failed cannot share 'data/OS_images': share(1M) failed # I notice that the two times aren't consistent but the data appears to be present and a 'zpool scrub' reported no errors. I have reverted back to Solaris 10 and successfully copied all the data off. -- Peter Jeremy pgpC0sjEufK37.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Problems while resilvering
I accidentially only replied to cindy but I wanted to reply to the list. I don't want to overstrain cindys time...maybe one of the list members can help me as well. -Ursprüngliche Nachricht- Von: Matthias Appel Gesendet: Dienstag, 08. Dezember 2009 03:34 An: 'cindy.swearin...@sun.com' Betreff: AW: [zfs-discuss] Problems while resilvering Hi Cindy, Thanks for your reply. # zpool replace performance c1t3d0 Then run zpool clear to clear the pool errors. Does this mean I don't have to do a cfgadm -c configure sata0/3 (which starts a resilver in my case) but it Is sufficient to do a zpool replace (I do not have a hot-spare)? In the meantime my pool resilvered correctly after 3 or 4 resilvering runs (each initiated by a zpool clear) I don't understand why issuing a zpool clear starts a resilvering. And I stumbled upon another issue: I've worked with software RAID in an Linux environment and I am used that if I relocate a disk to another controller and the disk is found and automatically attached back to the existing RAID set. I just added another controller to my system and the device changed from c1t0d0 to c2d0. Zpool status gives me this: NAMESTATE READ WRITE CKSUM performance DEGRADED 0 0 0 mirrorDEGRADED 0 0 0 c1t0d0 UNAVAIL 0 0 0 cannot open c1t1d0 ONLINE 0 0 0 mirrorONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 How can I replace the unavailable drive wit c2do. I read the manpage of zpool and it only gives me the options to replace a disk with an empty one. If I do a zpool replace c1t0d0 c2d0 it says that I have to use the -f option because the disk was in a zpool before. Do I want to use the -f option (will the existing data be only updated with changed blocks or will a complete resilver Kick in). If not, how can I relocate disks from one controller to another? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS pool unusable after attempting to destroy a dataset with dedup enabled
Howdy, I upgraded to snv_128a from snv_125 . I wanted to do some de-dup testing :). I have two zfs pools: rpool and vault. I upgraded my vault zpool version and turned on dedup on datastore vault/shared_storage. I also turned on gzip compression on this dataset as well. Before I turned on dedup, I made a new datastore and copied all data to vault/shared_storage_temp (just in case something crazy happened to my dedup'd datastore, since dedup is new). I removed all data on my dedup'd datastore and copied all data from my temp datastore. After I realized my space savings wasn't going to be that great, I decided to delete vault/shared_storage dataset. zfs destroy vault/shared_storage This hung, and couldn't be killed. I force rebooted my system, and I couldn't boot into Solaris. It hung at reading zfs config I then booted into single user mode (multiple times) and any zfs or zpool commands froze. I then rebooted to my snv_125 environment. As it should, it ignored my vault zpool, as it's version is higher than it can understand. I forced an zpool export of vault and rebooted, I could then boot back into snv_128 and zpool import listed the pool of vault. However, I cannot import via name or identifier, the command hangs, as well as any additional zfs or zpool commands. I cannot kill or kill -9 the processes. Is there anything I can do to get my pool imported? I haven't done much troubleshooting at all on opensolairs, I'd be happy to run any suggested commands and provide output. Thank you for the assistance. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] freeNAS moves to Linux from FreeBSD
Actually it appears that FreeNAS is forking with planned support for both linux (we can only speculate on the preferred backing file system) and FreeBSD with ZFS as preferred backing file system. In regards to OpenSolaris advocacy for using OpenSolaris vs. FreeBSD, I'm all ears if anybody is bold enough to clutter up this mailing list with it. A quick start from my perspective (and this is no way complete) would be: Basically, I have a need for a modern file systems with snapshots both for internal purposes and to support vmware instances. De-depluciation is a nice idea, but given our size, the balance between risk and dollars makes it easier to just have more disk space. Args for FreeBSD + ZFS: - Limited budget - We are familiar with managing FreeBSD. - We are familiar with tuning FreeBSD. - Licensing model Args against FreeBSD + ZFS: - Stability (?) - Possibly performance (although we have limited needs for CIFS) Args for OpenSolaris + ZFS: - Stability Args against OpenSolaris + ZFS: - Hardware compatibility - Lack of knowledge for tuning and associated costs for training staff to learn 'yet one more operating system' they need to support. - Licensing model On Dec 6, 2009, at 6:28 PM, Gary Gendel wrote: The only reason I thought this news would be of interest is that the discussions had some interesting comments. Basically, there is a significant outcry because zfs was going away. I saw NextentaOS and EON mentioned several times as the path to go. Seem that there is some opportunity for OpenSolaris advocacy in this arena while the topic is hot. Gary -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Accidentally added disk instead of attaching
What about removing attach/deattach and replace it with zpool add [-fn] 'pool' submirror 'device/mirrorname' 'new_device' e.g. NAMESTATE READ WRITE CKSUM rpoolONLINE 0 0 0 mirror-01 ONLINE 0 0 0 c4d0s0 ONLINE 0 0 0 c3d0s0 ONLINE 0 0 0 zpool add rpool submirror mirror-01 c5d0s0 # or zpool add rpool submirror c4d0s0 c5d0s0 zpool remove rpool c5d0s0 Some more examples zpool add 'pool' submirror log-01 c7d0s0 # create a mirror for the Intent Log And may be one day zpool add 'pool' subraidz raidz2-01 c5d0s0 to add extra disk to raidz group and have the disk restriped in the background Which would mean vdev in terms of syntax would support concat (was disk), concat-file (was file), mirror, submirror, raidz, raidzN, subraidz (one day), spare, log, cache -- And change zpool add rpool disk c5d0s0 to zpool add rpool concat c5d0s0 # instead of disk use concat or zpool add rpool concatfile path to file # instead of file Cheers -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] freeNAS moves to Linux from FreeBSD
Michael DeMan (OA) wrote: Actually it appears that FreeNAS is forking with planned support for both linux (we can only speculate on the preferred backing file system) and FreeBSD with ZFS as preferred backing file system. In regards to OpenSolaris advocacy for using OpenSolaris vs. FreeBSD, I'm all ears if anybody is bold enough to clutter up this mailing list with it. A quick start from my perspective (and this is no way complete) would be: Basically, I have a need for a modern file systems with snapshots both for internal purposes and to support vmware instances. De-depluciation is a nice idea, but given our size, the balance between risk and dollars makes it easier to just have more disk space. Args for FreeBSD + ZFS: - Limited budget - We are familiar with managing FreeBSD. - We are familiar with tuning FreeBSD. - Licensing model Args against FreeBSD + ZFS: - Stability (?) - Possibly performance (although we have limited needs for CIFS) Args for OpenSolaris + ZFS: - Stability Args against OpenSolaris + ZFS: - Hardware compatibility - Lack of knowledge for tuning and associated costs for training staff to learn 'yet one more operating system' they need to support. - Licensing model I'd have to go back over the CDDL, but my understanding is that since ZFS is CDDL, the inclusion of CDDL zfs code in the FreeBSD FreeNAS setup means that you have essentially the same licensing model as the all-CDDL OpenSolaris. In any case, the CDDL is very liberal (it's a variation of the MPL), so the differences should be very minor in terms of real impact on a business model (that is, vs a BSD-license). Tuning for Solaris is definitely a bit more wizardly magic than for FreeBSD, but there are significant mitigating factors: (1) there is now very good [if very dense] documentation on the tunables (and what they mean) for much of the Solaris kernel. (e.g. http://www.informit.com/store/product.aspx?isbn=0131482092 ) (2) Most tunable needs in something like FreeNAS apply to ZFS, which means that BOTH FreeBSD and OpenSolaris would need to be tuned; therefore, there is little difference between using either OS in terms of tunables. (3) Solaris itself is generally very, very good at NOT needing to be tuned. IMHO, it's probably the Best OS in these terms, meaning that the need to tune is significantly lower than other OSes, and thus, knowing HOW to tune is generally much less important. One other advantage of using OpenSolaris over FreeBSD is simply First Mover - that is, fixes show up first in OpenSolaris, it has a much larger user community around ZFS, and the primary engineers are using OpenSolaris as their development and testing platforms. I would also place the COMSTAR stuff as a major reason for a NAS-project to consider OpenSolaris over other OSes. It's just s nice. :-) That all said, in your specific case where you have significant in-house FreeBSD knowledge, I would stick with it for the time being. The differences for something like FreeNAS are relatively minor, and it's better to Go With What You Know. Exploring OpenSolaris for a future migration would be good, but for right now, I'd stick to FreeBSD. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
