Re: [zfs-discuss] zfs-discuss Digest, Vol 48, Issue 129

2009-12-22 Thread Amit G
Hi dan, Your problem regarding zpool add tank c1d1 got solved or not? If not i've one solution ,Whenever you add any disk to the pool you have to give full path of that disk, even if you are in current directory. So, could you try it by giving full path of c1d1?.. On Thu, Oct 29, 2009 at 11:38

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Ross Walker
On Dec 21, 2009, at 11:56 PM, Roman Naumenko ro...@naumenko.ca wrote: On Dec 21, 2009, at 4:09 PM, Michael Herf mbh...@gmail.com wrote: Anyone who's lost data this way: were you doing weekly scrubs, or did you find out about the simultaneous failures after not touching the bits for

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Marty Scholes
Hi Ross, What about old good raid10? It's a pretty reasonable choice for heavy loaded storages, isn't it? I remember when I migrated raidz2 to 8xdrives raid10 the application administrators were just really happy with the new access speed. (we didn't use stripped raidz2

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Bob Friesenhahn
On Tue, 22 Dec 2009, Ross Walker wrote: Raid10 provides excellent performance and if performance is a priority then I recommend it, but I was under the impression that resiliency was the priority, as raidz2/raidz3 provide greater resiliency for a sacrifice in performance. Why are people

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Marty Scholes
Bob Friesenhahn wrote: Why are people talking about RAID-5, RAID-6, and RAID-10 on this list? This is the zfs-discuss list and zfs does not do RAID-5, RAID-6, or RAID-10. Applying classic RAID terms to zfs is just plain wrong and misleading since zfs does not directly implement these

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Roman Naumenko
On Tue, 22 Dec 2009, Ross Walker wrote: Applying classic RAID terms to zfs is just plain wrong and misleading since zfs does not directly implement these classic RAID approaches even though it re-uses some of the algorithms for data recovery. Details do matter. Bob -- Bob

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Bob Friesenhahn
On Tue, 22 Dec 2009, Marty Scholes wrote: That's not entirely true, is it? * RAIDZ is RAID5 + checksum + COW * RAIDZ2 is RAID6 + checksum + COW * A stack of mirror vdevs is RAID10 + checksum + COW These are layman's simplifications that no one here should be comfortable with. Zfs borrows

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Joerg Moellenkamp
On 22.12.09 18:42, Roman Naumenko wrote: On Tue, 22 Dec 2009, Ross Walker wrote: Applying classic RAID terms to zfs is just plain wrong and misleading since zfs does not directly implement these classic RAID approaches even though it re-uses some of the algorithms for data recovery. Details do

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Bob Friesenhahn
On Tue, 22 Dec 2009, Roman Naumenko wrote: raid6 is raid6, not matter how you name it: raidz2, raid-dp, raid-ADG or somehow else. Sounds nice, but it's is just buzzwords. It is true that many vendors like to make their storage array seem special, but references to RAID6 when describing

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Travis Tabbal
Interesting discussion. I know the bias here is generally toward enterprise users. I was wondering if the same recommendations hold for home users that are generally more price sensitive. I'm currently running OpenSolaris on a system with 12 drives. I had split them into 3 sets of 4 raidz1

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Marty Scholes
Bob Friesenhahn wrote: On Tue, 22 Dec 2009, Marty Scholes wrote: That's not entirely true, is it? * RAIDZ is RAID5 + checksum + COW * RAIDZ2 is RAID6 + checksum + COW * A stack of mirror vdevs is RAID10 + checksum + COW These are layman's simplifications that no one here should be

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Toby Thain
On 22-Dec-09, at 12:42 PM, Roman Naumenko wrote: On Tue, 22 Dec 2009, Ross Walker wrote: Applying classic RAID terms to zfs is just plain wrong and misleading since zfs does not directly implement these classic RAID approaches even though it re-uses some of the algorithms for data

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Richard Elling
On Dec 22, 2009, at 11:49 AM, Toby Thain wrote: On 22-Dec-09, at 12:42 PM, Roman Naumenko wrote: On Tue, 22 Dec 2009, Ross Walker wrote: Applying classic RAID terms to zfs is just plain wrong and misleading since zfs does not directly implement these classic RAID approaches even though it

[zfs-discuss] Mirror of SAN Boxes with ZFS ? (split site mirror)

2009-12-22 Thread Lutz Schumann
Hello, I'm thinking about a setup that looks like this: - 2 headnodes with FC connectivity (OpenSolaris) - 2 backend FC srtorages (Disk Shelves with RAID Controllers presenting a huge 15 TB RAID5) - 2 datacenters (distance 1 km with dark fibre) - one headnode and one storage in each data

[zfs-discuss] Why would some disks in a raidz use partitions and others not?

2009-12-22 Thread Galen
I have a raidz1 that I have been trying to recover. I am missing one disk due to catastrophic mechanical failure, but the other disks are fully functional. The problem is that I cannot import the zpool, despite all the disks being present. Even with -f, zfs recovery mode set in the kernel

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread James Risner
ttabbal: If I understand correctly, raidz{1} is 1 drive protection and space is (drives - 1) available. Raidz2 is 2 drive protection and space is (drives - 2) etc. Same for raidz3 being 3 drive protection. Everything I've seen you should stay around 6-9 drives for raidz, so don't

Re: [zfs-discuss] Why would some disks in a raidz use partitions and others not?

2009-12-22 Thread James Risner
Can you post a zpool import -f for us to see? One thing I ran into recently is that if the drives arrangement was changed (like drives swapped) it can't recover. I moved an 8 drive array recently, and didn't worry about the order of the drives. It could not be mounted without reordering the

Re: [zfs-discuss] Why would some disks in a raidz use partitions and others not?

2009-12-22 Thread Galen
I am on different hardware, thus I cannot restore the drive configuration exactly. I have been able to shuffle and re-import disks in the past with different zpools, even without an export. Does anybody have more experience with what scenarios cause a failure to import with the disks being

Re: [zfs-discuss] Why would some disks in a raidz use partitions and others not?

2009-12-22 Thread James Risner
galenz: I am on different hardware, thus I cannot restore the drive configuration exactly. Actually, you can learn most of it, if not all of it you need. Do zpool import -f with no pool name and it should dump the issue with the pool (what is making it fail.) If that doesn't contain

[zfs-discuss] getting decent NFS performance

2009-12-22 Thread Charles Hedrick
We have a server using Solaris 10. It's a pair of systems with a shared J4200, with Solaris cluster. It works very nicely. Solaris cluster switches over transparently. However as an NFS server it is dog-slow. This is the usual synchronous write problem. Setting zfs_disable fixes the problem.

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Bob Friesenhahn
On Tue, 22 Dec 2009, James Risner wrote: I do consider RAID5 as 'Stripeset with an interleaved Parity', so I don't agree with the strong objection in this thread by many about the use of RAID5 to describe what raidz does. I don't think many particularly care about the nuanced differences

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Marty Scholes
risner wrote: If I understand correctly, raidz{1} is 1 drive protection and space is (drives - 1) available. Raidz2 is 2 drive protection and space is (drives - 2) etc. Same for raidz3 being 3 drive protection. Yes. Everything I've seen you should stay around 6-9 drives for raidz, so

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Travis Tabbal
Everything I've seen you should stay around 6-9 drives for raidz, so don't do a raidz3 with 12 drives. Instead make two raidz3 with 6 drives each (which is (6-3)*1.5 * 2 = 9 TB array.) So the question becomes, why? If it's performance, I can live with lower IOPS and max throughput. If it's

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Toby Thain
On 22-Dec-09, at 3:33 PM, James Risner wrote: ... Joerg Moellenkamp: I do consider RAID5 as 'Stripeset with an interleaved Parity', so I don't agree with the strong objection in this thread by many about the use of RAID5 to describe what raidz does. I don't think many particularly

Re: [zfs-discuss] Why would some disks in a raidz use partitions and others not?

2009-12-22 Thread Richard Elling
On Dec 22, 2009, at 12:48 PM, Galen wrote: I am on different hardware, thus I cannot restore the drive configuration exactly. I have been able to shuffle and re-import disks in the past with different zpools, even without an export. Does anybody have more experience with what scenarios

[zfs-discuss] cfgadm, luxadm changes in OpenSolaris Development snv_128a

2009-12-22 Thread zfsmonk
We recently upgraded last week to OpenSolaris Development snv_128a and all things went fine, zpool still works, etc. Our disks come off a Hitachi Disk Array and we have 2 qLogic fiber cards in our server. cfgadm -al use to report the cards and luxadm probe use to return all the disks on the

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Ross Walker
On Dec 22, 2009, at 11:46 AM, Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Tue, 22 Dec 2009, Ross Walker wrote: Raid10 provides excellent performance and if performance is a priority then I recommend it, but I was under the impression that resiliency was the priority, as

Re: [zfs-discuss] getting decent NFS performance

2009-12-22 Thread Erik Trimble
Charles Hedrick wrote: We have a server using Solaris 10. It's a pair of systems with a shared J4200, with Solaris cluster. It works very nicely. Solaris cluster switches over transparently. However as an NFS server it is dog-slow. This is the usual synchronous write problem. Setting

Re: [zfs-discuss] getting decent NFS performance

2009-12-22 Thread Charles Hedrick
Thanks. That's what I was looking for. Yikes! I hadn't realized how expensive the Zeus is. We're using Solaris cluster, so if the system goes down, the other one takes over. That means that if the ZIL is on a local disk, we lose it in a crash. Might as well just set zil_disable (something I'm

Re: [zfs-discuss] getting decent NFS performance

2009-12-22 Thread Charles Hedrick
It turns out that our storage is currently being used for * backups of various kinds, run daily by cron jobs * saving old log files from our production application * saving old versions of java files from our production application Most of the usage is write-only, and a fair amount of it

Re: [zfs-discuss] getting decent NFS performance

2009-12-22 Thread Ross Walker
On Dec 22, 2009, at 8:40 PM, Charles Hedrick hedr...@rutgers.edu wrote: It turns out that our storage is currently being used for * backups of various kinds, run daily by cron jobs * saving old log files from our production application * saving old versions of java files from our production

[zfs-discuss] image-update failed; ZFS returned an error

2009-12-22 Thread Garen Parham
Never seen this before: # pkg image-update DOWNLOAD PKGS FILESXFER (MB) Completed 2/2 194/19464.0/64.0 PHASEACTIONS Removal Phase4/4

Re: [zfs-discuss] getting decent NFS performance

2009-12-22 Thread Richard Elling
On Dec 22, 2009, at 5:40 PM, Charles Hedrick wrote: It turns out that our storage is currently being used for * backups of various kinds, run daily by cron jobs * saving old log files from our production application * saving old versions of java files from our production application Most of

Re: [zfs-discuss] getting decent NFS performance

2009-12-22 Thread Ross Walker
On Dec 22, 2009, at 8:58 PM, Richard Elling richard.ell...@gmail.com wrote: On Dec 22, 2009, at 5:40 PM, Charles Hedrick wrote: It turns out that our storage is currently being used for * backups of various kinds, run daily by cron jobs * saving old log files from our production

[zfs-discuss] Zones on shared storage - a warning

2009-12-22 Thread Mike Gerdts
I've been playing around with zones on NFS a bit and have run into what looks to be a pretty bad snag - ZFS keeps seeing read and/or checksum errors. This exists with S10u8 and OpenSolaris dev build snv_129. This is likely a blocker for anything thinking of implementing parts of Ed's Zones on

Re: [zfs-discuss] getting decent NFS performance

2009-12-22 Thread Bob Friesenhahn
On Tue, 22 Dec 2009, Ross Walker wrote: I think zil_disable may actually make sense. How about a zil comprised of two mirrored iSCSI vdevs formed from a SSD on each box? I would not have believed that this is a useful idea except that I have seen IOPS offload to a server on the network

Re: [zfs-discuss] getting decent NFS performance

2009-12-22 Thread Ross Walker
On Dec 22, 2009, at 9:08 PM, Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Tue, 22 Dec 2009, Ross Walker wrote: I think zil_disable may actually make sense. How about a zil comprised of two mirrored iSCSI vdevs formed from a SSD on each box? I would not have believed that this

[zfs-discuss] Recovering ZFS stops after syseventconfd can't fork

2009-12-22 Thread Paul Armstrong
I have a machine connected to an HDS with a corrupted pool. While running zpool import -nfFX on the pool, it spawns a large number of zfsdle processes and eventually the machine hangs for 20-30 seconds, spits out error messages zfs: [ID 346414 kern.warning] WARNING: Couldn't create process

Re: [zfs-discuss] Zones on shared storage - a warning

2009-12-22 Thread Mike Gerdts
On Tue, Dec 22, 2009 at 8:02 PM, Mike Gerdts mger...@gmail.com wrote: I've been playing around with zones on NFS a bit and have run into what looks to be a pretty bad snag - ZFS keeps seeing read and/or checksum errors.  This exists with S10u8 and OpenSolaris dev build snv_129.  This is likely

[zfs-discuss] invalid mountpoint 'mountpoint=legacy' ?

2009-12-22 Thread Dennis Clarke
Anyone seen this odd message ? It seems a tad counter intuitive. # uname -a SunOS gamma 5.11 snv_126 sun4u sparc SUNW,Sun-Fire-480R # cat /etc/release Solaris Express Community Edition snv_126 SPARC Copyright 2009 Sun Microsystems, Inc. All Rights Reserved.

Re: [zfs-discuss] invalid mountpoint 'mountpoint=legacy' ?

2009-12-22 Thread Dennis Clarke
I hate it when I do that .. 30 secs later I see -m mountpoint which is a Property but not specified as -o foo-bar format. erk # ptime zpool create -f -o autoreplace=on -o version=10 \ -m legacy \ fibre01 mirror c2t0d0 c3t16d0 \ mirror c2t1d0 c3t17d0 \ mirror c2t2d0 c3t18d0 \ mirror c2t3d0

Re: [zfs-discuss] Recovering ZFS stops after syseventconfd can't fork

2009-12-22 Thread Anton B. Rang
Something over 8000 sounds vaguely like the default maximum process count. What does 'ulimit -a' show? I don't know why you're seeing so many zfsdle processes, though — sounds like a bug to me. -- This message posted from opensolaris.org ___

Re: [zfs-discuss] Recovering ZFS stops after syseventconfd can't fork

2009-12-22 Thread Paul Armstrong
bash-4.0# ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited open files (-n) 256 pipe size(512 bytes, -p) 10 stack size (kbytes, -s) 10240 cpu time

Re: [zfs-discuss] Recovering ZFS stops after syseventconfd can't fork

2009-12-22 Thread Paul Armstrong
I'm surprised at the number as well. Running it again, I'm seeing it jump fairly high just before the fork errors: bash-4.0# ps -ef | grep zfsdle | wc -l 20930 (the next run of ps failed due to the fork error). So maybe it is running out of processes. ZFS file data from ::memstat just went

Re: [zfs-discuss] raidz data loss stories?

2009-12-22 Thread Frank Cusack
On December 21, 2009 10:45:29 PM -0500 Ross Walker rswwal...@gmail.com wrote: Scrubbing on a routine basis is good for detecting problems early, but it doesn't solve the problem of a double failure during resilver. As the size of disks become huge the chance of a double failure during

Re: [zfs-discuss] cfgadm, luxadm changes in OpenSolaris Development snv_128a

2009-12-22 Thread Tim Cook
On Tue, Dec 22, 2009 at 5:38 PM, zfsmonk leftlanep...@gmail.com wrote: We recently upgraded last week to OpenSolaris Development snv_128a and all things went fine, zpool still works, etc. Our disks come off a Hitachi Disk Array and we have 2 qLogic fiber cards in our server. cfgadm -al use

Re: [zfs-discuss] getting decent NFS performance

2009-12-22 Thread Erik Trimble
Charles Hedrick wrote: Is ISCSI reliable enough for this? YES. The original idea is a good one, and one that I'd not thought of. The (old) iSCSI implementation is quite mature, if not anywhere as nice (feature/flexibility-wise) as the new COMSTAR stuff. I'm thinking that just putting