Re[6]: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Hello Erik, Wednesday, February 28, 2007, 12:55:24 AM, you wrote: ET> Honestly, no, I don't consider UFS a modern file system. :-) ET> It's just not in the same class as JFS for AIX, xfs for IRIX, or even ET> VxFS. The point is that fsck was due to an array corrupting data. IMHO it would hit JFS, XFS or VxFS as bad as UFS if not worse. -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
> With modern journalling filesystems, I've never had to fsck anything or > run a filesystem repair. Ever. On any of my SAN stuff. you will.. even if the SAN is perfect, you will hit bugs in the filesystem code.. from lots of rsync hard links or like this one from raidtools last week: Feb 9 05:38:39 orbit kernel: mptbase: ioc2: IOCStatus(0x0043): SCSI Device Not There Feb 9 05:38:39 orbit kernel: md: write_disk_sb failed for device sdp1 Feb 9 05:38:39 orbit kernel: md: errors occurred during superblock update, repeating Feb 9 05:39:01 orbit kernel: raid6: Disk failure on sdp1, disabling device. Operation continuing on 13 devices Feb 9 05:39:09 orbit kernel: mptscsi: ioc2: attempting task abort! (sc=cb17c800) Feb 9 05:39:10 orbit kernel: RAID6 conf printout: Feb 9 05:39:10 orbit kernel: --- rd:14 wd:13 fd:1 Feb 9 05:44:37 orbit kernel: EXT3-fs error (device dm-0): ext3_readdir: bad entry in directory #10484: rec_len %$ Feb 9 05:44:37 orbit kernel: Aborting journal on device dm-0. Feb 9 05:44:37 orbit kernel: ext3_abort called. Feb 9 05:44:37 orbit kernel: EXT3-fs error (device dm-0): ext3_journal_start_sb: Detected aborted journal Feb 9 05:44:37 orbit kernel: Remounting filesystem read-only Feb 9 05:44:37 orbit kernel: attempt to access beyond end of device Feb 9 05:44:44 orbit kernel: oom-killer: gfp_mask=0xd0 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Re[4]: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Honestly, no, I don't consider UFS a modern file system. :-) It's just not in the same class as JFS for AIX, xfs for IRIX, or even VxFS. -Erik On Wed, 2007-02-28 at 00:40 +0100, Robert Milkowski wrote: > Hello Erik, > > Tuesday, February 27, 2007, 5:47:42 PM, you wrote: > > ET> > > > ET> The answer is: insufficient data. > > > ET> With modern journalling filesystems, I've never had to fsck anything or > ET> run a filesystem repair. Ever. On any of my SAN stuff. > > I'm not sure if you consider UFS in S10 as a modern journalling > filesystem but in case you do: > > Feb 13 12:03:16 ufs: [ID 879645 kern.notice] NOTICE: /opt/d1635: > unexpected free inode 54305084, run fsck(1M) -o f > > This file system is on a medium large array (IBM) in a SAN > environment. > > > -- Erik Trimble Java System Support Mailstop: usca14-102 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Hi Przemol, I think migration is a really important feature...think I said that... ;-) SAN/RAID is not awful...frankly there's not been better solution (outside of NetApp's WAFL) till ZFS. SAN/RAID just has its own reliability issues you accept unless you don't have toZFS :-) -J On 2/27/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: On Thu, Feb 22, 2007 at 12:21:50PM -0700, Jason J. W. Williams wrote: > Hi Przemol, > > I think Casper had a good point bringing up the data integrity > features when using ZFS for RAID. Big companies do a lot of things > "just because that's the certified way" that end up biting them in the > rear. Trusting your SAN arrays is one of them. That all being said, > the need to do migrations is a very valid concern. Jason, I don't claim that SAN/RAID solutions are the best and don't have any mistakes/failures/problems. But if SAN/RAID is so bad why companies using them survive ? Imagine also that some company is using SAN/RAID for a few years and doesn't have any problems (or once a few months). Also from time to time they need to migrate between arrays (for whatever reason). Now you come and say that they have unreliable SAN/RAID and you offer something new (ZFS) which is going to make it much more reliable but migration to another array will be painfull. What do you think what they choose ? BTW: I am a fan of ZFS. :-) przemol -- Ustawiaj rekordy DNS dla swojej domeny >> http://link.interia.pl/f1a1a ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re[4]: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Hello Erik, Tuesday, February 27, 2007, 5:47:42 PM, you wrote: ET> ET> The answer is: insufficient data. ET> With modern journalling filesystems, I've never had to fsck anything or ET> run a filesystem repair. Ever. On any of my SAN stuff. I'm not sure if you consider UFS in S10 as a modern journalling filesystem but in case you do: Feb 13 12:03:16 ufs: [ID 879645 kern.notice] NOTICE: /opt/d1635: unexpected free inode 54305084, run fsck(1M) -o f This file system is on a medium large array (IBM) in a SAN environment. -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
[EMAIL PROTECTED] wrote: Is the "true situation" really so bad ? The failure mode is silent error. By definition, it is hard to count silent errors. What ZFS does is improve the detection of silent errors by a rather considerable margin. So, what we are seeing is that suddenly people are seeing errors that they didn't see before (or do you "hear" silent errors? ;-). That has been surprising and leads some of us to recommend ZFS no matter what your storage looks like, even if silent error detection is the only benefit. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Re[2]: [zfs-discuss] Re: Re: How much do we really want zpool remove?
The answer is: insufficient data. With modern journalling filesystems, I've never had to fsck anything or run a filesystem repair. Ever. On any of my SAN stuff. The sole place I've run into filesystem corruption in the traditional sense is with faulty hardware controllers; and, I'm not even sure ZFS could recover from those situations, though less dire ones where the controllers are merely emitting slightly wonky problems certainly would be within ZFS's ability to fix, vice the inability of a SAN to determine that the data was bad. That said, the primary issue here is that nobody really has any idea about silent corruption - that is, blocks which change value, but are data, not filesystem-relevant. Bit flips and all. Realistically, the only way previous to ZFS to detect this was to do bit-wise comparisons against backups, which becomes practically impossible on an active data set. SAN/RAID equipment still has a very considerable place over JBODs in most large-scale places, particularly in areas of configuration flexibility, security, and management. That said, I think we're arguing at cross-purposes: the real solution for most enterprise customers is SAN + ZFS, not either just by itself. -- Erik Trimble Java System Support Mailstop: usca14-102 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re[2]: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Hello przemolicc, Tuesday, February 27, 2007, 11:28:59 AM, you wrote: ppf> On Tue, Feb 27, 2007 at 08:29:04PM +1100, Shawn Walker wrote: >> On 27/02/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> >On Thu, Feb 22, 2007 at 12:21:50PM -0700, Jason J. W. Williams wrote: >> >> Hi Przemol, >> >> >> >> I think Casper had a good point bringing up the data integrity >> >> features when using ZFS for RAID. Big companies do a lot of things >> >> "just because that's the certified way" that end up biting them in the >> >> rear. Trusting your SAN arrays is one of them. That all being said, >> >> the need to do migrations is a very valid concern. >> > >> >Jason, >> > >> >I don't claim that SAN/RAID solutions are the best and don't have any >> >mistakes/failures/problems. But if SAN/RAID is so bad why companies >> >using them survive ? >> >> I think he was trying to say that people that believe that those >> solutions are reliable just because they are based on SAN/RAID >> technology and are not aware of the true situation surrounding them. ppf> Is the "true situation" really so bad ? ppf> My feeling was that he was trying to say that there is no SAN/RAID ppf> solution without data integrity problem. Is it really true ? ppf> Does anybody have any paper (*) about percentage of problems in SAN/RAID ppf> because of data integrity ? Is it 5 % ? Or 30 % ? Or maybe 60 % ? ppf> (*) Maybe such paper/report should be a start point for our discussion. See http://sunsolve.sun.com/search/document.do?assetkey=1-26-102815-1 as one example. This is entry level array but still such things happens. I do also observer similar problems with IBM's array (larger). It's just that people are used to fsck from time to time not really knowing why and in many cases they do not realize that their data is not exactly what they expect it to be. However from my experience I must admit the problem is almost only seen with SATA drives. I had a problem with SCSI adapter which was sending some warnings (driver) but still was passing IOs - it turned out data were corrupted. Changing SCSI adapter solved the problem. The point is that thanks to ZFS we caught the problem, replaced bad card, did zpool scrub and everything was in perfect shape. No need to resynchronize data, etc. Another time I had a problem with FC array and lost some data but there's no ZFS on it :((( On all other arrays, jbods, etc. with SCSI and/or FC disks I haven't seen (yet) checksum errors reported by ZFS. -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
On Tue, Feb 27, 2007 at 08:29:04PM +1100, Shawn Walker wrote: > On 27/02/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > >On Thu, Feb 22, 2007 at 12:21:50PM -0700, Jason J. W. Williams wrote: > >> Hi Przemol, > >> > >> I think Casper had a good point bringing up the data integrity > >> features when using ZFS for RAID. Big companies do a lot of things > >> "just because that's the certified way" that end up biting them in the > >> rear. Trusting your SAN arrays is one of them. That all being said, > >> the need to do migrations is a very valid concern. > > > >Jason, > > > >I don't claim that SAN/RAID solutions are the best and don't have any > >mistakes/failures/problems. But if SAN/RAID is so bad why companies > >using them survive ? > > I think he was trying to say that people that believe that those > solutions are reliable just because they are based on SAN/RAID > technology and are not aware of the true situation surrounding them. Is the "true situation" really so bad ? My feeling was that he was trying to say that there is no SAN/RAID solution without data integrity problem. Is it really true ? Does anybody have any paper (*) about percentage of problems in SAN/RAID because of data integrity ? Is it 5 % ? Or 30 % ? Or maybe 60 % ? (*) Maybe such paper/report should be a start point for our discussion. przemol -- Gdy nie ma dzieci... - zobacz >> http://link.interia.pl/f19eb ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
On 27/02/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: On Thu, Feb 22, 2007 at 12:21:50PM -0700, Jason J. W. Williams wrote: > Hi Przemol, > > I think Casper had a good point bringing up the data integrity > features when using ZFS for RAID. Big companies do a lot of things > "just because that's the certified way" that end up biting them in the > rear. Trusting your SAN arrays is one of them. That all being said, > the need to do migrations is a very valid concern. Jason, I don't claim that SAN/RAID solutions are the best and don't have any mistakes/failures/problems. But if SAN/RAID is so bad why companies using them survive ? I think he was trying to say that people that believe that those solutions are reliable just because they are based on SAN/RAID technology and are not aware of the true situation surrounding them. -- "Less is only more where more is no good." --Frank Lloyd Wright Shawn Walker, Software and Systems Analyst [EMAIL PROTECTED] - http://binarycrusader.blogspot.com/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
On Thu, Feb 22, 2007 at 12:21:50PM -0700, Jason J. W. Williams wrote: > Hi Przemol, > > I think Casper had a good point bringing up the data integrity > features when using ZFS for RAID. Big companies do a lot of things > "just because that's the certified way" that end up biting them in the > rear. Trusting your SAN arrays is one of them. That all being said, > the need to do migrations is a very valid concern. Jason, I don't claim that SAN/RAID solutions are the best and don't have any mistakes/failures/problems. But if SAN/RAID is so bad why companies using them survive ? Imagine also that some company is using SAN/RAID for a few years and doesn't have any problems (or once a few months). Also from time to time they need to migrate between arrays (for whatever reason). Now you come and say that they have unreliable SAN/RAID and you offer something new (ZFS) which is going to make it much more reliable but migration to another array will be painfull. What do you think what they choose ? BTW: I am a fan of ZFS. :-) przemol -- Ustawiaj rekordy DNS dla swojej domeny >> http://link.interia.pl/f1a1a ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Hi Przemol, I think Casper had a good point bringing up the data integrity features when using ZFS for RAID. Big companies do a lot of things "just because that's the certified way" that end up biting them in the rear. Trusting your SAN arrays is one of them. That all being said, the need to do migrations is a very valid concern. Best Regards, Jason On 2/22/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: On Wed, Feb 21, 2007 at 04:43:34PM +0100, [EMAIL PROTECTED] wrote: > > >I cannot let you say that. > >Here in my company we are very interested in ZFS, but we do not care > >about the RAID/mirror features, because we already have a SAN with > >RAID-5 disks, and dual fabric connection to the hosts. > > But you understand that these underlying RAID mechanism give absolutely > no guarantee about data integrity but only that some data was found were > some (possibly other) data was written? (RAID5 never verifies the > checkum is correct on reads; it only uses it to reconstruct data when > reads fail) But you understand that he perhaps knows that but so far nothing wrong happened [*] and migration is still very important feature for him ? [*] almost every big company has its data center with SAN and FC connections with RAID-5 or RAID-10 in their storage arrays and they are treated as reliable przemol -- Wpadka w kosciele - zobacz >> http://link.interia.pl/f19ea ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
On Wed, Feb 21, 2007 at 04:43:34PM +0100, [EMAIL PROTECTED] wrote: > > >I cannot let you say that. > >Here in my company we are very interested in ZFS, but we do not care > >about the RAID/mirror features, because we already have a SAN with > >RAID-5 disks, and dual fabric connection to the hosts. > > But you understand that these underlying RAID mechanism give absolutely > no guarantee about data integrity but only that some data was found were > some (possibly other) data was written? (RAID5 never verifies the > checkum is correct on reads; it only uses it to reconstruct data when > reads fail) But you understand that he perhaps knows that but so far nothing wrong happened [*] and migration is still very important feature for him ? [*] almost every big company has its data center with SAN and FC connections with RAID-5 or RAID-10 in their storage arrays and they are treated as reliable przemol -- Wpadka w kosciele - zobacz >> http://link.interia.pl/f19ea ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
On February 21, 2007 10:55:43 AM -0800 Richard Elling <[EMAIL PROTECTED]> wrote: Valery Fouques wrote: The ability to shrink a pool by removing devices is the only reason my enterprise is not yet using ZFS, simply because it prevents us from easily migrating storage. That logic is totally bogus AFAIC. There are so many advantages to running ZFS that denying yourself that opportunity is very short sighted - especially when there are lots of ways of working around this minor feature deficiency. I cannot let you say that. Here in my company we are very interested in ZFS, but we do not care about the RAID/mirror features, because we already have a SAN with RAID-5 disks, and dual fabric connection to the hosts. We would have migrated already if we could simply migrate data from a storage array to another (which we do more often than you might think). Currently we use (and pay for) VXVM, here is how we do a migration: But you describe VxVM feature, not a file system feature. But in the context of zfs, this is appropriate. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
>On February 21, 2007 4:43:34 PM +0100 [EMAIL PROTECTED] wrote: >> >>> I cannot let you say that. >>> Here in my company we are very interested in ZFS, but we do not care >>> about the RAID/mirror features, because we already have a SAN with >>> RAID-5 disks, and dual fabric connection to the hosts. >> >> But you understand that these underlying RAID mechanism give absolutely >> no guarantee about data integrity but only that some data was found were >> some (possibly other) data was written? (RAID5 never verifies the >> checkum is correct on reads; it only uses it to reconstruct data when >> reads fail) > >um, I thought smarter arrays did that these days. Of course it's not >end-to-end so the parity verification isn't as useful as it should be; >gigo. Generate extra I/O and verify parity, is that not something that may be a problem in performance benchmarking? For mirroring, a similar problem exists, of course. ZFS reads from the right side of the mirror and corrects the wrong side if it finds an error. RAIDs do not. Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Valery Fouques wrote: The ability to shrink a pool by removing devices is the only reason my enterprise is not yet using ZFS, simply because it prevents us from easily migrating storage. That logic is totally bogus AFAIC. There are so many advantages to running ZFS that denying yourself that opportunity is very short sighted - especially when there are lots of ways of working around this minor feature deficiency. I cannot let you say that. Here in my company we are very interested in ZFS, but we do not care about the RAID/mirror features, because we already have a SAN with RAID-5 disks, and dual fabric connection to the hosts. We would have migrated already if we could simply migrate data from a storage array to another (which we do more often than you might think). Currently we use (and pay for) VXVM, here is how we do a migration: But you describe VxVM feature, not a file system feature. 1/ Allocate disks from the new array, visible by the host. 2/ Add the disks in the diskgroup. 3/ Run vxevac to evacuate data from "old" disks. 4/ Remove old disks from the DG. If you explain how to do that with ZFS, no downtime, and new disks with different capacities, you're my hero ;-) zpool replace old-disk new-disk The caveat is that new-disk must be as big or bigger than old-disk. This caveat is the core of the shrink "problem" -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
On February 21, 2007 4:43:34 PM +0100 [EMAIL PROTECTED] wrote: I cannot let you say that. Here in my company we are very interested in ZFS, but we do not care about the RAID/mirror features, because we already have a SAN with RAID-5 disks, and dual fabric connection to the hosts. But you understand that these underlying RAID mechanism give absolutely no guarantee about data integrity but only that some data was found were some (possibly other) data was written? (RAID5 never verifies the checkum is correct on reads; it only uses it to reconstruct data when reads fail) um, I thought smarter arrays did that these days. Of course it's not end-to-end so the parity verification isn't as useful as it should be; gigo. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
>I cannot let you say that. >Here in my company we are very interested in ZFS, but we do not care >about the RAID/mirror features, because we already have a SAN with >RAID-5 disks, and dual fabric connection to the hosts. But you understand that these underlying RAID mechanism give absolutely no guarantee about data integrity but only that some data was found were some (possibly other) data was written? (RAID5 never verifies the checkum is correct on reads; it only uses it to reconstruct data when reads fail) Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
On Wed, 21 Feb 2007, Valery Fouques wrote: > Here in my company we are very interested in ZFS, but we do not care > about the RAID/mirror features, because we already have a SAN with > RAID-5 disks, and dual fabric connection to the hosts. ... And presumably you've read the threads where ZFS has helped find (and repair) corruption in such setups? (But yeah, I agree the ability to shrink a pool is important.) -- Rich Teer, SCSA, SCNA, SCSECA, OpenSolaris CAB member President, Rite Online Inc. Voice: +1 (250) 979-1638 URL: http://www.rite-group.com/rich ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: Re: How much do we really want zpool remove?
> > The ability to shrink a pool by removing devices is the only reason my > > enterprise is not yet using ZFS, simply because it prevents us from > > easily migrating storage. > > That logic is totally bogus AFAIC. There are so many advantages to > running ZFS that denying yourself that opportunity is very short sighted - > especially when there are lots of ways of working around this minor > feature deficiency. I cannot let you say that. Here in my company we are very interested in ZFS, but we do not care about the RAID/mirror features, because we already have a SAN with RAID-5 disks, and dual fabric connection to the hosts. We would have migrated already if we could simply migrate data from a storage array to another (which we do more often than you might think). Currently we use (and pay for) VXVM, here is how we do a migration: 1/ Allocate disks from the new array, visible by the host. 2/ Add the disks in the diskgroup. 3/ Run vxevac to evacuate data from "old" disks. 4/ Remove old disks from the DG. If you explain how to do that with ZFS, no downtime, and new disks with different capacities, you're my hero ;-) This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Hi, I need to be a little bit more precise in how I formulate comments: 1. Yes, zpool remove is a desirable feature, no doubt about that. 2. Most of the cases where customers ask for "zpool remove" can be solved with zfs send/receive or with zpool replace. Think Pareto's 80-20 rule. 2a. The cost of doing 2., including extra scratch storage space or scheduling related work into planned downtimes is smaller than the cost of not using ZFS at all. 2b. Even in the remaining 20% of cases (figuratively speaking, YMMV) where zpool remove would be the only solution, I feel that the cost of sacrificing the extra storage space that would have become available through "zpool remove" is smaller than the cost of the project not benefitting from the rest of ZFS' features. 3. Bottom line: Everybody wants zpool remove as early as possible, but IMHO this is not an objective barrier to entry for ZFS. Note my use of the word "objective". I do feel that we have to implement zpool remove for subjective reasons, but that is a non technical matter. Is this an agreeable summary of the situation? Best regards, Constantin -- Constantin GonzalezSun Microsystems GmbH, Germany Platform Technology Group, Client Solutionshttp://www.sun.de/ Tel.: +49 89/4 60 08-25 91 http://blogs.sun.com/constantin/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Al Hopper wrote: On Fri, 26 Jan 2007, Torrey McMahon wrote: Al Hopper wrote: Now your accounting folks are going to be asking you to justify the purchase of that hi-end SAN box and why you're not using ZFS everywhere. :) Oh - and the accounting folks love it when you tell them there's no ongoing cost of ownership - because Joe Screwdriver can swap out a failed Seagate 500Gb SATA drive after he picks up a replacement from Frys on his lunch break! Because ZFS doesn't run everywhere. Because most low end JBODs are "low end" for a reason. They aren't as reliable, have crappy monitoring, etc. Agreed. There will never be one screwdriver that fits everything. I was simply trying to re-inforce my point. It's a good point. We just need to make sure we don't forget that part. People love to pull email threads out of contextor google for that matter. ;) Fix those two things when you get a chance. ;) Have a good weekend Torrey (and zfs-discuss). Same to you Al. (and zfs-discuss). ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Richard Elling wrote: Personally, I've never been in the situation where users ask for less storage, but maybe I'm just the odd guy out? ;-) You just realized that JoeSysadmin allocated ten luns to the zpool when he realy only should have allocated one. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
On Fri, 26 Jan 2007, Torrey McMahon wrote: > Al Hopper wrote: > > > > Now your accounting folks are going to be asking you to justify the > > purchase of that hi-end SAN box and why you're not using ZFS > > everywhere. :) > > > > Oh - and the accounting folks love it when you tell them there's no > > ongoing cost of ownership - because Joe Screwdriver can swap out a failed > > Seagate 500Gb SATA drive after he picks up a replacement from Frys on his > > lunch break! > > Because ZFS doesn't run everywhere. > Because most low end JBODs are "low end" for a reason. They aren't as > reliable, have crappy monitoring, etc. Agreed. There will never be one screwdriver that fits everything. I was simply trying to re-inforce my point. > Fix those two things when you get a chance. ;) Have a good weekend Torrey (and zfs-discuss). Regards, Al Hopper Logical Approach Inc, Plano, TX. [EMAIL PROTECTED] Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT OpenSolaris.Org Community Advisory Board (CAB) Member - Apr 2005 OpenSolaris Governing Board (OGB) Member - Feb 2006 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
On Fri, 26 Jan 2007, Toby Thain wrote: > > > > Oh - and the accounting folks love it when you tell them there's no > > ongoing cost of ownership - because Joe Screwdriver can swap out a > > failed > > Seagate 500Gb SATA drive after he picks up a replacement from Frys > > on his > > lunch break! > > Why do people think this will work? I never could figure it out. > > There's many a slip 'twixt cup and lip. You need the spare already > sitting there. Agreed. I remember years ago, when a Sun service tech came onsite at a fortune 100 company I was working in at the time and we stopped him, handed him a disk drive in an anti-static bag and said - "don't unpack your tools - it was a bad disk, we replaced it from our spares, here's the bad one - please replace it under the service agreement". He thought about this for about 5 Seconds and said; "I wish all my customers were like you guys". Then he was gone! :) Regards, Al Hopper Logical Approach Inc, Plano, TX. [EMAIL PROTECTED] Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT OpenSolaris.Org Community Advisory Board (CAB) Member - Apr 2005 OpenSolaris Governing Board (OGB) Member - Feb 2006 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Al Hopper wrote: Now your accounting folks are going to be asking you to justify the purchase of that hi-end SAN box and why you're not using ZFS everywhere. :) Oh - and the accounting folks love it when you tell them there's no ongoing cost of ownership - because Joe Screwdriver can swap out a failed Seagate 500Gb SATA drive after he picks up a replacement from Frys on his lunch break! Because ZFS doesn't run everywhere. Because most low end JBODs are "low end" for a reason. They aren't as reliable, have crappy monitoring, etc. Fix those two things when you get a chance. ;) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Oh - and the accounting folks love it when you tell them there's no ongoing cost of ownership - because Joe Screwdriver can swap out a failed Seagate 500Gb SATA drive after he picks up a replacement from Frys on his lunch break! Why do people think this will work? I never could figure it out. There's many a slip 'twixt cup and lip. You need the spare already sitting there. --T ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
To be fair, you can replace vdevs with same-sized or larger vdevs online. The issue is that you cannot replace with smaller vdevs nor can you eliminate vdevs. In other words, I can migrate data around without downtime, I just can't shrink or eliminate vdevs without send/recv. This is where the philosophical disconnect lies. Everytime we descend into this rathole, we stir up more confusion :-( We did just this to move off RAID-5 LUNs that were the vdevs for a pool, to RAID-10 LUNs. Worked very well, and as Richard said was done all on-line. Doesn't really address the shrinking issue though. :-) Best Regards, Jason ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
On Fri, 26 Jan 2007, Rainer Heilke wrote: > > So, if I was an enterprise, I'd be willing to keep > > enough empty LUNs > > available to facilitate at least the migration of > > one or more filesystems > > if not complete pools. reformatted ... > You might be, but don't be surprised when the Financials folks laugh you > out of their office. Large corporations do not make money by leaving > wads of cash lying around, and that's exactly what a few terabytes of > unused storage in a high-end SAN is. This is in addition to the laughter But this is exactly where ZFS distrupts "Large corporations" thinking. You're talking about (for example) 2 terabytes on a high-end SAN which costs (what ?) per GB (including the capital cost of the hi-end SAN) versus a dual Opteron box with 12 * 500Gb SATA disk drives that gives you 5TB of storage for (in round numbers) a total of ~ $6k. And how much are your ongoing monthlies on that hi-end SAN box? (Don't answer) So - aside from the occasional use of the box for data migration, this ZFS "storage box" has 1,001 other uses. Pick any two (uses), based on your knowledge of big corporation thinking and its an easy sell to management. Now your accounting folks are going to be asking you to justify the purchase of that hi-end SAN box and why you're not using ZFS everywhere. :) Oh - and the accounting folks love it when you tell them there's no ongoing cost of ownership - because Joe Screwdriver can swap out a failed Seagate 500Gb SATA drive after he picks up a replacement from Frys on his lunch break! > generated by the comment that, "not a big deal if the Financials and HR > databases are offline for three days while we do the migration." Good Again - sounds like more "legacy" thinking. With multiple gigabit ethernet connections you can move terrabytes of information in a hour, instead of in 24-hours - using legacy tape systems etc. This can be easily handled during scheduled downtime. > luck writing up a business case that justifies this sort of fiscal > generosity. > Sorry, this argument smacks a little too much of being out of touch with > the fiscal (and time) restrictions of working in a typical corporation, > as opposed to a well-funded research group. > > I hope I'm not sounding rude, but those of us working in medium to large > corporations simply do not have the money for such luxuries. Period. On the contrary - if you're not thinking ZFS, you're wasting a ton of IT $s and hurting the competitiveness of your business. Regards, Al Hopper Logical Approach Inc, Plano, TX. [EMAIL PROTECTED] Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT OpenSolaris.Org Community Advisory Board (CAB) Member - Apr 2005 OpenSolaris Governing Board (OGB) Member - Feb 2006 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Rainer Heilke wrote: So, if I was an enterprise, I'd be willing to keep enough empty LUNs available to facilitate at least the migration of one or more filesystems if not complete pools. You might be, but don't be surprised when the Financials folks laugh you out of their office. Large corporations do not make money by leaving wads of cash lying around, and that's exactly what a few terabytes of unused storage in a high-end SAN is. This is in addition to the laughter generated by the comment that, "not a big deal if the Financials and HR databases are offline for three days while we do the migration." Good luck writing up a business case that justifies this sort of fiscal generosity. To be fair, you can replace vdevs with same-sized or larger vdevs online. The issue is that you cannot replace with smaller vdevs nor can you eliminate vdevs. In other words, I can migrate data around without downtime, I just can't shrink or eliminate vdevs without send/recv. This is where the philosophical disconnect lies. Everytime we descend into this rathole, we stir up more confusion :-( If you consider your pool of storage as a zpool, then the management of subparts of the pool is done at the file system level. This concept is different than other combinations of devices and file systems such as SVM+UFS. When answering the ZFS shrink question, you need to make sure you're not applying the old concepts to the new model. Personally, I've never been in the situation where users ask for less storage, but maybe I'm just the odd guy out? ;-) Others have offered cases where a shrink or vdev restructuring could be useful. But I still see some confusion with file system management (including zvols) and device management. The shrink feature is primarily at the device management level. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: Re: How much do we really want zpool remove?
> So, if I was an enterprise, I'd be willing to keep > enough empty LUNs > available to facilitate at least the migration of > one or more filesystems > if not complete pools. You might be, but don't be surprised when the Financials folks laugh you out of their office. Large corporations do not make money by leaving wads of cash lying around, and that's exactly what a few terabytes of unused storage in a high-end SAN is. This is in addition to the laughter generated by the comment that, "not a big deal if the Financials and HR databases are offline for three days while we do the migration." Good luck writing up a business case that justifies this sort of fiscal generosity. Sorry, this argument smacks a little too much of being out of touch with the fiscal (and time) restrictions of working in a typical corporation, as opposed to a well-funded research group. I hope I'm not sounding rude, but those of us working in medium to large corporations simply do not have the money for such luxuries. Period. Rainer This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: Re: How much do we really want zpool remove?
> > The other point is, how many other volume management systems allow you > > to remove disks? I bet if the answer is not zero, it's not large. ;) > > As far as Solaris is concerned, I'm only aware of two significant such > systems. SVM and VxVM. Correct, in our case we've been using VxVM for years. We're comfortable with it, it does what we want, vxfs performance is outstanding, and we move storage around [u]A LOT[/u] as our SAN storage changes. So for the time being, the benefits won't allow us to give up the ability to shrink/remove on the fly. I should add that I am very excited about ZFS, and hope to be able to use it [i]everywhere[/i] as soon as possible. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?
Rainer Heilke wrote: For the "clone another system" zfs send/recv might be useful Keeping in mind that you only want to send/recv one half of the ZFS mirror... Huh ? That doesn't make any sense. You can't send half a mirror. When you are running zfs send it is a "read" and ZFS will read the data from all available mirrors to help performance. When it is zfs recv it will write to all sides of the mirror on the destination. What are you actually trying to say here ? -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: Re: How much do we really want zpool remove?
> For the "clone another system" zfs send/recv might be > useful Keeping in mind that you only want to send/recv one half of the ZFS mirror... Rainer This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
