Re: [zfs-discuss] If you have ZFS in production, willing to share some details (with me)?
On Fri, Sep 18, 2009 at 1:51 PM, Steffen Weiberle steffen.weibe...@sun.com wrote: I am trying to compile some deployment scenarios of ZFS. # of systems 3 amount of storage 10 TB on storage server (can scale to 30) application profile(s) NFS and CIFS type of workload (low, high; random, sequential; read-only, read-write, write-only) Boot drives, Nearline backup, Postgres DB (OpenNMS) storage type(s) SATA industry Software whether it is private or I can share in a summary anything else that might be of interest You can share my info :) Thanks in advance!! Steffen ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] deduplication
Thanks James! I look forward to these - we could really use dedup in my org. Blake On Thu, Sep 17, 2009 at 6:02 PM, James C. McPherson james.mcpher...@sun.com wrote: On Thu, 17 Sep 2009 11:50:17 -0500 Tim Cook t...@cook.ms wrote: On Thu, Sep 17, 2009 at 5:27 AM, Thomas Burgess wonsl...@gmail.com wrote: I think you're right, and i also think we'll still see a new post asking about it once or twice a week. [snip] As we should. Did the video of the talks about dedup ever even get posted to Sun's site? I never saw it. I remember being told we were all idiots when pointing out that it had mysteriously not been posted... Hi Tim, I certainly do not recall calling anybody an idiot for asking about the video or slideware. I definitely _do_ recall asking for people to be patient because (1) we had lighting problems with the auditorium which interfered with recording video (2) we have been getting the videos professionally edited so that when we can put them up on an appropriate site (which I imagine will be slx.sun.com), then the vids will adhere to the high standards which you have come to expect. (3) professional editing of videos takes time and money. We are getting this done as fast as we can. I asked Deirdre about the videos yesterday, she said that they are almost ready. Rest assured that when they are ready I will announce their availability as soon as I possibly can. James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Expanding a raidz pool?
On Fri, Aug 28, 2009 at 11:22 PM, Ty Newtonty.new...@copperchipgames.com wrote: Hi, I've read a few articles about the lack of 'simple' raidz pool expansion capability in ZFS. I am interested in having a go at developing this functionality. Is anyone working on this at the moment? I'll explain what I am proposing. As mentioned in many forums, the concept is really simple: allow a raidz pool to grow by adding one or more disks to an existing pool. My intended user group is the consumer market, as opposed to the enterprise, so I expect I'll put some rather strict limitations on how/when this functionality will operate: to make the first implementation more achievable. The use case I will try and solve first is, what I see as, the simplest. I have a raidz pool configured with 1 file system on top; no snapshots. I want to add an additional disk (must be at least the same size as the rest of the disks in the pool). I don't mind if there is some downtime. I want all my data to take advantage of the additional disk. Have you looked at the 'add' section of the zpool manpage? You can add another vdev, provided it provides similar parity, something like: zpool add data raidz2 c4t14d0 c4t15d0 c5t12d0 c5t13d0 c5t14d0 which I did a few weeks ago. Here, I had a raidz2 pool called 'data' made up of 5 disks. I added another 5 disks also configured as raidz2 with this command. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Status/priority of 6761786
I think the value of auto-snapshotting zvols is debatable. At least, there are not many folks who need to do this. What I'd rather see is a default property of 'auto-snapshot=off' for zvols. Blake On Thu, Aug 27, 2009 at 4:29 PM, Tim Cookt...@cook.ms wrote: On Thu, Aug 27, 2009 at 3:24 PM, Remco Lengers re...@lengers.com wrote: Dave, Its logged as an RFE (Request for Enhancement) not as a CR (bug). The status is 3-Accepted/ P1 RFE RFE's are generally looked at in a much different way then a CR. ..Remco Seriously? It's considered works as designed for a system to take 5+ hours to boot? Wow. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zfs+nfs: scary nfs log entries?
I have a zfs dataset that I use for network home directories. The box is running 2008.11 with the auto-snapshot service enabled. To help debug some mysterious file deletion issues, I've enabled nfs logging (all my clients are NFSv3 Linux boxes). I keep seeing lines like this in the nfslog: br br pre Wed Aug 19 10:20:48 2009 0 host.name.domain.com 1168 zfs-auto-snap.hourly-2009-08-17-09.00/username/incoming.file b _ i r 0 nfs 0 * /pre br Why is my path showing up with the name of a snapshot? This scares me since snapshots get rolled off automatically...on the other hand, I know that the snapshots are read-only. Any insights? br br Blake -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] missing disk space
On Mon, Aug 3, 2009 at 12:35 PM, David E. Andersondanders...@gmail.com wrote: I am new to ZFS, so please bear with me... I created a raidz1 pool from three 1.5TB disks on OpenSolaris 2009.6. I see less than 1TB useable space. What did I do wrong? $ zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT rpool 464G 42.2G 422G 9% ONLINE - storage 1.36T 143K 1.36T 0% ONLINE - $ df -h /storage Filesystem Size Used Avail Use% Mounted on storage 913G 26K 913G 1% /storage $ zpool status pool: rpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror ONLINE 0 0 0 c8t1d0s0 ONLINE 0 0 0 c10d0s0 ONLINE 0 0 0 errors: No known data errors pool: storage state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 raidz1 ONLINE 0 0 0 c9d0s2 ONLINE 0 0 0 c10d0s2 ONLINE 0 0 0 c10d1s2 ONLINE 0 0 0 errors: No known data errors -- David can you post the output of 'zfs get all storage' ? blake ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] missing disk space
On Mon, Aug 3, 2009 at 1:41 PM, David E. Andersondanders...@gmail.com wrote: $ zfs get all storage NAME PROPERTY VALUE SOURCE storage type filesystem - storage creation Fri Jul 10 21:19 2009 - storage used 89.2K - storage available 913G - storage referenced 25.3K - storage compressratio 1.00x - storage mounted yes - storage quota none default storage reservation none default storage recordsize 128K default storage mountpoint /storage default storage sharenfs off default storage checksum on default storage compression off default storage atime on default storage devices on default storage exec on default storage setuid on default storage readonly off default storage zoned off default storage snapdir hidden default storage aclmode groupmask default storage aclinherit restricted default storage canmount on default storage shareiscsi off default storage xattr on default storage copies 1 default storage version 3 - storage utf8only off - storage normalization none - storage casesensitivity sensitive - storage vscan off default storage nbmand off default storage sharesmb off default storage refquota none default storage refreservation none default storage primarycache all default storage secondarycache all default storage usedbysnapshots 0 - storage usedbydataset 25.3K - storage usedbychildren 63.9K - storage usedbyrefreservation 0 On Mon, Aug 3, 2009 at 10:34 AM, Blake blake.ir...@gmail.com wrote: On Mon, Aug 3, 2009 at 12:35 PM, David E. Andersondanders...@gmail.com wrote: I am new to ZFS, so please bear with me... I created a raidz1 pool from three 1.5TB disks on OpenSolaris 2009.6. I see less than 1TB useable space. What did I do wrong? snip can you post the output of 'zfs get all storage' ? blake -- David Looking back at your 'zpool status' output, I think you might have accidentally made put one of your big storage disks in your rpool, and one of your little rpool disks in your big storage pool. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] why is zpool import still hanging in opensolaris 2009.06 ??? no fix yet ???
This sounds like a bug I hit - if you have zvols on your pool, and automatic snapshots enabled, the thousands of resultant snapshots have to be polled by devfsadm during boot, which take a long time - several seconds per zvol. I remove the auto-snapshot property from my zvols and the slow boot stopped. Blake On Thu, Jul 23, 2009 at 5:53 AM, Luc De Meyerno-re...@opensolaris.org wrote: Follow-up : happy end ... It took quite some thinkering but... i have my data back... I ended up starting without the troublesome zfs storage array, de-installed the iscsitartget software and re-installed it...just to have solaris boot without complaining about missing modules... That left me with a system that would boot as long as the storage was disconnected... Reconnecting it made the boot stop at the hostname. Then the disk activity light would flash every second or so forever... I then rebooted using milestone=none. That worked also with the storage attached! So now I was sure that some software process was causing a hangup (or what appeared to be a hangup.) I could now in milestone none verify that the pool was intact: and so it was... fortunately I had not broken the pool itself... all online with no errors to report. I then went to milestone-all which again made the system hang with the disk activity every second forever. I think the task doing this was devfsadm. I then assumed on a gut feeling that somehow the system was scanning or checking the pool. I left the system overnight in a desperate attempt because I calculated the 500GB checking to take about 8 hrs if the system would *really* scan everything. (I copied a 1 TB drive last week which took nearly 20 hrs, so I learned that sometimes I need to wait... copying these big disks takes a *lot* of time!) This morning I switched on the monitor and lo and behold : a login screen The store was there! Lesson for myself and others: you MUST wait at the hostname line: the system WILL eventually come online... but don't ask how long it takes... I hate to think how long it would take if I had a 10TB system. (but then again, a file-system-check on an ext2 disk also takes forever...) I re-enabled the iscsitgtd and did a list : it saw one of the two targets ! (which was ok because I remembered that I had turned off the shareiscsi flag on the second share. I then went ahead and connected the system back into the network and repaired the iscsi-target on the virtual mainframe : WORKED ! Copied over the virtual disks to local store so I can at least start up these servers asap again. Then set the iscsishare on the second and most important share: OK! Listed the targets: THERE, BOTH! Repaired it's connection too: WORKED...! I am copying everything away from the ZFS pools now, but my data is recovered... fortunately. I now have mixed feelings about the ordeal: yes Sun Solaris kept its promise: I did not loose my data. But the time and trouble it took to recover in this case (just to restart a system for example taking an overnight wait!) is something that a few of my customers would *seriously* dislike... But: a happy end after all... most important data rescued and 2nd important : I learned a lot in the process... Bye Luc De Meyer Belgium -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Things I Like About ZFS
On Fri, Jun 19, 2009 at 10:30 PM, Dave Ringkorno-re...@opensolaris.org wrote: I'll start: - The commands are easy to remember -- all two of them. Which is easier, SVM or ZFS, to mirror your disks? I've been using SVM for years and still have to break out the manual to use metadb, metainit, metastat, metattach, metadetach, etc. I hardly ever have to break out the ZFS manual. I can actually remember the commands and options to do things. Don't even start me on VxVM. - Boasting to the unconverted. We still have a lot of VxVM and SVM on Solaris, and LVM on AIX, in the office. The other admins are always having issues with storage migrations, full filesystems, Live Upgrade, corrupted root filesystems, etc. I love being able to offer solutions to their immediate problems, and follow it up with, You know, if your box was on ZFS this wouldn't be an issue. Interesting. Usually the problems make their way to this list more than the successes. Glad to hear it! BTW, ZFS just saved my skin tonight after I botched an OpenNMS upgrade and was able to go back to my auto-snapshots :) And there was a power failure earlier that took down a bunch of hosts that rely on our multi-terabyte ZFS filer, as well as the filer itself - no waiting around for fsck, thanks! Blake ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] replicating a root pool
On Fri, May 22, 2009 at 12:40 AM, Ian Collins i...@ianshome.com wrote: Mark J Musante wrote: On Thu, 21 May 2009, Ian Collins wrote: I'm trying to use zfs send/receive to replicate the root pool of a system and I can't think of a way to stop the received copy attempting to mount the filesystem over the root of the destination pool. If you're using build 107 or later, there's a hidden -u option available for zfs receive to tell it not to mount the dataset. See http://tinyurl.com/crgog8 for more details. Thanks for the tip Mark, unfortunately I'm stuck with Solaris 10 on this system. I just did this the old way, and it wasn't that hard. I didn't even script it (yet), but it seems like it should be easy to do if you use the solarisinternals recipe. Blake ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs reliability under xen
On Fri, May 22, 2009 at 2:44 PM, Ahmed Kamal email.ahmedka...@googlemail.com wrote: However, if you need to decide, whether to use Xen, test your setup before going into production and ask your boss, whether he can live with innovative ... solutions ;-) Thanks a lot for the informative reply. It has been definitely helpful I am however interested in the reliability of running the ZFS stack as Xen domU (and not dom0). For instance, I am worried that the emulated disk controller would not obey flushes, or write ordering thus stabbing zfs in the back. Regards I've gotten very good performance numbers for I/O out of a 2008.11 PV domU with a zfs zvol as the storage device/install disk Blake ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Compression/copies on root pool RFE
On Wed, May 6, 2009 at 11:14 AM, Rich Teer rich.t...@rite-group.com wrote: On Wed, 6 May 2009, Richard Elling wrote: popular interactive installers much more simplified. I agree that interactive installation needs to remain as simple as possible. How about offering a choice an installation time: Custom or default?? Those that don't want/need the interactive flexibility can pick default whereas others who want more flexibility (but still want or need an interactive installation) can pick the custom option. Just a thought... If you do propose this on caiman-discuss, I'd suggest an option to mirror 2 boot devices as well. Doing the slice attach/installgrub is nontrivial for, say, a user who's primarily a dev. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Raidz vdev size... again.
On Tue, Apr 28, 2009 at 10:08 AM, Tim t...@tcsac.net wrote: On Mon, Apr 27, 2009 at 8:25 PM, Richard Elling richard.ell...@gmail.com wrote: I do not believe you can achieve five 9s with current consumer disk drives for an extended period, say 1 year. Just to pipe up, while very few vendors can pull it off, we've seen five 9's with Hitachi gear using SATA. Can you specify the hardware? I've recently switched to LSI SAS1068E controllers and am swimmingly happy. (That's my $.02 - controllers (not surprisingly) affect the niceness of a software RAID solution like ZFS quite a bit - maybe even more than the actual drives...?) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [storage-discuss] Supermicro AOC-SASLP-MV8
I'm quite happy so far with my LSI cards, which replaced a couple of the Supermicro Marvell cards: # scanpci ... pci bus 0x0007 cardnum 0x00 function 0x00: vendor 0x1000 device 0x0058 LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS On Wed, Apr 22, 2009 at 2:45 AM, James Andrewartha jam...@daa.com.au wrote: myxi...@googlemail.com wrote: Bouncing a thread from the device drivers list: http://opensolaris.org/jive/thread.jspa?messageID=357176 Does anybody know if OpenSolaris will support this new Supermicro card, based on the Marvell 88SE6480 chipset? It's a true PCI Express 8 port JBOD SAS/SATA controller with pricing apparently around $125. If it works with OpenSolaris it sounds pretty much perfect. The Linux support for the 6480 builds on the 6440 mvsas support, so I don't think marvell88sx would work, and there doesn't seem to be a Marvell SAS driver for Solaris at all, so I'd say it's not supported. http://www.hardforum.com/showthread.php?t=1397855 has a fair few people testing it out, but mostly under Windows. -- James Andrewartha ___ storage-discuss mailing list storage-disc...@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/storage-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] What can I do to shorten the long awkward names of snapshots?
The cool thing about the way Tim has built the service is that you can edit the variable values in the method script to make snapshot titles pretty much whatever you want. I think he made a good compromise choice between simplicity and clarity in the current titling system. Remember that the Time Slider snapshot viewer essentially makes this transparent to a the end user. And you can make use of Time Slider remotely using ssh -X hostname and then nautilus --no-desktop. On Wed, Apr 15, 2009 at 8:32 PM, Andre van Eyssen an...@purplecow.org wrote: On Wed, 15 Apr 2009, Harry Putnam wrote: Would become: a:freq-041509_1630 Can I suggest perhaps something inspired by the old convention for DNS serials, along the lines of fmmddtt? Like: a:f200904151630 This makes things easier to sort and lines up in a tidy manner. -- Andre van Eyssen. mail: an...@purplecow.org jabber: an...@interact.purplecow.org purplecow.org: UNIX for the masses http://www2.purplecow.org purplecow.org: PCOWpix http://pix.purplecow.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [storage-discuss] Supermicro SAS/SATA controllers?
On Apr 15, 2009, at 8:28 AM, Nicholas Lee emptysa...@gmail.com wrote: On Tue, Apr 14, 2009 at 5:57 AM, Will Murnane will.murn...@gmail.com wrote: Has anyone done any specific testing with SSD devices and solaris other than the FISHWORKS stuff? Which is better for what - SLC and MLC? My impression is that the flash controllers make a much bigger difference than the type of flash inside. You should take a look at AnandTech's review of the new OCZ Vertex drives [1], which has a fairly comprehensive set of benchmarks. I don't think any of the products they review are really optimal choices, though; the Intel X25-E drives look good until you see the price tag, and even they only do 30-odd MB/s random writes. Couple excellent articles about SSD from adandtech last month: http://www.anandtech.com/showdoc.aspx?i=3532 - SSD versus Enterprise SAS and SATA disks (20/3/09) http://www.anandtech.com/storage/showdoc.aspx?i=3531 - The SSD Anthology: Understanding SSDs and New Drives from OCZ (18/3/09) And it looks like the Intel fragmentation issue is fixed as well: http://techreport.com/discussions.x/16739 It's a shame the Sun Writezilla devices are almost 10k USD - seems there are the only units on the market that (apart from cost) work well all around as slog devices - form factor, interface/drivers and performance. What about the new flash drives Andy was showing off in Vegas? Those looked small (capacity) - perhaps cheap too? How much of an issue does the random write bandwidth limit have on a slog device? What about latency? I would have thought the write traffic pattern for slog io was more sequential and bursty. Nicholas ___ storage-discuss mailing list storage-disc...@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/storage-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How recoverable is an 'unrecoverable error'?
On Wed, Apr 15, 2009 at 11:49 AM, Uwe Dippel udip...@gmail.com wrote: Bob Friesenhahn wrote: Since it was not reported that user data was impacted, it seems likely that there was a read failure (or bad checksum) for ZFS metadata which is redundantly stored. (Maybe I am too much of a linguist to not stumble over the wording here.) If it is 'redundant', it is 'recoverable', am I right? Why, if this is the case, does scrub not recover it, and scrub even fails to correct the CKSUM error as long as it is flagged 'unrecoverable', but can do exactly that after the 'clear' command? Ubuntu Linux is unlikely to notice data problems unless the drive reports hard errors. ZFS is much better at checking for errors. No doubt. But ext3 also seems to need much less attention, very much fewer commands. Which leaves it as a viable alternative. I still hope that one day ZFS will be maintainable as simple as ext3; respectively do all that maintenance on its own. :) Uwe You only need to decide what you want here. Yes, ext3 requires less maintenance, because it can't tell you if a block becomes corrupt (though fsck-in when that *does* happen can require hours, compared to zfs replacing with a good block from the other half of your mirror). ZFS can *fully* do it's job only when it has several copies of blocks to choose from. Since you have only one disk here, ZFS can only say 'hey, your checksum for this block is bad - sorry'. ext3 might do the same thing, though only if you tried to use the block with an application that knew what the block was supposed to look like. That said, I think your comments raise a valid point that ZFS could be a little easier for individuals to use. I totally understand why Sun doesn't focus on end-user management tools (not their market) - on the other hand, the code is out there, so if you see a problem, get some people together to write some management tools! :) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [Fwd: ZFS user/group quotas space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009]]
much cheering ensues! 2009/3/31 Matthew Ahrens matthew.ahr...@sun.com: FYI, I filed this PSARC case yesterday, and expect to integrate into OpenSolaris in April. Your comments are welcome. http://arc.opensolaris.org/caselog/PSARC/2009/204/ --matt -- Forwarded message -- From: Matthew Ahrens ahr...@dm-eng-01.sfbay.sun.com To: psarc-...@sun.com Date: Mon, 30 Mar 2009 20:39:05 -0700 (PDT) Subject: ZFS user/group quotas space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: ZFS user/group quotas space accounting 1.2. Name of Document Author/Supplier: Author: Matthew Ahrens 1.3 Date of This Document: 30 March, 2009 4. Technical Description ZFS user/group space accounting A. SUMMARY This case adds support to ZFS for user/group quotas per-uid/gid space tracking. B. PROBLEM Enterprise customers often want to know who is using space, based on what uid and gid owns each file. Education customers often want to apply per-user quotas to hundreds of thousands of users. In these situations, the number of users and/or existing infrastructure prohibits using one filesystem per user and setting filesystem-wide quotas. C. PROPOSED SOLUTION 1. Overview Each filesystem keeps track of how much space inside it is owned by each user (uid) and group (gid). This is the amount of space referenced, so relationships between filesystems, descendents, clones, and snapshots are ignored, and each tracks their user used and group used independently. This is the same policy behind the referenced, refquota, and refreservation properties. The amount of space charged is the amount of space reported by struct stat's st_blocks and du(1). Both POSIX ids (uid gid) and untranslated SIDs are supported (eg, when sharing filesystems over SMB without a name service translation set up). ZFS will now enforce quotas on the amount of space referenced by files owned by particular users and groups. Enforcement may be delayed by several seconds. In other words, users may go a bit over their quota before the system notices that they are over quota and begins to refuse additional writes with EDQUOT. This decision was made to get the feature to market in a reasonable time, with a minimum of engineering resources expended. The design and implementation do not preclude implementing strict enforcement at a later date. User space accounting and quotas stick with each dataset (snapshot, filesystem, and clone). This means that user quotas (and space accounting) are not inherited. They will be copied to a new snapshot, and keep the values they had at the time the snapshot was taken. Likewise, user quotas will be copied to a clone (from its origin snapshot), and they will be copied with zfs send (even without -R). (User accounting and quota information is not actually copied to snapshots and clones, just referenced and copied-on-write like other filesystem contents.) The user space accounting and quotas is reported by the new userused@user, groupused@group, userquota@user, and groupquota@group properties, and by the new zfs userspace and zfs groupspace subcommands, which are detailed below. 2. Version Compatibility To use these features, the pool must be upgraded to a new on-disk version (15). Old filesystems must have their space accounting information initialized by running zfs userspace fs or upgrading the old filesystem to a new on-disk version (4). To set user quotas, the pool and filesystem must both be upgraded. 3. Permissions Setting or changing user quotas are administrative actions, subject to the same privilege requirements as other zfs subcommands. There are new userquota and groupquota permissions which can be granted with zfs allow, to allow those properties to be viewed and changed. Unprivileged users can only view their own userquota and userused, and the groupquota and groupused of any groups they belong to. The new userused and groupused permissions can be granted with zfs allow to permit users to view these properties. The existing version permission (granted with zfs allow) permits the accounting information to be initialized by zfs userspace. 4. New Properties user/group space accounting information and quotas can be manipulated with 4 new properties: zfs get userused@user fs|snap zfs get groupused@group fs|snap zfs get userquota@user fs|snap zfs get groupquota@group fs|snap zfs set userquota@user=quota fs zfs set groupquota@user=quota fs The user or group is specified using one of the following forms: posix name (eg. ahrens) posix numeric id (eg. 126829) sid name (eg. ahr...@sun) sid numeric id (eg. S-1-12345-12423-125829) For zfs set, if a nonexistent name is specified, an error is
Re: [zfs-discuss] Data corruption during resilver operation
You are seeing snapshots from Time-Slider's automatic snapshot service. If you have a copy of each of these 58 files elsewhere, I suppose you could re-copy them to the mirror and then do 'zpool clear [poolname]' to reset the error counter. On Sun, Mar 29, 2009 at 10:28 PM, Harry Putnam rea...@newsguy.com wrote: I'm in well over my head with this report from zpool status saying: root # zpool status z3 pool: z3 state: DEGRADED status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scrub: resilver completed after 0h7m with 38 errors on Sun Mar 29 18:37:28 2009 config: NAME STATE READ WRITE CKSUM z3 DEGRADED 0 0 40 mirror DEGRADED 0 0 80 c5d0 DEGRADED 0 0 80 too many errors c6d0 DEGRADED 0 0 80 too many errors This is that last thing and apparently the result of a series of steps I've taken to increase a zpool mirrors size. There was quite a lot of huffing and puffing with the sata controller that holds this mirror but the short version is: zpool z3 created as mirror on 2 older 200gb SATAI disks. On an adaptec 1205sa PCI controller. After deciding I wanted increase the size of this pool, I detached 1 disk, then pulled it out. I replaced it with a newer bigger sata II wd750 gb disk. When I attempted to startup and attach this disk, I didn't get by the boot process, and discovered my sata controller could not handle the newer SATAII disk.. No boot was possible. I finally got the sata contoller in shape to work by flashing the 2 part BIOS with latest bios for that card. (Sil 3112a chip). Restarted with 1 original 200gb disk and 1 new 750gb disk. It booted and I was abble to attach the new larger drive and begin the resilvering process. I went on to other things, but when I checked back I found the error report cited above. I stared looking through the data but didn't really see much wrong. I check the byte size with `du -sb' on the zpool and the source of the data on a remote linux host. They were not the same but quite close. I didn't think that meant much since it was on different filesystems. zfs and reiserfs. I went to the web page cited in the report to see what I could learn. To summarize it said this was serious business. That data might not even be able to be removed but that for sure it needed to be replaced from clean backup. Using zpool status -v z3 I learned there were 51 files said to be corrupt. But when I looked at the files they were not part of the original data. The original data was put there by an rsync process from a remote host. and contained none of the named files. There files are of the form (wrapped for mail): z3/www/rea...@zfs-auto-snap:frequent-2009-03-29-18:55:\ /www/localhost/htdocs/lcweb/TrainingVids/VegasTraining/\ VegasTraiiningTransitions.avi (All on one line) I'm not at all clear on what this is. The part after the colon is what was rsynced over. The files that turned up in the report are all *.mov *.avi, *.mpg or *.pdf. I didn't make any snapshots, nor did I set anything to have them made automatically... so not sure where this snapshot came from or really even if it is in fact a snapshot. Is it somehow a product of the resilvering? When I go to the root of this filesystem (/www) and run a find command like: find . -name 'VegasTraiiningTransitions.avi' The file is found. I haven't been able to test if they play yet but wondering what this snapshot stuff means. And what I should do about it. The warning clearly suggests they must be replaced with good copies. That wouldn't be too big a deal, but I do still have the other new disk to insert and resilver. So what is the smart move here?... Replace the data before continuing with the enlargement of the pool? Or something else? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Growing a zpool mirror breaks on Adaptec 1205sa PCI
On Sat, Mar 28, 2009 at 6:57 PM, Ian Collins i...@ianshome.com wrote: Please stop top-posting to threads where everyone else is normal-posting, it mucks up the flow of the thread. Thanks, -- Ian. Apologies - top-posting seems to be the Gmail default (or I set it so long ago that I forgot it was there). ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Timeslider causing errors..?
Do you have more than one Boot Environment? pfexec beadm list On Mon, Mar 30, 2009 at 1:33 PM, Harry Putnam rea...@newsguy.com wrote: After messing around with Timeslider... I started getting errors and the frequent and hourly services were failing, causing the service to be put into maintenance status. Not really being sure what to do with it I first tried `svcadm restart' on them. But they went right back into maintenance mode. I did save the logs and have posted them here: Is it the failure to open crontab that's causing all the fuss? www.jtan.com/~reader/slider/disp.cgi I then turned the services for frequent and hourly clear off with `svcadm disable' I thought that would allow me to access the timeslider applet which had been showing only the same error as svcs -xv showed. But it didn't . the dialog still shows: Snapshot manager service dependency error but now of course: disabled svc:/system/filesystem/zfs/auto-snapshot:frequent disabled svc:/system/filesystem/zfs/auto-snapshot:hourly Before it showed some of the same stuff as `svcs -xv' (see the logs a url above) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Data corruption during resilver operation
Sounds like the best way - I was about to suggest that anyway :) On Mon, Mar 30, 2009 at 3:03 PM, Harry Putnam rea...@newsguy.com wrote: Blake blake.ir...@gmail.com writes: You are seeing snapshots from Time-Slider's automatic snapshot service. If you have a copy of each of these 58 files elsewhere, I suppose you could re-copy them to the mirror and then do 'zpool clear [poolname]' to reset the error counter. Thanks... I did try coping from the source to replace those but it didn't appear to make any difference... still got the errors. I finally just assumed I'd done something untoward during all the shuffle of upgrading a 200gb mirror to a 750 gb mirror and flashing the bios of the PCI sata controller card in the middle. So resorted to zpool destroy badpool Finished the switch from 200gb to 750gb with no zpool on either. Created the mirror using the 2 750gb disks. And finally rsynced the data across from a linux machine to the new zpool as before. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] j4200 drive carriers
no idea how many of these there are: http://www.google.com/products?q=570-1182hl=enshow=li 2009/3/30 Tim t...@tcsac.net: On Mon, Mar 30, 2009 at 3:56 AM, Mike Futerko m...@maytech.net wrote: Hello 1) Dual IO module option 2) Multipath support 3) Zone support [multi host connecting to same JBOD or same set of JBOD's connected in series. ] This sounds interesting - where I can read more about connecting two hosts to same J4200 etc? Thanks Mike FWIW, it looks like someone at Sun saw the complaints in this thread and or (more likely) had enough customer complaints. It appears you can buy the tray independently now. Although, it's $500 (so they're apparently made entirely of diamond and platinum). In Sun marketing's defense, that was a great way of making the drive prices seem reasonable. http://sunsolve.sun.com/handbook_pub/validateUser.do?target=Systems/J4200/components --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] About snapshots or versioned backups
you need zfs list -t snapshot by default, snapshots aren't shown in zfs list anymore, hence the -t option On Mon, Mar 30, 2009 at 11:41 AM, Harry Putnam rea...@newsguy.com wrote: Richard Elling richard.ell...@gmail.com writes: It can go very fine, though you'll need to set the parameters yourself, if you want to use different settings. A few weeks ago, I posted a way to see the settings, which the time slider admin tool won't show. There is a diminishing return for exposing such complexity, but you might try an RFE if you feel strongly about it. http://opensolaris.org/jive/thread.jspa?messageID=353761 I meant in terms of per directory not frequency. If I wanted to be able to go back in time with just /etc for example. Is it also possible? Possible? Yes, to some degree. But that is probably not worth the complexity involved. The contents of /etc just doesn't change very often. Snapshots are done on a per-file system basis and /etc doesn't really warrant a separate file system -- and I'm not sure you can separate /etc from /, since it is required early in the boot sequence. The part about etc not changing may be true after you've established a good setup. But while getting there, I've always found it a good choice for frequent backup. That has been on linux, not solaris. Maybe Osol doesn't use etc as much as linux systems. But then other directories may need more frequent backup than the filesystem they are on. I guess one could create a filesystem for such a directory. And I think I may be getting confused between filesystem snapshots and BE snapshots. The etc directory must be included in a BE. So would BE snapshots cover all of rpool? Or just `/' or are they even different. The whole scheme I see with gnu/bin/df is kind of confusing too. Its not really even clear if /etc is part of rpool. zfs list -r rpool doesn't show etc, just '/'so I guess not. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zfs using java
Can you list the exact command you used to launch the control panel? I'm not sure what tool you are referring to. 2009/3/25 Howard Huntley hhuntle...@comcast.net: I once installed ZFS on my home Sun Blade 100 and it worked fine on the sun blade 100 running solaris 10. I reinstalled Solaris 10 09 version and created a zpool which is not visible using the java control panel. When I attempt to run the Java control panel to manage the zfs system I receive an error message stating !Launch Error, No application is registered with this sun Java or I have no rights to use any applications that are registered, see my sys admin. Can any one tell me how to get this straightened out. I have been fooling around with it for some time now. Is any one is Jacksonville, Florida?? -- Howard Huntley Jr. MCP, MCSE Micro-Computer Systems Specialist ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] About snapshots or versioned backups
There is a bug where the automatic snapshot service dies if there are multiple boot environments. Do you have these? I think you can check with Update Manager. On Mon, Mar 30, 2009 at 7:20 PM, Harry Putnam rea...@newsguy.com wrote: you need zfs list -t snapshot by default, snapshots aren't shown in zfs list anymore, hence the -t option Yikes, I've got dozens of the things... I monkeyed around a bit with timeslider but thought I canceled out whatever settings I'd messed with. Frequent and hourly both are way to often for most of my data. I think I've kind of painted myself into a corner. I apparently turned on timeslider... but one pool had some kind of corruption problem that I fixed by zpool destroy entire pool. But I keep getting errors from timeslider that would put frequent and hourly into maintenance mode. Which meant I couldn't do anything with the timeslider applet. It seems a little light on robustness.. not able to be used if there is any problem. Finally I disabled both frequent and hourly... and of course then the timeslider I unusable because the services are off. I tried restarting them again after getting the offending pool rebuilt involving at least 2 reboots. But now restarting them, and immediately they go to maintenance mode. And of course the timeslider applet is useless. Looking at the log output its the same as what I posted earlier... in a different thread. www.jtan.com/~reader/slider/disp.cgi It appears to be related to not being able to open a crontab file. Doesn't say which but I see several in /var/pool/cron/crontabs ls -l /var/spool/cron/crontabs total 9 -rw--- 1 root sys 1004 2008-11-19 18:13 adm -r 1 root root 1365 2008-11-19 18:30 lp -rw--- 1 root root 1241 2009-03-30 17:15 root -rw--- 1 root sys 1122 2008-11-19 18:33 sys -rw--- 1 root daemon 394 2009-03-30 18:06 zfssnap So I'm not sure what the problem is. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Growing a zpool mirror breaks on Adaptec 1205sa PCI
Have you checked the specs of the 1205 to see what maximum drive size it supports? That's an older card, IIRC, so it might top out at 500gb or something. On Sat, Mar 28, 2009 at 10:33 AM, Harry Putnam rea...@newsguy.com wrote: casper@sun.com writes: I mentioned that pressing F3 doesn't do anything. That is, I have now way to enter the configuration tool. Does it work when you first remove the 200GB drive, reboot, press F3 and see what you can do in that configuration tool. It is possible that it first needs to forget the 200GB drive. OK, booted with no sata drives attached. At the press F3 screen, again pressing F3 appears to have no effect. But after a moment two messages saying `no drives found' print out, and then boot proceeds. Once booted up I see the recurring message where I should see a login prompt (I'm setup to boot into console mode). ata_id_common Busy Status 0xfe error 0x0 Repeated 4 times, then after maybe a 2-3 minute wait the regular login prompt appears. There is something more going on that is dragging the machine down so that typing at the prompt is delayed by huge pauses. I'm not really able to find out what it is yet but will post back later after a reboot. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] j4200 drive carriers
This is true. Unfortunately, in my experience, controller quality is still very important. ZFS can preserve data all day long, but that doesn't help much if the controller misbehaves (you may have good data that can't be retrieved or manipulated properly - it's happened to me with whitebox hardware). If anyone buys whitebox hardware for ZFS in production, make sure the vendor will give you support/warranty for OpenSolaris/ZFS. On Fri, Mar 27, 2009 at 7:33 PM, jpdrawneek j...@drawneek.demon.co.uk wrote: Mertol Ozyoney wrote: But the whole point of zfs is that you can use inexpensive drives and with enough in RAID to make it reliable. Best regards Mertol Mertol Ozyoney Storage Practice - Sales Manager Sun Microsystems, TR Istanbul TR Phone +902123352200 Mobile +905339310752 Fax +90212335 Email mertol.ozyo...@sun.com -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Richard Elling Sent: Sunday, February 01, 2009 10:24 PM To: John-Paul Drawneek Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] j4200 drive carriers John-Paul Drawneek wrote: the J series is far to new to be hitting ebay yet. Any alot of people will not be buying the J series for obvious reasons The obvious reason is that Sun cannot service random disk drives you buy from Fry's (or elsewhere). People who value data tend to value service contracts for disk drives. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] is zpool export/import | faster than rsync or cp
zfs send/recv *is* faster (especially since b105) than rsync, especially when you are dealing with lots of small files. rsync has to check each file, which can take a long time - zfs send/recv just moves blocks. 2009/3/27 Ahmed Kamal email.ahmedka...@googlemail.com: ZFS replication basics at http://cuddletech.com/blog/pivot/entry.php?id=984 Regards On Sat, Mar 28, 2009 at 1:57 AM, Harry Putnam rea...@newsguy.com wrote: [...] Harry wrote: Now I'm wondering if the export/import sub commands might not be a good bit faster. Ian Collins i...@ianshome.com answered: I think you are thinking of zfs send/receive. I've never done a direct comparison, but zfs send/receive would be my preferred way to move data between pools. Why is that? I'm too new to know what all it encompasses (and a bit dense to boot) Fajar A. Nugraha fa...@fajar.net writes: On Sat, Mar 28, 2009 at 5:05 AM, Harry Putnam rea...@newsguy.com wrote: Now I'm wondering if the export/import sub commands might not be a good bit faster. I believe the greatest advantage of zfs send/receive over rsync is not about speed, but rather it's on zfs send -R, which would (from man page) Generate a replication stream package, which will replicate the specified filesystem, and all descen- dant file systems, up to the named snapshot. When received, all properties, snapshots, descendent file systems, and clones are preserved. pretty much allows you to clone a complete pool preserving its structure. As usual, compressing the backup stream (whether rsync or zfs) might help reduce transfer time a lot. My favorite is lzop (since it's very fast), but gzip should work as well. Nice... good reasons it appears. Robert Milkowski mi...@task.gda.pl writes: Hello Harry, [...] As Ian pointed you want zfs send|receive and not import/export. For a first full copy zfs send not necessarily will be noticeably faster than rsync but it depends on data. If for example you have milions of small files zfs send could be much faster then rsync. But it shouldn't be slower in any case. zfs send|receive really shines when it comes to sending incremental changes. Now that would be something to make it stand out. Can you tell me a bit more about that would work..I mean would you just keep receiving only changes at one end and how do they appear on the filesystem. There is a backup tool called `rsnapshot' that uses rsync but creates hard links to all unchanged files and moves only changes to changed files. This is all put in a serial directory system and ends up taking a tiny fraction of the space that full backups would take, yet retains a way to get to unchanged files right in the same directory (the hard link). Is what your talking about similar in some way. = * = * = * = To all posters... many thanks for the input. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Growing a zpool mirror breaks on Adaptec 1205sa PCI
what's the output of 'fmadm faulty'? On Sat, Mar 28, 2009 at 12:22 PM, Harry Putnam rea...@newsguy.com wrote: Harry Putnam rea...@newsguy.com writes: Once booted up I see the recurring message where I should see a login prompt (I'm setup to boot into console mode). ata_id_common Busy Status 0xfe error 0x0 Repeated 4 times, then after maybe a 2-3 minute wait the regular login prompt appears. There is something more going on that is dragging the machine down so that typing at the prompt is delayed by huge pauses. I'm not really able to find out what it is yet but will post back later after a reboot. Those 4 lines continue to appear on ever boot now.. (still with old 200gb sata drives out. Seems when the boot prompt does appear there is something dragging down the os making logging in really sluggish. It appears to be the fmd daemon running periodically and driving resource drain on cpu up to 99 percent. That seems to continue to occur for at least several cycles watching it in `top'. They last maybe close to 1 minute then disappear for a while. Several minutes at least. The cylces seem to keep coming after being booted up for 10 minutes now. I'm not sure which logs to look in to see whats happing. But earlier today these showed up in /var/adm/messages: Mar 28 09:38:25 zfs fmd: [ID 441519 daemon.error] SUNW-MSG-ID: ZFS-8000-D3, TYPE: Fault, VER: 1, SEVERITY: Major Mar 28 09:38:25 zfs IMPACT: Fault tolerance of the pool may be compromised. The console login continues to be nearly useless with the delays and pauses while typing. However ssh in and the terminal I get seem to be less effected, or even un-effected so I can do things in some kind or reasonable way. I'm really not sure what to do though. I did `zpool destroy thatpool' on the pool that was on the sata drives. That appears to have worked, but didn't help with resource drain coming from `fmd' The tail of the log pointed to by svcs -l system/fmd shows: (/var/svc/log/system-fmd:default.log) [...] [ Mar 28 09:20:31 Enabled. ] [ Mar 28 09:25:11 Executing start method (/usr/lib/fm/fmd/fmd). ] [ Mar 28 09:28:24 Method start exited with status 0. ] [ Mar 28 09:53:50 Enabled. ] [ Mar 28 09:54:50 Executing start method (/usr/lib/fm/fmd/fmd). ] [ Mar 28 09:55:44 Method start exited with status 0. ] [ Mar 28 09:59:21 Stopping because service disabled. ] [ Mar 28 09:59:21 Executing stop method (:kill). ] [ Mar 28 10:01:09 Enabled. ] [ Mar 28 10:02:17 Executing start method (/usr/lib/fm/fmd/fmd). ] [ Mar 28 10:02:49 Method start exited with status 0. ] I have no idea is that is normal or what. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to increase rpool size in a VM?
You need to use 'installgrub' to get the right boot pits in place on your new disk. The manpage for installgrub is pretty helpful. On Wed, Mar 25, 2009 at 6:18 PM, Bob Doolittle robert.doolit...@sun.com wrote: Hi, I have a build 109 system installed in a VM, and my rpool capacity is getting close to full. Since it's a VM, I can easily increase the size of the disk, or add another, larger disk to the VM. What's the easiest strategy for increasing my capacity? I tried adding a 2nd larger disk, did a zpool attach, waited for resilvering to complete, did a zpool detach of the 1st disk, but then it seemed it couldn't find my grub menu... I couldn't figure out a way to simply add a 2nd disk to the rpool, it seems like it's limited to a single device. Suggestions? Please keep me on the reply list, I'm not subscribed to this list currently. Thanks, Bob ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zpools on USB zpool.cache zpool import
+1 On Mon, Mar 23, 2009 at 8:43 PM, Damon Atkins damon.atk...@yahoo.com.au wrote: PS it would be nice to have a zpool diskinfo devicepath reports if the device belongs to a zpool imported or not, and all the details about any zpool it can find on the disk. e.g. file-systems (zdb is only for ZFS engineers says the man page). 'zpool import' needs an option to list the file systems of a pool which is not yet imported and its properties so you can have more information about it before importing it. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Size discrepancy (beyond expected amount?)
Replies inline (I really would recommend reading the whole ZFS Best Practices guide a few times - many of your questions are answered in that document): On Fri, Mar 20, 2009 at 3:15 PM, Harry Putnam rea...@newsguy.com wrote: I didn't make it clear. 1 disk, the one with rpool on it is 60gb. The other 3 are 500GB. Using a 500gb to mirror a 60gb would be something of a waste .. eh? In the near-term, yes, but it would work. And is a mirror of rpool really important? I assumed there would be some way to backup rpool in a format that could be written onto a new disk and booted in the event of rpool disk failure. With the backup not kept on rpool disk. You say you want a storage server you can forget about - that sounds like zfs self-healing, which requires a mirror at least. Someone had suggested even creating an rpool mirror and putting the bootmanager bits on its mbr but then keeping it in storage instead of on the machine (freeing a controller port). This is not a replacement for live redundancy. It has the downside of having to be re mirrored every now and then. Actually, the point of a backup is to have a known-good copy of data somewhere. Re-mirroring would be a mistake, as it destroys your old data state. But could be safe enough if nothing of great importance was kept on the rpool... Just OS and config changes, some BE's. But nothing that couldn't be lost. Then in the event of disk failure... You'd have to just install the spare, boot it, and bring it up to date. Something kind of like what people do with ghost images on windows OS. Start with two mirrored pools of two disks each. In the future, you will be able to add two or more disks to your non-root pool. You can't do that with a RAIDZ pool. Well one thing there... if I use 5 500gb disks (no counting rpool disk - 6 total), by the time my raidz fills up, I'll need a whole new machine really since I'll be out of controller ports and its getting hard to find controllers that are not PCI express already. (My hardware is plain PCI only and even then the onboard sata is not recognized and I'm addding a PCI sata controller already) If the hardware is old/partially supported/flaky, all the more reason to use mirrors. Any single disk from a mirror can be used standalone. Big disks are cheap: http://tinyurl.com/5tzguf Also some of the older data will have outlived its usefulness so what needs transferring to a new setup may not be really hard to accommodate or insurmountable. And finally, I'm 65 yrs old... Its kind of hard to imagine my wife and I filling up the nearly 2tb of spc the above mentioned raidz1 would afford before we go before the old grim reaper. Even with lots of pictures and video projects thrown in. I'm really thinking now to go to 5 500gb disks in raidz1, and one hotswap (Plus the rpool on 1 60gb disk). I would be clear out of both sata and IDE controller ports then, so I'm hoping I can add a hot swap by pulling one of the raidz disks long enough to add the hotswap... then take it back out and replace the missing raidz disk. See the zfs docs for more about hot spares. The 'hot' part means the disk is in the chassis and spinning all the time, ready to replace a failed drive automatically. Not something easy work out the hardware for in a situation like yours. If you don't have room for yet another disk in the chassis, you won't be able to use a hot spare. I could do this by getting 3 more 500gb disks 2 more for the raidz and 1 for hotswap. No other hardware would be needed. All the while assuming I can mix 3 500GB IDE and 2 500GB SATA with no problems. If you need to, you can even detach one side of the mirror of each pool. You can't do that with a RAIDZ pool. If you need larger pools you can replace all the disks in both pools with larger disks. You can do that with a RAIDZ pool, but more flexibility exists with mirrored pools. 1. Yes, sensible. 2. Saving space isn't always the best configuration. 3. I don't know. 4. Yes, with more disks, you can identify hot spares to be used in the case of a disk failure. Nice thanks (To Bob F as well). And I'm not being hard headed about using a mirror config. Its just that I have limited controller ports (4 ide 2 sata), limited budget, and kind of wanted to get this backup machine setup to where I could basically just leave it alone and let the backups run. On 3) Mixing IDE and SATA on same zpool I'd really like to hear from someone who has done that. In my experience, zfs doesn't care what kind of block device you give it. About 4).. so if all controllers are already full with either a zpool or rpool. Do you pull out one of the raidz1 disks to add a hotswap then remove the hotswap and put the pulled disk from the raidz back? If so, does that cause some kind of resilvering or does some other thing happen when a machine is booted with a raidz1 disk misssing, and then rebooted with it back in place?
Re: [zfs-discuss] Size discrepancy (beyond expected amount?)
IIRC, that's about right. If you look at the zfs best practices wiki (genunix.org I think?), there should be some space calculations linked in there somewhere. On Thu, Mar 19, 2009 at 6:50 PM, Harry Putnam rea...@newsguy.com wrote: I'm finally getting close to the setup I wanted, after quite a bit of experimentation and bugging these lists endlessly. So first, thanks for your tolerance and patience. My setup consists of 4 disks. One holds the OS (rpool) and 3 more all the same model and brand, all 500gb. I've created a zpool in raidz1 configuration with: zpool create zbk raidz1 c3d0 c4d0 c4d1 No errors showed up and zpool status shows no problems with those three: pool: zbk state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM zbk ONLINE 0 0 0 raidz1 ONLINE 0 0 0 c3d0 ONLINE 0 0 0 c4d0 ONLINE 0 0 0 c4d1 ONLINE 0 0 0 However, I appear to have lost an awfull lot of space... even above what I expercted. df -h [...] zbk 913G 26K 913G 1% /zbk It appears something like 1 entire disk is gobbled up by raidz1. The same disks configured in zpool with no raidz1 shows 1.4tb with df. I was under the impression raidz1 would take something like 20%.. but this is more like 33.33%. So, is this to be expected or is something wrong here? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Size discrepancy (beyond expected amount?)
This verifies my guess: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#RAID-Z_Configuration_Requirements_and_Recommendations On Thu, Mar 19, 2009 at 6:57 PM, Blake blake.ir...@gmail.com wrote: IIRC, that's about right. If you look at the zfs best practices wiki (genunix.org I think?), there should be some space calculations linked in there somewhere. On Thu, Mar 19, 2009 at 6:50 PM, Harry Putnam rea...@newsguy.com wrote: I'm finally getting close to the setup I wanted, after quite a bit of experimentation and bugging these lists endlessly. So first, thanks for your tolerance and patience. My setup consists of 4 disks. One holds the OS (rpool) and 3 more all the same model and brand, all 500gb. I've created a zpool in raidz1 configuration with: zpool create zbk raidz1 c3d0 c4d0 c4d1 No errors showed up and zpool status shows no problems with those three: pool: zbk state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM zbk ONLINE 0 0 0 raidz1 ONLINE 0 0 0 c3d0 ONLINE 0 0 0 c4d0 ONLINE 0 0 0 c4d1 ONLINE 0 0 0 However, I appear to have lost an awfull lot of space... even above what I expercted. df -h [...] zbk 913G 26K 913G 1% /zbk It appears something like 1 entire disk is gobbled up by raidz1. The same disks configured in zpool with no raidz1 shows 1.4tb with df. I was under the impression raidz1 would take something like 20%.. but this is more like 33.33%. So, is this to be expected or is something wrong here? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Size discrepancy (beyond expected amount?)
I'd be careful about raidz unless you have either: 1 - automatic notification of failure set up using fmadm 2 - at least one hot spare Because raidz is parity-based (it does some math-magic to give you redundancy), replacing a disk that's failed can take a very long time compared to mirror resilvering (the zfs term for rebuilding redundancy). You can get a nice 1000gb SATA drive on newegg or a similar site for about $90 - well worth the extra money ($120) for the convenience of mirroring. Mirrors are probably also faster for any kind of video playback (like the video projects you mention). I use raidz2 with 2 hot spares at my company, yes, but only for data warehousing. User data (windows home dirs) I put on mirrors. On Thu, Mar 19, 2009 at 10:00 PM, Harry Putnam rea...@newsguy.com wrote: Bob Friesenhahn bfrie...@simple.dallas.tx.us writes: On Thu, 19 Mar 2009, Harry Putnam wrote: I've created a zpool in raidz1 configuration with: zpool create zbk raidz1 c3d0 c4d0 c4d1 This is not a very useful configuration. With this number of disks, it is best to use two of them to build a mirror, and save the other disk for something else (e.g. to build a mirrored root pool). Mirrors perform better, are more fault tolerant, and are easier to administer. Ok, I was going by comments on a site called Simons' blog that tells you how to set things up with zfs. Of course it is just one guys opinion. Let me just say a couple of words about my intended usage. I'm building a home NAS server for my home lan using opensolaris and zfs. I will be backing up 4 Windows XP boxes, 2 of which are used primarily for processing video. And eventully I'd be storing finished video projects too. Often these projects run to 50gb or so. But I don't do so many. Maybe 9-12 in a yr. So I'd want to put ghosted images of each machine OS (several apiece) and running backups (incremental) that would stretch back a few weeks. And the projects mentioned above. As well as what is becoming quite a large photo collection. Also 2 linux boxes will getting backed up there. I thought the kind of redundancy offered by raidz1 would be enough for my needs and would allow me to get a little more storage room out of my disks than mirrored setup. I suspect as well, that the access times in raidz1 or mirrored would be vastly higher than what the low end consumer grade NASs that are availabe offer. I did try one out.. a WD Worldbook (about $200 US) that advertises gigabit access but in use cannot even come close to what a 10/100 connect can handle. So raidz1 would probably be adequate for me... I wouldn't be putting it to the test like a commercial operation might. You mentioned admistration was a bigger problem with raidz1, can you be more specific there?... I have really know idea what to expect in that regard with either technique. With five disks, raidz1 becomes useful. The three 500gb I have now are all one brand and model number and IDE ata. If I were to expand to 5, those 2 would need to be sata or else I'd also need to add a PCI IDE controller. With that in mind would it be problematic to make up the 5 by adding 2 sata 500gb to the mix? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Freezing OpenSolaris with ZFS
This sounds quite like the problems I've been having with a spotty sata controller and/or motherboard. See my thread from last week about copying large amounts of data that forced a reboot. Lots of good info from engineers and users in that thread. On Sun, Mar 15, 2009 at 1:17 PM, Markus Denhoff denh...@net-bite.net wrote: Hi there, we set up an OpenSolaris/ZFS based storage server with two zpools: rpool is a mirror for the operating system. tank is a raidz for data storage. The system is used to store large video files and has attached 12x1GB SATA-drives (2 mirrored for the system). Everytime large files are copied around the system hangs without apparent reason, 50% kernel CPU usage (so one core is occupied totally) and about 2GB of free RAM (8GB installed). On idle nothing crashes. Furthermore every scrub on tank hangs the system up below 1% finished. Neither the /var/adm/messages nor the /var/log/syslog file contains any errors or warnings. We limited the ZFS ARC cache to 4GB with an entry in /etc/system. Does anyone has an idea what's happening there and how to solve the problem? Below some outputs which may help. Thanks and greetings from germany, Markus Denhoff, Sebastian Friederichs # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz1 ONLINE 0 0 0 c6t2d0 ONLINE 0 0 0 c6t3d0 ONLINE 0 0 0 c6t4d0 ONLINE 0 0 0 c6t5d0 ONLINE 0 0 0 c6t6d0 ONLINE 0 0 0 c6t7d0 ONLINE 0 0 0 c6t8d0 ONLINE 0 0 0 c6t9d0 ONLINE 0 0 0 c6t10d0 ONLINE 0 0 0 c6t11d0 ONLINE 0 0 0 errors: No known data errors # zpool iostat capacity operations bandwidth pool used avail read write read write -- - - - - - - rpool 37.8G 890G 3 2 94.7K 17.4K tank 2.03T 7.03T 112 0 4.62M 906 -- - - - - - - # zfs list NAME USED AVAIL REFER MOUNTPOINT rpool 39.8G 874G 72K /rpool rpool/ROOT 35.7G 874G 18K legacy rpool/ROOT/opensolaris 35.6G 874G 35.3G / rpool/ROOT/opensolaris-1 89.9M 874G 2.47G /tmp/tmp8CN5TR rpool/dump 2.00G 874G 2.00G - rpool/export 172M 874G 19K /export rpool/export/home 172M 874G 21K /export/home rpool/swap 2.00G 876G 24K - tank 1.81T 6.17T 32.2K /tank tank/data 1.81T 6.17T 1.77T /data tank/public-share 34.9K 6.17T 34.9K /public-share ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS GSoC ideas page rough draft
I just thought of an enhancement to zfs that would be very helpful in disaster recovery situations - having zfs cache device serial/model numbers - the information we see in cfgadm -v. I'm feeling the pain of this now as I try to figure out which disks on my failed filer belonged to my raidz2 pool - zpool status tells me the pool is faulted (I don't have enough working SATA ports to connect all the drives from the pool), but doesn't tell me which individual devices were in that pool (just the devids of the devices). On Mon, Mar 9, 2009 at 9:30 AM, C. cbergst...@netsyncro.com wrote: Here's my rough draft of GSoC ideas http://www.osunix.org/docs/DOC-1022 Also want to thank everyone for their feedback. Please keep in mind that for creating a stronger application we only have a few days. We still need to : 1) Find more mentors. (Please add your name to the doc or confirm via email and which idea you're most interested in) 2) Add contacts from each organization that may be interested (OpenSolaris, FreeBSD...) 3) Finalize the application, student checklist, mentor checklist and template 4) Start to give ideas for very accurate project descriptions/details (We have some time for this) Thanks ./Christopher --- Community driven OpenSolaris Technology - http://www.osunix.org blog: http://www.codestrom.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS GSoC ideas page rough draft
That's excellent information Richard. I have just exported the faulted pool (the disks aren't attached), but what you are suggesting is verified by looking at the cache as it is - I can see this information for my rpool mirror disks. One other note - I see that using the old prtvtoc | fmthard trick to create my 2nd mirror disk for the rpool created a generic label, and it looks like zdb only stores the disk label: # format AVAILABLE DISK SELECTIONS: 0. c4t0d0 ATA-ST3250310NS-SN04 cyl 30398 alt 2 hd 255 sec 63 /p...@0,0/pci15d9,1...@5/d...@0,0 1. c4t1d0 DEFAULT cyl 30398 alt 2 hd 255 sec 63 /p...@0,0/pci15d9,1...@5/d...@1,0 # zdb ... children[0] type='disk' id=0 guid=9660743866801172401 path='/dev/dsk/c4t0d0s0' devid='id1,s...@f00439638498493aa00037d740001/a' phys_path='/p...@0,0/pci15d9,1...@5/d...@0,0:a' whole_disk=0 DTL=101 children[1] type='disk' id=1 guid=5364095963942871931 path='/dev/dsk/c4t1d0s0' devid='id1,s...@ast3250310ns=9sf06bc8/a' phys_path='/p...@0,0/pci15d9,1...@5/d...@1,0:a' whole_disk=0 DTL=99 Cfgadm, however, returns the full serial number and model number for each disk. So I think storing the information cfgadm collects in zpool.cache would be preferable to reading disk labels. On Sat, Mar 14, 2009 at 12:47 PM, Richard Elling richard.ell...@gmail.com wrote: Toby Thain wrote: On 14-Mar-09, at 12:09 PM, Blake wrote: I just thought of an enhancement to zfs that would be very helpful in disaster recovery situations - having zfs cache device serial/model numbers - the information we see in cfgadm -v. +1 I haven't needed this but it sounds very sensible. I can imagine it could help a lot in some drive replacement situations. What do your devids look like? Mine are something like: devid='id1,s...@sata_st3500320as_9qm3fwft/g' Where ST3500320AS is the model number and 9QM3FWFT is the serial number. These are stored verbatim in the zpool.cache. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] What to do with a disk partition
I think you will be helped by looking at this document: http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide#ZFS_Root_Pool_Recommendations_and_Requirements It addresses many of your questions. I think the easiest way to back up your OS might be to attach a disk to the rpool as a mirror, use 'installgrub' to get the grub boot blocks onto the new mirror disk, then detach this disk and put it in storage. There are also more advanced backup methods covered in the document above. On Sat, Mar 14, 2009 at 11:51 PM, Harry Putnam rea...@newsguy.com wrote: I'm still learning and haven't setup anything I can't destroy if it can be better put together some other way. I would like to save any updates and customizations so far if possible. But that isn't critical either. Originally I installed osol-11 101b (now at 108) on a 60gb disk partitioned 25/45 with the OS on the 25. These were just fdisk partitions, no slices. I've since installed 2 more drives and have 2 more to install once I get a working sata controller. The purpose of this server once I quit tinkering so much and decide I know enough to put it into service is just a home backup server. So currently I have 3 disks to work with. Disk 0 (60gb) as mentioned holds the OS on a 25gb fdisk partition c3d0p1. And another zpool on the remaining 45gb at c3d0p2 Disk 1 (250gb) 1 fdisk partition (full disk) Disk 2 ( 500gb 1 fdisk partition (full disk) I've done nothing so far with the last 2, just did get them installed but I had planned to put them into a raidz1 configuration, which would give me 750GB in that zpool, and use them to backup 4 windows machines and one linux machine. That is, assuming that isn't a nitwit thing to do and some other configuration would be better. Any opinions/suggestions .. welcome. I'd like some opinions on how best to utilize the remaining 45gb partition on disk 0. The one with the OS on the other partition. I originally thought I'd use it to backup the OS itself, but after listening here a while I wonder if that would be better served if I had made rpool encompass that whole disk. Instead of having rpool as the OS of 25gb on c3d0p1 and rpool.bk as a zpool on the other 45gb partition at c3d0p2. I guess I'm asking if its wise to have more than 1 zpool on a disk? Especially if one of them holds the OS. Or is it wiser to reinstall on a single partition of the whole disk? How do people backup there OS, in general. Just keep a herd of snapshots letting the oldest be destroyed over time...? Is Keeping those on the same zpool and disk as the OS wise? Its a home setup so I don't need to get too fancy like having backups offsite or whatever.. nobody elses data will be at risk. Of course an irate wife with lost data might not be too pretty of picture... ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] reboot when copying large amounts of data
This is really great information, though most of the controllers mentioned aren't on the OpenSolaris HCL. Seems like that should be corrected :) My thanks to the community for their support. On Mar 12, 2009, at 10:42 PM, James C. McPherson james.mcpher...@sun.com wrote: On Thu, 12 Mar 2009 22:24:12 -0400 Miles Nordin car...@ivy.net wrote: wm == Will Murnane will.murn...@gmail.com writes: * SR = Software RAID IT = Integrate. Target mode. IR mode is not supported. wm Integrated target mode lets you export some storage attached wm to the host system (through another adapter, presumably) as a wm storage device. IR mode is almost certainly Internal RAID, wm which that card doesn't have support for. no, the supermicro page for AOC-USAS-L8i does claim support for all three, and supermicro has an ``IR driver'' available for download for Linux and Windows, or at least a link to one. I'm trying to figure out what's involved in determining and switching modes, why you'd want to switch them, what cards support which modes, which solaris drivers support which modes, u.s.w. The answer may be very simple, like ``the driver supports only IR. Most cards support IR, and cards that don't support IR won't work. IR can run in single-LUN mode. Some IR cards support RAID5, others support only RAID 0, 1, 10.'' Or it could be ``the driver supports only SR. The driver is what determines the mode, and it does this by loading firmware into the card, and the first step in initializing the card is always for the driver to load in a firmware blob. All currently-produced cards support SR.'' so...actually, now that I say it, I guess the answer cannot be very simple. It's going to have to be a little complicated. Anyway, I can guess, too. I was hoping someone would know for sure off-hand. Hi Miles, the mpt(7D) driver supports that card. mpt(7D) supports both IT and IR firmware variants. You can find out the specifics for what RAID volume levels are supported by reading the raidctl(1M) manpage. I don't think you can switch between IT and IR firmware, but not having needed to know this before, I haven't tried it. James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcphttp://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] reboot when copying large amounts of data
I start the cp, and then, with prstat -a, watch the cpu load for the cp process climb to 25% on a 4-core machine. Load, measured for example with 'uptime', climbs steadily until the reboot. Note that the machine does not dump properly, panic or hang - rather, it reboots. I attached a screenshot earlier in this thread of the little bit of error message I could see on the console. The machine is trying to dump to the dump zvol, but fails to do so. Only sometimes do I see an error on the machine's local console - mos times, it simply reboots. On Thu, Mar 12, 2009 at 1:55 AM, Nathan Kroenert nathan.kroen...@sun.com wrote: Hm - Crashes, or hangs? Moreover - how do you know a CPU is pegged? Seems like we could do a little more discovery on what the actual problem here is, as I can read it about 4 different ways. By this last piece of information, I'm guessing the system does not crash, but goes really really slow?? Crash == panic == we see stack dump on console and try to take a dump hang == nothing works == no response - might be worth looking at mdb -K or booting with a -k on the boot line. So - are we crashing, hanging, or something different? It might simply be that you are eating up all your memory, and your physical backing storage is taking a while to catch up? Nathan. Blake wrote: My dump device is already on a different controller - the motherboards built-in nVidia SATA controller. The raidz2 vdev is the one I'm having trouble with (copying the same files to the mirrored rpool on the nVidia controller work nicely). I do notice that, when using cp to copy the files to the raidz2 pool, load on the machine climbs steadily until the crash, and one proc core pegs at 100%. Frustrating, yes. On Thu, Mar 12, 2009 at 12:31 AM, Maidak Alexander J maidakalexand...@johndeere.com wrote: If you're having issues with a disk contoller or disk IO driver its highly likely that a savecore to disk after the panic will fail. I'm not sure how to work around this, maybe a dedicated dump device not on a controller that uses a different driver then the one that you're having issues with? -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Blake Sent: Wednesday, March 11, 2009 4:45 PM To: Richard Elling Cc: Marc Bevand; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] reboot when copying large amounts of data I guess I didn't make it clear that I had already tried using savecore to retrieve the core from the dump device. I added a larger zvol for dump, to make sure that I wasn't running out of space on the dump device: r...@host:~# dumpadm Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/bigdump (dedicated) Savecore directory: /var/crash/host Savecore enabled: yes I was using the -L option only to try to get some idea of why the system load was climbing to 1 during a simple file copy. On Wed, Mar 11, 2009 at 4:58 PM, Richard Elling richard.ell...@gmail.com wrote: Blake wrote: I'm attaching a screenshot of the console just before reboot. The dump doesn't seem to be working, or savecore isn't working. On Wed, Mar 11, 2009 at 11:33 AM, Blake blake.ir...@gmail.com wrote: I'm working on testing this some more by doing a savecore -L right after I start the copy. savecore -L is not what you want. By default, for OpenSolaris, savecore on boot is disabled. But the core will have been dumped into the dump slice, which is not used for swap. So you should be able to run savecore at a later time to collect the core from the last dump. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- // // Nathan Kroenert nathan.kroen...@sun.com // // Systems Engineer Phone: +61 3 9869-6255 // // Sun Microsystems Fax: +61 3 9869-6288 // // Level 7, 476 St. Kilda Road Mobile: 0419 305 456 // // Melbourne 3004 Victoria Australia // // ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] reboot when copying large amounts of data
So, if I boot with the -k boot flags (to load the kernel debugger?) what do I need to look for? I'm no expert at kernel debugging. I think this is a pci error judging by the console output, or at least is i/o related... thanks for your feedback, Blake On Thu, Mar 12, 2009 at 2:18 AM, Nathan Kroenert nathan.kroen...@sun.com wrote: definitely time to bust out some mdb -K or boot -k and see what it's moaning about. I did not see the screenshot earlier... sorry about that. Nathan. Blake wrote: I start the cp, and then, with prstat -a, watch the cpu load for the cp process climb to 25% on a 4-core machine. Load, measured for example with 'uptime', climbs steadily until the reboot. Note that the machine does not dump properly, panic or hang - rather, it reboots. I attached a screenshot earlier in this thread of the little bit of error message I could see on the console. The machine is trying to dump to the dump zvol, but fails to do so. Only sometimes do I see an error on the machine's local console - mos times, it simply reboots. On Thu, Mar 12, 2009 at 1:55 AM, Nathan Kroenert nathan.kroen...@sun.com wrote: Hm - Crashes, or hangs? Moreover - how do you know a CPU is pegged? Seems like we could do a little more discovery on what the actual problem here is, as I can read it about 4 different ways. By this last piece of information, I'm guessing the system does not crash, but goes really really slow?? Crash == panic == we see stack dump on console and try to take a dump hang == nothing works == no response - might be worth looking at mdb -K or booting with a -k on the boot line. So - are we crashing, hanging, or something different? It might simply be that you are eating up all your memory, and your physical backing storage is taking a while to catch up? Nathan. Blake wrote: My dump device is already on a different controller - the motherboards built-in nVidia SATA controller. The raidz2 vdev is the one I'm having trouble with (copying the same files to the mirrored rpool on the nVidia controller work nicely). I do notice that, when using cp to copy the files to the raidz2 pool, load on the machine climbs steadily until the crash, and one proc core pegs at 100%. Frustrating, yes. On Thu, Mar 12, 2009 at 12:31 AM, Maidak Alexander J maidakalexand...@johndeere.com wrote: If you're having issues with a disk contoller or disk IO driver its highly likely that a savecore to disk after the panic will fail. I'm not sure how to work around this, maybe a dedicated dump device not on a controller that uses a different driver then the one that you're having issues with? -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Blake Sent: Wednesday, March 11, 2009 4:45 PM To: Richard Elling Cc: Marc Bevand; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] reboot when copying large amounts of data I guess I didn't make it clear that I had already tried using savecore to retrieve the core from the dump device. I added a larger zvol for dump, to make sure that I wasn't running out of space on the dump device: r...@host:~# dumpadm Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/bigdump (dedicated) Savecore directory: /var/crash/host Savecore enabled: yes I was using the -L option only to try to get some idea of why the system load was climbing to 1 during a simple file copy. On Wed, Mar 11, 2009 at 4:58 PM, Richard Elling richard.ell...@gmail.com wrote: Blake wrote: I'm attaching a screenshot of the console just before reboot. The dump doesn't seem to be working, or savecore isn't working. On Wed, Mar 11, 2009 at 11:33 AM, Blake blake.ir...@gmail.com wrote: I'm working on testing this some more by doing a savecore -L right after I start the copy. savecore -L is not what you want. By default, for OpenSolaris, savecore on boot is disabled. But the core will have been dumped into the dump slice, which is not used for swap. So you should be able to run savecore at a later time to collect the core from the last dump. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- // // Nathan Kroenert nathan.kroen...@sun.com // // Systems Engineer Phone: +61 3 9869-6255 // // Sun Microsystems Fax: +61 3 9869-6288 // // Level 7, 476 St. Kilda Road Mobile: 0419 305 456 // // Melbourne 3004 Victoria Australia
Re: [zfs-discuss] User quota design discussion..
That is pretty freaking cool. On Thu, Mar 12, 2009 at 11:38 AM, Eric Schrock eric.schr...@sun.com wrote: Note that: 6501037 want user/group quotas on ZFS Is already committed to be fixed in build 113 (i.e. in the next month). - Eric On Thu, Mar 12, 2009 at 12:04:04PM +0900, Jorgen Lundman wrote: In the style of a discussion over a beverage, and talking about user-quotas on ZFS, I recently pondered a design for implementing user quotas on ZFS after having far too little sleep. It is probably nothing new, but I would be curious what you experts think of the feasibility of implementing such a system and/or whether or not it would even realistically work. I'm not suggesting that someone should do the work, or even that I will, but rather in the interest of chatting about it. Feel free to ridicule me as required! :) Thoughts: Here at work we would like to have user quotas based on uid (and presumably gid) to be able to fully replace the NetApps we run. Current ZFS are not good enough for our situation. We simply can not mount 500,000 file-systems on all the NFS clients. Nor do all servers we run support mirror-mounts. Nor do auto-mount see newly created directories without a full remount. Current UFS-style-user-quotas are very exact. To the byte even. We do not need this precision. If a user has 50MB of quota, and they are able to reach 51MB usage, then that is acceptable to us. Especially since they have to go under 50MB to be able to write new data, anyway. Instead of having complicated code in the kernel layer, slowing down the file-system with locking and semaphores (and perhaps avoiding learning indepth ZFS code?), I was wondering if a more simplistic setup could be designed, that would still be acceptable. I will use the word 'acceptable' a lot. Sorry. My thoughts are that the ZFS file-system will simply write a 'transaction log' on a pipe. By transaction log I mean uid, gid and 'byte count changed'. And by pipe I don't necessarily mean pipe(2), but it could be a fifo, pipe or socket. But currently I'm thinking '/dev/quota' style. User-land will then have a daemon, whether or not it is one daemon per file-system or really just one daemon does not matter. This process will open '/dev/quota' and empty the transaction log entries constantly. Take the uid,gid entries and update the byte-count in its database. How we store this database is up to us, but since it is in user-land it should have more flexibility, and is not as critical to be fast as it would have to be in kernel. The daemon process can also grow in number of threads as demand increases. Once a user's quota reaches the limit (note here that /the/ call to write() that goes over the limit will succeed, and probably a couple more after. This is acceptable) the process will blacklist the uid in kernel. Future calls to creat/open(CREAT)/write/(insert list of calls) will be denied. Naturally calls to unlink/read etc should still succeed. If the uid goes under the limit, the uid black-listing will be removed. If the user-land process crashes or dies, for whatever reason, the buffer of the pipe will grow in the kernel. If the daemon is restarted sufficiently quickly, all is well, it merely needs to catch up. If the pipe does ever get full and items have to be discarded, a full-scan will be required of the file-system. Since even with UFS quotas we need to occasionally run 'quotacheck', it would seem this too, is acceptable (if undesirable). If you have no daemon process running at all, you have no quotas at all. But the same can be said about quite a few daemons. The administrators need to adjust their usage. I can see a complication with doing a rescan. How could this be done efficiently? I don't know if there is a neat way to make this happen internally to ZFS, but from a user-land only point of view, perhaps a snapshot could be created (synchronised with the /dev/quota pipe reading?) and start a scan on the snapshot, while still processing kernel log. Once the scan is complete, merge the two sets. Advantages are that only small hooks are required in ZFS. The byte updates, and the blacklist with checks for being blacklisted. Disadvantages are that it is loss of precision, and possibly slower rescans? Sanity? But I do not really know the internals of ZFS, so I might be completely wrong, and everyone is laughing already. Discuss? Lund -- Jorgen Lundman | lund...@lundman.net Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work) Shibuya-ku, Tokyo | +81 (0)90-5578-8500 (cell) Japan | +81 (0)3 -3375-1767 (home) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Eric Schrock, Fishworks http://blogs.sun.com/eschrock ___
Re: [zfs-discuss] reboot when copying large amounts of data
I've managed to get the data transfer to work by rearranging my disks so that all of them sit on the integrated SATA controller. So, I feel pretty certain that this is either an issue with the Supermicro aoc-sat2-mv8 card, or with PCI-X on the motherboard (though I would think that the integrated SATA would also be using the PCI bus?). The motherboard, for those interested, is an HD8ME-2 (not, I now find after buying this box from Silicon Mechanics, a board that's on the Solaris HCL...) http://www.supermicro.com/Aplus/motherboard/Opteron2000/MCP55/h8dme-2.cfm So I'm not considering one of LSI's HBA's - what do list members think about this device: http://www.provantage.com/lsi-logic-lsi00117~7LSIG03X.htm On Thu, Mar 12, 2009 at 2:18 AM, Nathan Kroenert nathan.kroen...@sun.com wrote: definitely time to bust out some mdb -K or boot -k and see what it's moaning about. I did not see the screenshot earlier... sorry about that. Nathan. Blake wrote: I start the cp, and then, with prstat -a, watch the cpu load for the cp process climb to 25% on a 4-core machine. Load, measured for example with 'uptime', climbs steadily until the reboot. Note that the machine does not dump properly, panic or hang - rather, it reboots. I attached a screenshot earlier in this thread of the little bit of error message I could see on the console. The machine is trying to dump to the dump zvol, but fails to do so. Only sometimes do I see an error on the machine's local console - mos times, it simply reboots. On Thu, Mar 12, 2009 at 1:55 AM, Nathan Kroenert nathan.kroen...@sun.com wrote: Hm - Crashes, or hangs? Moreover - how do you know a CPU is pegged? Seems like we could do a little more discovery on what the actual problem here is, as I can read it about 4 different ways. By this last piece of information, I'm guessing the system does not crash, but goes really really slow?? Crash == panic == we see stack dump on console and try to take a dump hang == nothing works == no response - might be worth looking at mdb -K or booting with a -k on the boot line. So - are we crashing, hanging, or something different? It might simply be that you are eating up all your memory, and your physical backing storage is taking a while to catch up? Nathan. Blake wrote: My dump device is already on a different controller - the motherboards built-in nVidia SATA controller. The raidz2 vdev is the one I'm having trouble with (copying the same files to the mirrored rpool on the nVidia controller work nicely). I do notice that, when using cp to copy the files to the raidz2 pool, load on the machine climbs steadily until the crash, and one proc core pegs at 100%. Frustrating, yes. On Thu, Mar 12, 2009 at 12:31 AM, Maidak Alexander J maidakalexand...@johndeere.com wrote: If you're having issues with a disk contoller or disk IO driver its highly likely that a savecore to disk after the panic will fail. I'm not sure how to work around this, maybe a dedicated dump device not on a controller that uses a different driver then the one that you're having issues with? -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Blake Sent: Wednesday, March 11, 2009 4:45 PM To: Richard Elling Cc: Marc Bevand; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] reboot when copying large amounts of data I guess I didn't make it clear that I had already tried using savecore to retrieve the core from the dump device. I added a larger zvol for dump, to make sure that I wasn't running out of space on the dump device: r...@host:~# dumpadm Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/bigdump (dedicated) Savecore directory: /var/crash/host Savecore enabled: yes I was using the -L option only to try to get some idea of why the system load was climbing to 1 during a simple file copy. On Wed, Mar 11, 2009 at 4:58 PM, Richard Elling richard.ell...@gmail.com wrote: Blake wrote: I'm attaching a screenshot of the console just before reboot. The dump doesn't seem to be working, or savecore isn't working. On Wed, Mar 11, 2009 at 11:33 AM, Blake blake.ir...@gmail.com wrote: I'm working on testing this some more by doing a savecore -L right after I start the copy. savecore -L is not what you want. By default, for OpenSolaris, savecore on boot is disabled. But the core will have been dumped into the dump slice, which is not used for swap. So you should be able to run savecore at a later time to collect the core from the last dump. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs
[zfs-discuss] reboot when copying large amounts of data
I have a H8DM8-2 motherboard with a pair of AOC-SAT2-MV8 SATA controller cards in a 16-disk Supermicro chassis. I'm running OpenSolaris 2008.11, and the machine performs very well unless I start to copy a large amount of data to the ZFS (software raid) array that's on the Supermicro SATA controllers. If I do this, the machine inevitably reboots. What can I do to troubleshoot? The BIOS of the motherboard and the SATA card firmware are fully updated. I'm running the latest stable OpenSolaris, and see nothing amiss in the system logs when this happens. I've enabled savecore and debug-level syslog, but am getting no indicators from Solaris as to what's wrong. Interestingly, I can push the same amount of data to the mirror boot disks, which are on the board's built-in nVidia SATA controller without issue. The vdev I'm pushing to is a 5-disk raidz2 with 2 hot spares. Help! :) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] reboot when copying large amounts of data
I'm working on testing this some more by doing a savecore -L right after I start the copy. BTW, I'm copying to a raidz2 of only 5 disks, not 16 (the chassis supports 16, but isn't fully populated). So far as I know, there is no spinup happening - these are not RAID controllers, just dumb SATA JBOD controllers, so I don't think they control drive spin in any particular way. Correct me if I'm wrong, of course. On Wed, Mar 11, 2009 at 11:23 AM, Marc Bevand m.bev...@gmail.com wrote: The copy operation will make all the disks start seeking at the same time and will make your CPU activity jump to a significant percentage to compute the ZFS checksum and RAIDZ parity. I think you could be overloading your PSU because of the sudden increase in power consumption... However if you are *not* using SATA staggered spin-up, then the above theory is unlikely because spinning up consumes much more power than when seeking. So, in a sense, a successful boot proves your PSU is powerful enough. Trying reproducing the problem by copying data on a smaller number of disks. You tried 2 and 16. Try 8. -marc ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Export ZFS via ISCSI to Linux - Is it stable for production use now?
I blogged this a while ago: http://blog.clockworm.com/2007/10/connecting-linux-centos-5-to-solaris.html On Wed, Mar 11, 2009 at 1:02 PM, howard chen howac...@gmail.com wrote: Hello, On Wed, Mar 11, 2009 at 10:20 PM, Darren J Moffat darr...@opensolaris.org wrote: 1. Is this setup suitable for mission critical use now? Yes, why wouldn't it be ? Because I just wonder why some other people are using zfs/fuse on Linux, e.g. http://www.drwetter.org/blog/zfs_under_linux.en.html http://www.wizy.org/wiki/ZFS_on_FUSE Also seems hard to find a complete tutorial on accessing ZFS from Linux using ISCSI. This should be attractive, isn't? So I don't know if it is experimental. Howard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] reboot when copying large amounts of data
fmdump is not helping much: r...@host:~# fmdump -eV TIME CLASS fmdump: /var/fm/fmd/errlog is empty comparing that screenshot to the output of cfgadm is interesting - looks like the controller(s): r...@host:~# cfgadm -v Ap_Id Receptacle Occupant Condition Information When Type Busy Phys_Id sata4/0::dsk/c4t0d0connectedconfigured ok Mod: ST3250310NS FRev: SN06 SN: 9SF06CZZ unavailable disk n/devices/p...@0,0/pci15d9,1...@5:0 sata4/1::dsk/c4t1d0connectedconfigured ok Mod: ST3250310NS FRev: SN06 SN: 9SF06BC8 unavailable disk n/devices/p...@0,0/pci15d9,1...@5:1 sata5/0emptyunconfigured ok unavailable sata-portn/devices/p...@0,0/pci15d9,1...@5,1:0 sata5/1::dsk/c7t1d0connectedconfigured ok Mod: WDC WD10EACS-00D6B0 FRev: 01.01A01 SN: WD-WCAU40244615 unavailable disk n/devices/p...@0,0/pci15d9,1...@5,1:1 sata6/0emptyunconfigured ok unavailable sata-portn/devices/p...@0,0/pci15d9,1...@5,2:0 sata6/1emptyunconfigured ok unavailable sata-portn/devices/p...@0,0/pci15d9,1...@5,2:1 sata7/0emptyunconfigured ok unavailable sata-portn /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@4:0 sata7/1emptyunconfigured ok unavailable sata-portn /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@4:1 sata7/2::dsk/c5t2d0connectedconfigured ok Mod: WDC WD7500AYYS-01RCA0 FRev: 30.04G30 SN: WD-WCAPT0376631 unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@4:2 sata7/3::dsk/c5t3d0connectedconfigured ok Mod: WDC WD7500AYYS-01RCA0 FRev: 30.04G30 SN: WD-WCAPT0350798 unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@4:3 sata7/4::dsk/c5t4d0connectedconfigured ok Mod: WDC WD7500AYYS-01RCA0 FRev: 30.04G30 SN: WD-WCAPT0403574 unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@4:4 sata7/5::dsk/c5t5d0connectedconfigured ok Mod: WDC WD7500AYYS-01RCA0 FRev: 30.04G30 SN: WD-WCAPT0312592 unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@4:5 sata7/6::dsk/c5t6d0connectedconfigured ok Mod: WDC WD7500AYYS-01RCA0 FRev: 30.04G30 SN: WD-WCAPT0399779 unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@4:6 sata7/7::dsk/c5t7d0connectedconfigured ok Mod: WDC WD7500AYYS-01RCA0 FRev: 30.04G30 SN: WD-WCAPT0441660 unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@4:7 sata8/0::dsk/c6t0d0connectedconfigured ok Mod: WDC WD7500AYYS-01RCA0 FRev: 30.04G30 SN: WD-WCAPT1000344 unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@6:0 sata8/1emptyunconfigured ok unavailable sata-portn /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@6:1 sata8/2emptyunconfigured ok unavailable sata-portn /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@6:2 sata8/3emptyunconfigured ok unavailable sata-portn /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@6:3 sata8/4emptyunconfigured ok unavailable sata-portn /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@6:4 sata8/5emptyunconfigured ok unavailable sata-portn /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@6:5 sata8/6emptyunconfigured ok unavailable sata-portn /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@6:6 sata8/7emptyunconfigured ok unavailable sata-portn /devices/p...@0,0/pci10de,3...@a/pci1033,1...@0/pci11ab,1...@6:7 On Wed, Mar 11, 2009 at 2:40 PM, Blake blake.ir...@gmail.com wrote: I'm attaching a screenshot of the console just before reboot. The dump doesn't seem to be working, or savecore isn't working. On Wed, Mar 11, 2009 at 11:33 AM, Blake blake.ir...@gmail.com wrote: I'm working on testing this some more by doing a savecore -L right after I start the copy. BTW, I'm copying to a raidz2 of only 5 disks, not 16 (the chassis supports 16, but isn't fully populated). So far as I know, there is no spinup happening - these are not RAID controllers, just dumb SATA JBOD controllers, so I don't think they control drive spin in any particular way. Correct me if I'm wrong, of course. On Wed, Mar 11, 2009 at 11:23 AM, Marc Bevand m.bev...@gmail.com wrote: The copy operation will make all the disks
Re: [zfs-discuss] reboot when copying large amounts of data
I think that TMC Research is the company that designed the Supermicro-branded controller card that has the Marvell SATA controller chip on it. Googling around I see connections between Supermicro and TMC. This is the card: http://www.supermicro.com/products/accessories/addon/AOC-SAT2-MV8.cfm On Wed, Mar 11, 2009 at 3:08 PM, Remco Lengers re...@lengers.com wrote: Something is not right in the IO space. The messages talk about vendor ID = 11AB 0x11AB Marvell Semiconductor TMC Research Vendor Id: 0x1030 Short Name: TMC Does fmdump -eV give any clue when the box comes back up? ..Remco Blake wrote: I'm attaching a screenshot of the console just before reboot. The dump doesn't seem to be working, or savecore isn't working. On Wed, Mar 11, 2009 at 11:33 AM, Blake blake.ir...@gmail.com wrote: I'm working on testing this some more by doing a savecore -L right after I start the copy. BTW, I'm copying to a raidz2 of only 5 disks, not 16 (the chassis supports 16, but isn't fully populated). So far as I know, there is no spinup happening - these are not RAID controllers, just dumb SATA JBOD controllers, so I don't think they control drive spin in any particular way. Correct me if I'm wrong, of course. On Wed, Mar 11, 2009 at 11:23 AM, Marc Bevand m.bev...@gmail.com wrote: The copy operation will make all the disks start seeking at the same time and will make your CPU activity jump to a significant percentage to compute the ZFS checksum and RAIDZ parity. I think you could be overloading your PSU because of the sudden increase in power consumption... However if you are *not* using SATA staggered spin-up, then the above theory is unlikely because spinning up consumes much more power than when seeking. So, in a sense, a successful boot proves your PSU is powerful enough. Trying reproducing the problem by copying data on a smaller number of disks. You tried 2 and 16. Try 8. -marc ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] reboot when copying large amounts of data
Could the problem be related to this bug: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6793353 I'm testing setting the maximum payload size as a workaround, as noted in the bug notes. On Wed, Mar 11, 2009 at 3:14 PM, Blake blake.ir...@gmail.com wrote: I think that TMC Research is the company that designed the Supermicro-branded controller card that has the Marvell SATA controller chip on it. Googling around I see connections between Supermicro and TMC. This is the card: http://www.supermicro.com/products/accessories/addon/AOC-SAT2-MV8.cfm On Wed, Mar 11, 2009 at 3:08 PM, Remco Lengers re...@lengers.com wrote: Something is not right in the IO space. The messages talk about vendor ID = 11AB 0x11AB Marvell Semiconductor TMC Research Vendor Id: 0x1030 Short Name: TMC Does fmdump -eV give any clue when the box comes back up? ..Remco Blake wrote: I'm attaching a screenshot of the console just before reboot. The dump doesn't seem to be working, or savecore isn't working. On Wed, Mar 11, 2009 at 11:33 AM, Blake blake.ir...@gmail.com wrote: I'm working on testing this some more by doing a savecore -L right after I start the copy. BTW, I'm copying to a raidz2 of only 5 disks, not 16 (the chassis supports 16, but isn't fully populated). So far as I know, there is no spinup happening - these are not RAID controllers, just dumb SATA JBOD controllers, so I don't think they control drive spin in any particular way. Correct me if I'm wrong, of course. On Wed, Mar 11, 2009 at 11:23 AM, Marc Bevand m.bev...@gmail.com wrote: The copy operation will make all the disks start seeking at the same time and will make your CPU activity jump to a significant percentage to compute the ZFS checksum and RAIDZ parity. I think you could be overloading your PSU because of the sudden increase in power consumption... However if you are *not* using SATA staggered spin-up, then the above theory is unlikely because spinning up consumes much more power than when seeking. So, in a sense, a successful boot proves your PSU is powerful enough. Trying reproducing the problem by copying data on a smaller number of disks. You tried 2 and 16. Try 8. -marc ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] reboot when copying large amounts of data
Any chance this could be the motherboard? I suspect the controller. The boot disks are on the built-in nVidia controller. On Wed, Mar 11, 2009 at 3:41 PM, Remco Lengers re...@lengers.com wrote: - Upgrade FW of controller to highest or known working level I think I have the latest controller firmware. - Upgrade driver or OS level. I'm going to try to go from 101b to 108 or whatever the current dev release is. - Try another controller (may be its broken and barfs under stress ?) In the works. - Analyze the crash dump (if any is saved) Crash dump is not saving properly. - It may be its a know Solaris or driver bug and somebody has heard of it before. Any takers on this? :) hth, Thanks! ..Remco Blake wrote: Could the problem be related to this bug: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6793353 I'm testing setting the maximum payload size as a workaround, as noted in the bug notes. On Wed, Mar 11, 2009 at 3:14 PM, Blake blake.ir...@gmail.com wrote: I think that TMC Research is the company that designed the Supermicro-branded controller card that has the Marvell SATA controller chip on it. Googling around I see connections between Supermicro and TMC. This is the card: http://www.supermicro.com/products/accessories/addon/AOC-SAT2-MV8.cfm On Wed, Mar 11, 2009 at 3:08 PM, Remco Lengers re...@lengers.com wrote: Something is not right in the IO space. The messages talk about vendor ID = 11AB 0x11AB Marvell Semiconductor TMC Research Vendor Id: 0x1030 Short Name: TMC Does fmdump -eV give any clue when the box comes back up? ..Remco Blake wrote: I'm attaching a screenshot of the console just before reboot. The dump doesn't seem to be working, or savecore isn't working. On Wed, Mar 11, 2009 at 11:33 AM, Blake blake.ir...@gmail.com wrote: I'm working on testing this some more by doing a savecore -L right after I start the copy. BTW, I'm copying to a raidz2 of only 5 disks, not 16 (the chassis supports 16, but isn't fully populated). So far as I know, there is no spinup happening - these are not RAID controllers, just dumb SATA JBOD controllers, so I don't think they control drive spin in any particular way. Correct me if I'm wrong, of course. On Wed, Mar 11, 2009 at 11:23 AM, Marc Bevand m.bev...@gmail.com wrote: The copy operation will make all the disks start seeking at the same time and will make your CPU activity jump to a significant percentage to compute the ZFS checksum and RAIDZ parity. I think you could be overloading your PSU because of the sudden increase in power consumption... However if you are *not* using SATA staggered spin-up, then the above theory is unlikely because spinning up consumes much more power than when seeking. So, in a sense, a successful boot proves your PSU is powerful enough. Trying reproducing the problem by copying data on a smaller number of disks. You tried 2 and 16. Try 8. -marc ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] reboot when copying large amounts of data
I guess I didn't make it clear that I had already tried using savecore to retrieve the core from the dump device. I added a larger zvol for dump, to make sure that I wasn't running out of space on the dump device: r...@host:~# dumpadm Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/bigdump (dedicated) Savecore directory: /var/crash/host Savecore enabled: yes I was using the -L option only to try to get some idea of why the system load was climbing to 1 during a simple file copy. On Wed, Mar 11, 2009 at 4:58 PM, Richard Elling richard.ell...@gmail.com wrote: Blake wrote: I'm attaching a screenshot of the console just before reboot. The dump doesn't seem to be working, or savecore isn't working. On Wed, Mar 11, 2009 at 11:33 AM, Blake blake.ir...@gmail.com wrote: I'm working on testing this some more by doing a savecore -L right after I start the copy. savecore -L is not what you want. By default, for OpenSolaris, savecore on boot is disabled. But the core will have been dumped into the dump slice, which is not used for swap. So you should be able to run savecore at a later time to collect the core from the last dump. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] reboot when copying large amounts of data
My dump device is already on a different controller - the motherboards built-in nVidia SATA controller. The raidz2 vdev is the one I'm having trouble with (copying the same files to the mirrored rpool on the nVidia controller work nicely). I do notice that, when using cp to copy the files to the raidz2 pool, load on the machine climbs steadily until the crash, and one proc core pegs at 100%. Frustrating, yes. On Thu, Mar 12, 2009 at 12:31 AM, Maidak Alexander J maidakalexand...@johndeere.com wrote: If you're having issues with a disk contoller or disk IO driver its highly likely that a savecore to disk after the panic will fail. I'm not sure how to work around this, maybe a dedicated dump device not on a controller that uses a different driver then the one that you're having issues with? -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Blake Sent: Wednesday, March 11, 2009 4:45 PM To: Richard Elling Cc: Marc Bevand; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] reboot when copying large amounts of data I guess I didn't make it clear that I had already tried using savecore to retrieve the core from the dump device. I added a larger zvol for dump, to make sure that I wasn't running out of space on the dump device: r...@host:~# dumpadm Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/bigdump (dedicated) Savecore directory: /var/crash/host Savecore enabled: yes I was using the -L option only to try to get some idea of why the system load was climbing to 1 during a simple file copy. On Wed, Mar 11, 2009 at 4:58 PM, Richard Elling richard.ell...@gmail.com wrote: Blake wrote: I'm attaching a screenshot of the console just before reboot. The dump doesn't seem to be working, or savecore isn't working. On Wed, Mar 11, 2009 at 11:33 AM, Blake blake.ir...@gmail.com wrote: I'm working on testing this some more by doing a savecore -L right after I start the copy. savecore -L is not what you want. By default, for OpenSolaris, savecore on boot is disabled. But the core will have been dumped into the dump slice, which is not used for swap. So you should be able to run savecore at a later time to collect the core from the last dump. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Is there a limit to snapshotting?
I think it's filesystems, not snapshots, that take a long time to enumerate. (If I'm wrong, somebody correct me :) On Sun, Mar 8, 2009 at 10:10 PM, mike mike...@gmail.com wrote: I do a daily snapshot of two filesystems, and over the past few months it's obviously grown to a bunch. zfs list shows me all of those. I can change it to use the -t flag to not show them, so that's good. However, I'm worried about boot times and other things. Will it get to a point with 1000's of snapshots that it takes a long time to boot, or do any sort of sync or scrub activities? Thanks :) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] large file copy bug?
I have savecore enabled, but it doesn't look like the machine is dumping core as it should - that is, I don't think it's a panic - I suspect interrupt handling. Speaking of which, does OpenSolaris support Plug'n'Play IRQ assignment? On Thu, Mar 5, 2009 at 3:12 PM, Mark J Musante mmusa...@east.sun.com wrote: On Thu, 5 Mar 2009, Blake wrote: I had a 2008.11 machine crash while moving a 700gb file from one machine to another using cp. I looked for an existing bug for this, but found nothing. Has anyone else seen behavior like this? I wanted to check before filing a bug. Have you got a copy of the stack dump? That would make it easier to track down. Regards, markm ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] large file copy bug?
I have savecore enabled, but nothing in /var/crash: r...@filer:~# savecore -v savecore: dump already processed r...@filer:~# ls /var/crash/filer/ r...@filer:~# On Fri, Mar 6, 2009 at 4:21 PM, Mark J Musante mmusa...@east.sun.com wrote: On Fri, 6 Mar 2009, Blake wrote: I have savecore enabled, but it doesn't look like the machine is dumping core as it should - that is, I don't think it's a panic - I suspect interrupt handling. Then when you say you had a machine crash, what did you mean? Did you look in /var/crash/* to see if there's something there? If not, it's possible your dump zvol just needs to have its core retrieved, which means you can just run 'savecore' at any time and get it. Regards, markm ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] large file copy bug?
These are fair questions, answered inline below :) On Fri, Mar 6, 2009 at 4:45 PM, Mark J Musante mmusa...@east.sun.com wrote: On Fri, 6 Mar 2009, Blake wrote: OK, just to ask the dumb questions: is dumpadm configured for /var/crash/filer? Is the dump zvol big enough? How do you know the whole machine crashed instead of, for example, the X server just coincidentally died? r...@filer:~# dumpadm Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash/filer Savecore enabled: yes I walked over to the console to see the machine rebooting. gdm is disabled on this box. One point of possible interest is that the target fs is on a zpool with 'compression=6' enabled - that is, gzip level 6 compression. The machine has 4gb ram, which means the 2gb dump zvol should be big enough, IIRC. I'm not trying to be difficult, just trying to narrow down where the problem is. If you file a bug that just says 'cp killed my machine', it's likely to be marked incomplete until there's a reliable way of reproducing it. Regards, markm ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] GSoC 09 zfs ideas?
How I do recursive, selective snapshot destroys: http://blog.clockworm.com/2008/03/remove-old-zfs-snapshots.html Saturday, February 28, 2009, 10:14:20 PM, you wrote: TW I would really add : make insane zfs destroy -r| poolname as TW harmless as zpool destroy poolname (recoverable) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] large file copy bug?
I had a 2008.11 machine crash while moving a 700gb file from one machine to another using cp. I looked for an existing bug for this, but found nothing. Has anyone else seen behavior like this? I wanted to check before filing a bug. cheers, Blake ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs related google summer of code ideas - your vote
When I go here: http://opensolaris.org/os/project/isns/bui I get an error. Where are you getting BUI from? On Tue, Mar 3, 2009 at 5:16 PM, Richard Elling richard.ell...@gmail.comwrote: FWIW, I just took at look at the BUI in b108 and it seems to have garnered some love since the last time I looked at it (a year ago?) I encourage folks to take a fresh look at it. https://localhost:6789 -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs related google summer of code ideas - your vote
That's what I thought you meant, and I got excited thinking that you were talking about OpenSolaris :) I'll see about getting the new packages and trying them out. On Tue, Mar 3, 2009 at 8:36 PM, Richard Elling richard.ell...@gmail.comwrote: Blake wrote: When I go here: http://opensolaris.org/os/project/isns/bui I get an error. Â Where are you getting BUI from? The BUI is in webconsole which is available on your local machine at port 6789 https://localhost:6798 If you want to access it remotely, you'll need to change the configuration as documented http://docs.sun.com/app/docs/doc/817-1985/gdhgt?a=view -- richard On Tue, Mar 3, 2009 at 5:16 PM, Richard Elling richard.ell...@gmail.commailto: richard.ell...@gmail.com wrote: FWIW, I just took at look at the BUI in b108 and it seems to have garnered some love since the last time I looked at it (a year ago?) I encourage folks to take a fresh look at it. Â https://localhost:6789 -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS volume corrupted?
It looks like you only have one physical device in this pool. Is that correct? On Mon, Mar 2, 2009 at 9:01 AM, Lars-Gunnar Persson lars-gunnar.pers...@nersc.no wrote: Hey to everyone on this mailing list (since this is my first post)! We've a Sun Fire X4100 M2 server running Solaris 10 u6 and after some system work this weekend we have a problem with only one ZFS volume. We have a pool called /Data with many file systems and two volumes. The status of my zpool is: -bash-3.00$ zpool status pool: Data state: ONLINE scrub: scrub in progress, 5.99% done, 13h38m to go config: NAME STATE READ WRITE CKSUM Data ONLINE 0 0 0 c4t5000402001FC442Cd0 ONLINE 0 0 0 errors: No known data errors Yesterday I started the scrub process because I read that was a smart thing to do after a zpool export and zpool import procedure. I did this because I wanted to move the zpool to another OS installation but changed my mind and did a zpool import on the same OS as I did an export. After checking as much information as I could find on the web, I was advised to to run the zpool scrub after an import. Well, the problem now is that one volume in this zpool is not working. I've shared it via iscsi to a Linux host (all of this was working on Friday). The Linux host reports that it can't find a partition table. Here is the log from the Linux host: Mar 2 11:09:36 eva kernel: SCSI device sdb: 524288000 512-byte hdwr sectors (268435 MB) Mar 2 11:09:36 eva kernel: SCSI device sdb: drive cache: write through Mar 2 11:09:36 eva kernel: SCSI device sdb: 524288000 512-byte hdwr sectors (268435 MB) Mar 2 11:09:37 eva kernel: SCSI device sdb: drive cache: write through Mar 2 11:09:37 eva kernel: sdb: unknown partition table Mar 2 11:09:37 eva kernel: Attached scsi disk sdb at scsi28, channel 0, id 0, lun 0 So I checked the status on my Solaris server and I found this information a bit strange;: -bash-3.00$ zfs list Data/subversion1 NAME USED AVAIL REFER MOUNTPOINT Data/subversion1 22.5K 519G 22.5K - How can it bed 519GB available on a volume that is 250GB in size? Here are more details: -bash-3.00$ zfs get all Data/subversion1 NAME PROPERTY VALUE SOURCE Data/subversion1 type volume - Data/subversion1 creation Wed Apr 2 9:06 2008 - Data/subversion1 used 22.5K - Data/subversion1 available 519G - Data/subversion1 referenced 22.5K - Data/subversion1 compressratio 1.00x - Data/subversion1 reservation 250G local Data/subversion1 volsize 250G - Data/subversion1 volblocksize 8K - Data/subversion1 checksum on default Data/subversion1 compression off default Data/subversion1 readonly off default Data/subversion1 shareiscsi off local Will this be fixed after the scrub process is finished tomorrow or is this volume lost forever? Hoping for some quick answers as the data is quite important for us. Regards, Lars-Gunnar Persson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Virutal zfs server vs hardware zfs server
yes, most nvidia hardware will give you much better performance on OpenSolaris (provided the card is fairly recent) On Mon, Mar 2, 2009 at 6:18 AM, Juergen Nickelsen n...@jnickelsen.de wrote: Juergen Nickelsen n...@jnickelsen.de writes: Solaris Bundled Driver: * vgatext/ ** radeon Video ATI Technologies Inc R360 NJ [Radeon 9800 XT] I *think* this is the same driver used with my work laptop (which I don't have at hand to check, unfortunately), also with ATI graphics hardware. Confirmed. Regards, Juergen. -- What you won was the obligation to pay more for something than anybody else thought it was worth. -- Delainey and Rasmussen's Betty about eBay ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] invalid vdev configuration after power failure
that link suggests that this is a problem with a dirty export: http://www.sun.com/msg/ZFS-8000-EY maybe try importing on system A again, doing a 'zpool export', waiting for completion, then moving to system B to import? On Sun, Mar 1, 2009 at 2:29 PM, Kyle Kakligian small...@gmail.com wrote: What does it mean for a vdev to have an invalid configuration and how can it be fixed or reset? As you can see, the following pool can no longer be imported: (Note that the last accessed by another system warning is because I moved these drives to my test workstation.) ~$ zpool import -f pool0 cannot import 'pool0': invalid vdev configuration ~$ zpool import pool: pool0 id: 5915552147942272438 state: UNAVAIL status: The pool was last accessed by another system. action: The pool cannot be imported due to damaged devices or data. see: http://www.sun.com/msg/ZFS-8000-EY config: pool0 UNAVAIL insufficient replicas raidz1 UNAVAIL corrupted data c5d1p0 ONLINE c4d0p0 ONLINE c4d1p0 ONLINE c6d0p0 ONLINE c5d0p0 ONLINE ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] GSoC 09 zfs ideas?
excellent! i wasn't sure if that was the case, though i had heard rumors. On Mon, Mar 2, 2009 at 12:36 PM, Matthew Ahrens matthew.ahr...@sun.com wrote: Blake wrote: zfs send is great for moving a filesystem with lots of tiny files, since it just handles the blocks :) I'd like to see: pool-shrinking (and an option to shrink disk A when i want disk B to become a mirror, but A is a few blocks bigger) I'm working on it. install to mirror from the liveCD gui zfs recovery tools (sometimes bad things happen) automated installgrub when mirroring an rpool --matt ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Can VirtualBox run a 64 bit guests on 32 bit host
Check out http://www.sun.com/bigadmin/hcl/data/os Sent from my iPhone On Feb 28, 2009, at 2:20 AM, Harry Putnam rea...@newsguy.com wrote: Brian Hechinger wo...@4amlunch.net writes: [...] I think it would be better to answer this question that it would to attempt to answer the VirtualBox question (I run it on a 64-bit OS, so I can't really answer that anyway). Thanks yes and appreciated here The benefit to running ZFS on a 64-bit OS is if you have a large amount of RAM. I don't know what the breaking point is, but I can definitely tell you that a 32-bit kernel and 4GB ram doesn't mix well. If all you are doing is testing ZFS on VMs you probably aren't all that worried about performance so it really shouldn't be an issue for you to run 32-bit. I'd say keep your RAM allocations down, and I wish I knew what to tell you to keep it under. Hopefully someone who has a better grasp of all that can chime in. Once you put it on real hardware, however, you really want a 64-bit CPU and as much RAM as you can toss at the machine. Sounds sensible, thanks for common sense input. Just the little I've tinkered with zfs so far I'm in love already. zfs is much more responive to some kinds of things I'm used to waiting for on linux reiserfs. Commands like du, mv, rm etc on hefty amounts of data are always slow as molasses on linux/reiserfs (and reiserfs is faster than ext3). I have'nt tried ext4 but have been told it is no faster. Whereas zfs gets those jobs done in short order... very noticably faster but I am just going by feel but at least on very similar hardware (cpu wise). (The linux is on Intel 3.06 celeron 2gb ram) I guess there is something called btrfs (nicknamed butter fs) that is supposed to be linux answer to zfs but it isn't ready for primetime yet and I can say it will have a ways to go to compare to zfs. My usage and skill level is probably the lowest on this list easily but even I see some real nice features with zfs. It seams taylor made for semi-ambitious home NAS. So Brian, If you can bear with my windyness a bit more, one of the things flopping around in the back of my mind is something already mentioned here too.. change out the mobo instead of dinking around with addon pci sata controller.. I have 64 bit hardware... but am a bit scared of having lots of trouble getting opensol to run peacefully on it. Its a (somewhat old fashioned now) athlon64 2.2 ghz +3400/Aopen AK86-L mobo. (socket 754) The little jave tool that tests the hardware says my sata controller wont work (the testing tool saw it as a VIA raid controller) and suggests I turn off RAID in the bios. After a carefull look in the bios menus I'm not finding any way to turn it off so guessing the sata ports will be useless unless I install a pci addon sata controller. So thinking of justs changing out the mobo for something with stuff that is known to work. The machine came with an Asus mobo that I ruined myself by dicking aournd installing RAM... somehow shorted out something, then mobo became useless. But I'm thinking of turning to Asus again and making sure there is onboard SATA with at least 4 prts and preferebly 6. So cutting to the chase here... would you happen to have a recommendation from your own experience, or something you've heard will work and that can stand more ram... my current setup tops out at 3gb. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] GSoC 09 zfs ideas?
Shrinking pools would also solve the right-sizing dilemma. Sent from my iPhone On Feb 28, 2009, at 3:37 AM, Joe Esposito j...@j-espo.com wrote: I'm using opensolaris and zfs at my house for my photography storage as well as for an offsite backup location for my employer and several side web projects. I have an 80g drive as my root drive. I recently took posesion of 2 74g 10k drives which I'd love to add as a mirror to replace the 80 g drive. From what I gather it is only possible if I zfs export my storage array and reinstall solaris on the new disks. So I guess I'm hoping zfs shrink and grow commands show up sooner or later. Just a data point. Joe Esposito www.j-espo.com On 2/28/09, C. Bergström cbergst...@netsyncro.com wrote: Blake wrote: Gnome GUI for desktop ZFS administration On Fri, Feb 27, 2009 at 9:13 PM, Blake blake.ir...@gmail.com wrote: zfs send is great for moving a filesystem with lots of tiny files, since it just handles the blocks :) I'd like to see: pool-shrinking (and an option to shrink disk A when i want disk B to become a mirror, but A is a few blocks bigger) This may be interesting... I'm not sure how often you need to shrink a pool though? Could this be classified more as a Home or SME level feature? install to mirror from the liveCD gui I'm not working on OpenSolaris at all, but for when my projects installer is more ready /we/ can certainly do this.. zfs recovery tools (sometimes bad things happen) Agreed.. part of what I think keeps zfs so stable though is the complete lack of dependence on any recovery tools.. It forces customers to bring up the issue instead of dirty hack and nobody knows. automated installgrub when mirroring an rpool This goes back to an installer option? ./C ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Virutal zfs server vs hardware zfs server
On Fri, Feb 27, 2009 at 12:31 PM, Harry Putnam rea...@newsguy.com wrote: Are you talking about the official Opensol-11 install iso or something else? The official 2008.11 LiveCD has the tool on the default desktop as an icon. A big issue with running a VM is that ZFS prefers direct access to storage. What effect does this preferance have? Does it perform badly when it does not have direct access to storage? Are the virtual disks supplied by vmware less functional in some way? I would expect pretty bad performance adding VMWare as a layer in between ZFS and your block devices. ZFS documentation specifically advises against abstracting block devices whenever possible. Since ZFS is trying to checksum blocks, the fewer abstraction layers you have in between ZFS and spinning rust, the less points of error/failure. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Virutal zfs server vs hardware zfs server
I meant that the more layers you remove, the less layers there are that can tell ZFS something that's not true. I guess ZFS would still catch those errors in most cases - it would still be a pain to deal with needless errors. Also I like to do what the manual says, and the manual says avoid abstraction layers :) Harry, Richard is probably right. There are plenty of boards with nVidia or Intel SATA that should work fine. Search for 'opensolaris hcl' (hardware compatibility list) - there are about 400+ mobos listed there that are reported to work. On Fri, Feb 27, 2009 at 1:48 PM, Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Fri, 27 Feb 2009, Blake wrote: SinceZFS is trying to checksum blocks, the fewer abstraction layers youhave in between ZFS and spinning rust, the less points oferror/failure. Are you saying that ZFS checksums are responsible for the failure? In what way does more layers of abstraction cause particular problems for ZFS which won't also occur with some other filesystem? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] GSoC 09 zfs ideas?
zfs send is great for moving a filesystem with lots of tiny files, since it just handles the blocks :) I'd like to see: pool-shrinking (and an option to shrink disk A when i want disk B to become a mirror, but A is a few blocks bigger) install to mirror from the liveCD gui zfs recovery tools (sometimes bad things happen) automated installgrub when mirroring an rpool On Fri, Feb 27, 2009 at 8:02 PM, Richard Elling richard.ell...@gmail.com wrote: David Magda wrote: On Feb 27, 2009, at 18:23, C. Bergström wrote: Blake wrote: Care to share any of those in advance? It might be cool to see input from listees and generally get some wheels turning... raidz boot support in grub 2 is pretty high on my list to be honest.. Which brings up another question of where is the raidz stuff mostly? usr/src/uts/common/fs/zfs/vdev_raidz.c ? Any high level summary, docs or blog entries of what the process would look like for a raidz boot support is also appreciated. Given the threads that have appeared on this list lately, how about codifying / standardizing the output of zfs send so that it can be backed up to tape? :) It wouldn't help. zfs send is a data stream which contains parts of files, not files (in the usual sense), so there is no real way to take a send stream and extract a file, other than by doing a receive. At the risk of repeating the Best Practices Guide (again): The zfs send and receive commands do not provide an enterprise-level backup solution. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Virutal zfs server vs hardware zfs server
Brandon makes a good point. I think that's an option to pursue if you don't want to risk messing up your Windows install. If you can, dedicate entire disks, rather that partitions, to ZFS. It's easier to manage. ZFS is managed by the VMs processor in this case, so you will take a bigger performance hit than running on bare metal. That said, my filer exporting ZFS over NFS to 10 busy CentOS clients barely breaks a sweat. On Fri, Feb 27, 2009 at 7:51 PM, Harry Putnam rea...@newsguy.com wrote: Brandon High bh...@freaks.com writes: On Thu, Feb 26, 2009 at 8:35 AM, Blake blake.ir...@gmail.com wrote: A big issue with running a VM is that ZFS prefers direct access to storage. VMWare can give VMs direct access to the actual disks. This should avoid the overhead of using virtual disks. Can you say if it makes a noticeable difference to zfs. I'd noticed that option but didn't connect it to this conversation. Also, if I recall there is some warning about being an advanced user to use that option or something similar. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] GSoC 09 zfs ideas?
Gnome GUI for desktop ZFS administration On Fri, Feb 27, 2009 at 9:13 PM, Blake blake.ir...@gmail.com wrote: zfs send is great for moving a filesystem with lots of tiny files, since it just handles the blocks :) I'd like to see: pool-shrinking (and an option to shrink disk A when i want disk B to become a mirror, but A is a few blocks bigger) install to mirror from the liveCD gui zfs recovery tools (sometimes bad things happen) automated installgrub when mirroring an rpool On Fri, Feb 27, 2009 at 8:02 PM, Richard Elling richard.ell...@gmail.com wrote: David Magda wrote: On Feb 27, 2009, at 18:23, C. Bergström wrote: Blake wrote: Care to share any of those in advance? It might be cool to see input from listees and generally get some wheels turning... raidz boot support in grub 2 is pretty high on my list to be honest.. Which brings up another question of where is the raidz stuff mostly? usr/src/uts/common/fs/zfs/vdev_raidz.c ? Any high level summary, docs or blog entries of what the process would look like for a raidz boot support is also appreciated. Given the threads that have appeared on this list lately, how about codifying / standardizing the output of zfs send so that it can be backed up to tape? :) It wouldn't help. zfs send is a data stream which contains parts of files, not files (in the usual sense), so there is no real way to take a send stream and extract a file, other than by doing a receive. At the risk of repeating the Best Practices Guide (again): The zfs send and receive commands do not provide an enterprise-level backup solution. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Can VirtualBox run a 64 bit guests on 32 bit host
The changelog says 64-bit guest on 32-bit host support was added in 2.1: http://www.virtualbox.org/wiki/Changelog On Thu, Feb 26, 2009 at 10:48 AM, Brian Hechinger wo...@4amlunch.net wrote: On Wed, Feb 25, 2009 at 07:14:14PM -0600, Harry Putnam wrote: My whole purpose is to experiment with zfs... would I see much difference if opensol was installed 64 bit as compared to 32 bit? I noticed the Simon blogs that describe how to setup a home zfs server ( http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ ) mention it is best setup 64 bit, but no real reason is given. I think it would be better to answer this question that it would to attempt to answer the VirtualBox question (I run it on a 64-bit OS, so I can't really answer that anyway). The benefit to running ZFS on a 64-bit OS is if you have a large amount of RAM. I don't know what the breaking point is, but I can definitely tell you that a 32-bit kernel and 4GB ram doesn't mix well. If all you are doing is testing ZFS on VMs you probably aren't all that worried about performance so it really shouldn't be an issue for you to run 32-bit. I'd say keep your RAM allocations down, and I wish I knew what to tell you to keep it under. Hopefully someone who has a better grasp of all that can chime in. Once you put it on real hardware, however, you really want a 64-bit CPU and as much RAM as you can toss at the machine. -brian -- Coding in C is like sending a 3 year old to do groceries. You gotta tell them exactly what you want or you'll end up with a cupboard full of pop tarts and pancake mix. -- IRC User (http://www.bash.org/?841435) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Migrating to ZFS
Rafael, If you are talking just about moving a bunch of data, take a look at rsync. I think it will work nicely for moving files from one volume to another, preserving attributes. It comes bundled with 2008.11 and up. On Thu, Feb 26, 2009 at 10:41 AM, cindy.swearin...@sun.com wrote: Hi Rafael, The information on that site looks very out-of-date. I will attempt to resolve this problem. Other than using Live Upgrade to migrate a UFS root file system to a ZFS root file system, you can use ufsdump and ufsrestore to migrate UFS data to ZFS file system. Other data migration processes are mostly manual. If something else is in the works, we'll let you know. Cindy Rafael Friedlander wrote: Hi, According to http://www.sun.com/emrkt/campaign_docs/expertexchange/knowledge/solaris_zfs_install.html#2 we will have a tool for data migration to ZFS (not a root file system). Do you know when this is happening? Thanks, Rafael. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Virutal zfs server vs hardware zfs server
Harry, The LiveCD for OpenSolaris has a driver detection tool on it - this will let you see if your hardware is supported without touching the installed XP system. A big issue with running a VM is that ZFS prefers direct access to storage. On Wed, Feb 25, 2009 at 11:48 PM, Harry Putnam rea...@newsguy.com wrote: I'm experimenting with a zfs home server. Running Opensol-11 by way of vmware on WinXP. It seems one way to avoid all the hardware problems one might run into trying to install opensol on available or spare hardware. Are there some bad gotchas about running opensol/zfs through vmware and never going to real hardware? One thing comes to mind is the overhead of two OSs on one processor. An Athlon64 2.2 +3400 running 32bit Windows XP and opensol in VMware. But if I lay off the windows OS... like not really working it with transcibing video or compressing masses of data or the like. Is this likely to be a problem? Also I'm loosing out on going 64 bit since its not likely this machine supports the AMD V extensions... and I'm short on SATA connections. I only have two onboard, but plan to install a pci style sata controller to squeeze in some more discs. Its a big old ANTEC case so I don't think getting the discs in there will be much of a problem. But have wondered if a PCI sata controller is likely to be a big problem. So, are there things I need to know about that will make running a zfs home server from vmware a bad idea? The server will be serving as backup destination for 5 home machines and most likely would see service only about 2-3 days a week far as any kind of heavy usage like ghosted disc images and other large chunks of data + a regular 3 day a week backup running from windows using `retrospect' to backup user directories and changed files in C:\. A 6th (linux) machine may eventually start using the server but for now its pretty selfcontained and has lots of disc space. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] GSoC 09 zfs ideas?
Care to share any of those in advance? It might be cool to see input from listees and generally get some wheels turning... On Wed, Feb 25, 2009 at 4:39 AM, C. Bergström cbergst...@netsyncro.com wrote: Hi everyone. I've got a couple ideas for good zfs GSoC projects, but wanted to stir some interest. Anyone interested to help mentor? The deadline is around the corner so if planning hasn't happened yet it should start soon. If there is interest who would the org administrator be? Thanks ./Christopher ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Motherboard for home zfs/solaris file server -- ECC claims
IIRC, the AMD board I have at my office has hardware ECC scrub. I have no idea if Solaris knows about this or makes any use of it (or needs to?) On Tue, Feb 24, 2009 at 2:50 PM, Miles Nordin car...@ivy.net wrote: rl == Rob Logan r...@logan.com writes: rl that's why this X58 MB claims ECC support: the claim is worth something. People always say ``AMD supports ECC because the memory controller is in the CPU so they all support it, it cannot be taken away from you by lying idiot motherboard manufacturers or greedy marketers trying to segment users into different demand groups'' but you still need some motherboard BIOS to flip the ECC switch to ``wings stay on'' mode before you start down the runway. Here is a rather outdated and Linux-specific workaround for cheapo AMD desktop boards that don't have an ECC option in their BIOS: http://newsgroups.derkeiler.com/Archive/Alt/alt.comp.periphs.mainboard.asus/2005-10/msg00365.html http://hyvatti.iki.fi/~jaakko/sw/ The discussion about ECC-only vs scrub-and-fix, about how to read from PCI if ECC errors are happening (though not necessarily which stick), and his 10-ohm testing method, is also interesting. I still don't understand what chip-kill means. I remember something about a memory scrubbing kernel thread in Solaris. This sounds like the AMD chips have a hardware scrubber? Also how are ECC errors reported in Solaris? I guess this is getting OT though. Anyway ECC is not just a feature bullet to gather up and feel good. You have to finish the job and actually interact with it. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Migrating to ZFS
Ah - I think I was getting confused by my experience with the modified rsync on OS X. On Thu, Feb 26, 2009 at 1:54 PM, Ian Collins i...@ianshome.com wrote: Blake wrote: Rafael, If you are talking just about moving a bunch of data, take a look at rsync. I think it will work nicely for moving files from one volume to another, preserving attributes. It comes bundled with 2008.11 and up. But not ACLs. -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Backing up ZFS snapshots
I'm sure that's true. My point was that, given the choice between a zfs send/recv from one set of devices to another, where the target is another pool, and sending a zfs stream to a tarball, I'd sooner choose a solution that's all live filesystems. If backups are *really* important, then it's certainly better to use a product with commercial support. I think Amanda is zfs-aware now? On Mon, Feb 23, 2009 at 12:16 PM, Miles Nordin car...@ivy.net wrote: b == Blake blake.ir...@gmail.com writes: c There are other problems besides the versioning. b Agreed - I don't think that archiving simply the send stream b is a smart idea (yet, until the stream format is stabilized *there* *are* *other* *problems* *besides* *the* *versioning*! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about zfs recv -d, apparently
I'm actually working on this for an application at my org. I'll try to post my work somewhere when done (hopefully this week). Are you keeping in mind the fact that the '-i' option needs a pair of snapshots (original and current) to work properly? On Sun, Feb 22, 2009 at 2:14 PM, David Dyer-Bennet d...@dd-b.net wrote: On Sun, February 22, 2009 00:15, David Dyer-Bennet wrote: First, it fails because the destination directory doesn't exist. Then it fails because it DOES exist. I really expected one of those to work. So, what am I confused about now? (Running 2008.11) # zpool import -R /backups/bup-ruin bup-ruin # zfs send -R z...@bup-20090222-054457utc | zfs receive -dv bup-ruin/fsfs/zp1 cannot receive: specified fs (bup-ruin/fsfs/zp1) does not exist # zfs create bup-ruin/fsfs/zp1 # zfs send -R z...@bup-20090222-054457utc | zfs receive -dv bup-ruin/fsfs/zp1 cannot receive new filesystem stream: destination 'bup-ruin/fsfs/zp1' exists must specify -F to overwrite it I've tried some more things. Isn't this the same operations as above, but with different (and more reasonable) results? But isn't this the same, with different results? # zfs list -r bup-ruin NAME USED AVAIL REFER MOUNTPOINT bup-ruin 79.5K 913G18K /backups/bup-ruin # zfs create bup-ruin/fsfs # zfs send -R rpool/export/h...@bup-2009\ 0216-044512UTC | zfs recv -d bup-ruin/fsfs/rpool cannot receive: specified fs (bup-ruin/fsfs/rpool) does not exist r...@fsfs:/export/home/localddb/src/bup2# zfs create bup-ruin/fsfs/rpool r...@fsfs:/export/home/localddb/src/bup2# zfs send -R rpool/export/h...@bup-2009\ 0216-044512UTC | zfs recv -d bup-ruin/fsfs/rpool These second results are what I expected, after reading the error messages and the manual. But the first example is what I actually got, originally. (Different pools, slightly). Here's what I'm trying to do: I'm trying to store backups of multiple pools (which are on disks mounted in the desktop chassis) on external pools consisting of a single external USB drive. My concept is to do a send -R for the initial backup of each, and then to do send -i with suitable params for the later backups. This should keep the external filesystems in synch with the internal filesystems up to the snapshot most recently synced. But I can't find commands to make this work. (Note that I need to back up two pools, rpool and zp1, from the destkop on the the single external pool bup-ruin. I'm importing bup-ruin with altroot to avoid the mountoints of the backed-up filesystems on it conflicting with each other or with stuff already mounted on the desktop.) -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Backing up ZFS snapshots
I thinks that's legitimate so long as you don't change ZFS versions. Personally, I'm more comfortable doing a 'zfs send | zfs recv' than I am storing the send stream itself. The problem I have with the stream is that I may not be able to receive it in a future version of ZFS, while I'm pretty sure that I can upgrade an actual pool/fs pretty easily. On Sun, Feb 22, 2009 at 4:48 PM, David Abrahams d...@boostpro.com wrote: on Wed Feb 18 2009, Frank Cusack fcusack-AT-fcusack.com wrote: On February 17, 2009 3:57:34 PM -0800 Joe S js.li...@gmail.com wrote: On Tue, Feb 17, 2009 at 3:35 PM, David Magda dma...@ee.ryerson.ca wrote: If you want to do back ups of your file system use a documented utility (tar, cpio, pax, zip, etc.). I'm going to try to use Amanda and backup my data (not snapshots). You missed the point, which is not to avoid snapshots, but to avoid saving the stream as a backup. Backing up a snapshot is typically preferred to backing up a live filesystem. Has anyone here noticed that http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide suggests in several places that zfs send streams be stored for backup? -- Dave Abrahams BoostPro Computing http://www.boostpro.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Backing up ZFS snapshots
Agreed - I don't think that archiving simply the send stream is a smart idea (yet, until the stream format is stabilized in some way). I'd much rather archive to a normal ZFS filesystem. With ZFS's enormous pool capacities, it's probably the closest thing we have right now to a future-proof filesystem. On Sun, Feb 22, 2009 at 6:58 PM, Miles Nordin car...@ivy.net wrote: well fine, but there's certainly not a consensus on that, which makes it not a ``best practice.'' There are other problems besides the versioning. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Oracle doc: Unable to run Forms applications when using Solaris Zettabyte File System (ZFS)
If this happens if ZFS is in use anywhere in the system, I'm not sure of a solution. If you just need Oracle files and activity to be on something other than ZFS, could you try creating a ZFS block device and formatting it UFS? (disclaimer: I'm not an Oracle user) On Fri, Feb 20, 2009 at 12:12 PM, Jose Gregores jose.grego...@sun.com wrote: Hi all, We have a Suncluster/ZFS customer complaining about this issue. Subject: Unable to run Forms applications when using Solaris Zettabyte File System (ZFS) Doc ID: 730691.1 Type: PROBLEM Modified Date: 24-SEP-2008 Status: PUBLISHED In this Document Symptoms Changes Cause Solution References Applies to: Oracle Application Server 10g Enterprise Edition - Version: 1.0.2.0.0 to 10.1.3.0.0 Oracle Forms - Version: 6.0 to 10.1.2 Solaris Operating System (SPARC 64-bit) Solaris Operating System (SPARC 32-bit) Symptoms A common symptom will include Oracle Forms applications intermittently or consistently failing at startup or while in use. The error displayed on the client will be FRM-92101. Changes This problem can be exposed when enabling Solaris Zettabyte File System (ZFS). Cause Oracle Fusion Middleware and all of it subcomponents (i.e. Forms, Reports, etc) have not been certified to be used in a Solaris ZFS environment or any other specific file systems. This includes all Application Server versions 1.0 - 10.1.3. Therefore, Oracle technical support will be limited for installations configured in this type of environment. Usually, Oracle Fusion Middleware software can be installed on file systems provided by the OS vendor. If there is a compatibility issue specific to an OS version and/or file system that is being used, contact the OS vendors directly. Internal Bug 7308848 - CLARIFICATION ON AS CERTIFICATION USING ZFS Solution Because Solaris ZFS has not been certified and technical support will be limited for this configuration, it is strongly recommended that ZFS not be used without extensive testing prior to moving to production. Problems reproducible only when using a specific file system type should be directed toward the OS vendor and not Oracle. It is not expected that ZFS will be supported for use with Application Server 10.1.2 or 10.1.3 before these product versions reach their desupport dates. Information about Fusion Middleware support and support dates can be found in a brochure titled, Lifetime Support Policy: Oracle Technology Products found on the Oracle web site. http://www.oracle.com/support/library/brochure/lifetime-support-technology.pdf Please refer to the Application Server Certification documentation for the latest updates regarding this issue. http://www.oracle.com/technology/software/products/ias/files/as_certification_r2_101202.html References Do we have a solution or suggestion for this problem ? Thanks. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Oracle doc: Unable to run Forms applications when using Solaris Zettabyte File System (ZFS)
It looks like this is a bigger Oracle environment than any database environment that I've ever worked in :) Anyway - when you create a zfs filesystem, you can do it with the '-V' option, which makes a block device of a fixed size that's accessible through the usual /dev device path. You can then put a UFS filesystem on this ZFS-backed block device. See 'man zfs' for details. On Fri, Feb 20, 2009 at 1:43 PM, Jose Gregores jose.grego...@sun.com wrote: Blake escreveu: If this happens if ZFS is in use anywhere in the system, I'm not sure of a solution. Yes, this happens on our customer that has a 3 node Suncluster with 150 zpools and 50 zones and 15 Oracle databases. If you just need Oracle files and activity to be on something other than ZFS, could you try creating a ZFS block device and formatting it UFS? How can we do that ? (disclaimer: I'm not an Oracle user) On Fri, Feb 20, 2009 at 12:12 PM, Jose Gregores jose.grego...@sun.com wrote: Hi all, We have a Suncluster/ZFS customer complaining about this issue. Subject: Unable to run Forms applications when using Solaris Zettabyte File System (ZFS) Doc ID: 730691.1 Type: PROBLEM Modified Date: 24-SEP-2008 Status: PUBLISHED In this Document Symptoms Changes Cause Solution References Applies to: Oracle Application Server 10g Enterprise Edition - Version: 1.0.2.0.0 to 10.1.3.0.0 Oracle Forms - Version: 6.0 to 10.1.2 Solaris Operating System (SPARC 64-bit) Solaris Operating System (SPARC 32-bit) Symptoms A common symptom will include Oracle Forms applications intermittently or consistently failing at startup or while in use. The error displayed on the client will be FRM-92101. Changes This problem can be exposed when enabling Solaris Zettabyte File System (ZFS). Cause Oracle Fusion Middleware and all of it subcomponents (i.e. Forms, Reports, etc) have not been certified to be used in a Solaris ZFS environment or any other specific file systems. This includes all Application Server versions 1.0 - 10.1.3. Therefore, Oracle technical support will be limited for installations configured in this type of environment. Usually, Oracle Fusion Middleware software can be installed on file systems provided by the OS vendor. If there is a compatibility issue specific to an OS version and/or file system that is being used, contact the OS vendors directly. Internal Bug 7308848 - CLARIFICATION ON AS CERTIFICATION USING ZFS Solution Because Solaris ZFS has not been certified and technical support will be limited for this configuration, it is strongly recommended that ZFS not be used without extensive testing prior to moving to production. Problems reproducible only when using a specific file system type should be directed toward the OS vendor and not Oracle. It is not expected that ZFS will be supported for use with Application Server 10.1.2 or 10.1.3 before these product versions reach their desupport dates. Information about Fusion Middleware support and support dates can be found in a brochure titled, Lifetime Support Policy: Oracle Technology Products found on the Oracle web site. http://www.oracle.com/support/library/brochure/lifetime-support-technology.pdf Please refer to the Application Server Certification documentation for the latest updates regarding this issue. http://www.oracle.com/technology/software/products/ias/files/as_certification_r2_101202.html References Do we have a solution or suggestion for this problem ? Thanks. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about prerequisites for ZFS to work
You definitely need SUNWsmbskr - the cifs server provided with OpenSolaris is tied to the kernel at some low level. I found this entry helpful: http://blogs.sun.com/timthomas/entry/solaris_cifs_in_workgroup_mode On Wed, Feb 18, 2009 at 1:03 PM, Harry Putnam rea...@newsguy.com wrote: Ian Collins wrote: Harry Putnam wrote: [...] Still when I look again... its still in maintenance mode. What does tail /var/svc/log/network-smb-server:default.log show? The log file for a service listed as part of the long listing (svcs -l smb/server). Following these two commands: svcadm disable sbm/server svcadm enable -r smb/server [...] [ Feb 18 11:53:42 Executing start method (/usr/lib/smbsrv/smbd start). ] smbd: NetBIOS services started smbd: kernel bind error: No such file or directory smbd: daemon initialization failed [ Feb 18 11:53:43 Method start exited with status 95. ] I wondered about the kernel in previous post: I noticed a special kernel package right next to the smb server called. SUNWsmbskr (smb/server kernel). So does this require a special kernel? Does that mean I need to do something to the kernel..? Replace it with SUNWsmbskr? Or is it something less ambitious? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zpool scrub in cron hangs u3/u4 server, stumps tech support.
Bob is correct to praise LiveUpgrade. It's pretty much risk-free when
used properly, provided you have some spare slices/disks.
At the same time, I'd say that this is probably an appropriate time to
escalate the bug with support - the answers you are getting aren't
satisfactory.
I would also consider creating a user/role with zfs admin privileges
only, and trying to run the scrub command from cron as this user - I
had a similar problem with an old ZFS version which I worked around by
issuing commands as a user other than root.
On Wed, Feb 18, 2009 at 12:44 PM, Bob Friesenhahn
bfrie...@simple.dallas.tx.us wrote:
On Wed, 18 Feb 2009, Elizabeth Schwartz wrote:
It's an old version but it's a *supported* version and we have a
five-figure support contract. That used to matter.
I can understand your frustration. ZFS in Solaris 10U3 was a bit rough
around the edges. It is definitely improved in later releases.
I've never used Live Upgrade; I want to try it out but not on my
production file server, and I want to know that this particular bug is
fixed first, something more definite than many improvements
As long as you have spare bootable partitions, Live Upgrade is exceedingly
useful. It allows you to create a new boot environment with the newer
Solaris installed, and with all of your local changes applied. You can
double-check to make sure that everything is ready to go via a mount to the
new boot evironment. Switching to the new boot environment is as simple as
'luactivate' followed by a reboot. It is likely to work first time, but if
it does not, you can reboot to your previous boot environment for minimal
server down time. If you are using Grub, then each boot environment is
listed in the Grub boot menu.
With proper care, using Live Upgrade is safer (and faster) for production
systems than applying large numbers of patches spanning many Solaris 10
generations. You can also use multiple boot environments to apply patches,
in order to minimize risk and minimize down time.
If you are able to install Solaris 10U6 with ZFS boot, then subsequent Live
Upgrades should be far easier since boot evironments are directories in the
root pool ('rpool') rather than in dedicated partitions.
Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about prerequisites for ZFS to work
have you made sure that samba is *disabled*? svcs samba ? On Wed, Feb 18, 2009 at 4:14 PM, Harry Putnam rea...@newsguy.com wrote: Blake wrote: You definitely need SUNWsmbskr - the cifs server provided with OpenSolaris is tied to the kernel at some low level. I found this entry helpful: http://blogs.sun.com/timthomas/entry/solaris_cifs_in_workgroup_mode Looks like it will be immensely so.. However it appears from the comments there that I need to have the smb server online before the directions there are very usefull. I've installed both SUNWsmbs and SUNWsmbskr and rebooted as directed. pkg shows them installed. pkg list|grep smbs SUNWsmbs 0.5.11-0.101installed SUNWsmbskr0.5.11-0.101installed However I still get the same error when trying to start smb server. [ Feb 18 14:36:36 Enabled. ] [ Feb 18 14:36:36 Executing start method (/usr/lib/smbsrv/smbd start). ] smbd: NetBIOS services started smbd: kernel bind error: No such file or directory smbd: daemon initialization failed [ Feb 18 14:36:37 Method start exited with status 95. ] I'm not finding many clues with google as to what to do, but as always its not doubt that my search strings are lacking, since I don't have enough experience yet to know what to look for exactly. Something like this: how to install SUNWsmbskr Turns up man messges about the need to reboot after installing but not any I see about how to deal with any problems with the install itself. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] scrub on snv-b107
Do you have more data on the 107 pool than on the sol10 pool? On Tue, Feb 17, 2009 at 6:11 AM, dick hoogendijk d...@nagual.nl wrote: scrub completed after 1h9m with 0 errors on Tue Feb 17 12:09:31 2009 This is about twice as slow as the same srub on a solaris 10 box with a mirrored zfs root pool. Has scrub become that much slower? And if so, why? -- Dick Hoogendijk -- PGP/GnuPG key: 01D2433D + http://nagual.nl/ | SunOS sxce snv107 ++ + All that's really worth doing is what we do for others (Lewis Carrol) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs destroy hanging
I think you can kill the destroy command process using traditional methods. Perhaps your slowness issue is because the pool is an older format. I've not had these problems since upgrading to the zfs version that comes default with 2008.11 On Fri, Feb 13, 2009 at 4:14 PM, David Dyer-Bennet d...@dd-b.net wrote: This shouldn't be taking anywhere *near* half an hour. The snapshots differ trivially, by one or two files and less than 10k of data (they're test results from working on my backup script). But so far, it's still sitting there after more than half an hour. local...@fsfs:~/src/bup2# zfs destroy ruin/export cannot destroy 'ruin/export': filesystem has children use '-r' to destroy the following datasets: ruin/export/h...@bup-20090210-202557utc ruin/export/h...@20090210-213902utc ruin/export/home/local...@first ruin/export/home/local...@second ruin/export/home/local...@bup-20090210-202557utc ruin/export/home/local...@20090210-213902utc ruin/export/home/localddb ruin/export/home local...@fsfs:~/src/bup2# zfs destroy -r ruin/export It's still hung. Ah, here's zfs list output from shortly before I started the destroy: ruin 474G 440G 431G /backups/ruin ruin/export 35.0M 440G18K /backups/ruin/export ruin/export/home35.0M 440G19K /export/home ruin/export/home/localddb 35M 440G 27.8M /export/home/localddb As you can see, the ruin/export/home filesystem (and subs) is NOT large. iostat shows no activity on pool ruin over a minute. local...@fsfs:~$ pfexec zpool iostat ruin 10 capacity operationsbandwidth pool used avail read write read write -- - - - - - - ruin 474G 454G 10 0 1.13M840 ruin 474G 454G 0 0 0 0 ruin 474G 454G 0 0 0 0 ruin 474G 454G 0 0 0 0 ruin 474G 454G 0 0 0 0 ruin 474G 454G 0 0 0 0 ruin 474G 454G 0 0 0 0 ruin 474G 454G 0 0 0 0 ruin 474G 454G 0 0 0 0 The pool still thinks it is healthy. local...@fsfs:~$ zpool status -v ruin pool: ruin state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using 'zpool upgrade'. Once this is done, the pool will no longer be accessible on older software versions. scrub: scrub completed after 4h42m with 0 errors on Mon Feb 9 19:10:49 2009 config: NAMESTATE READ WRITE CKSUM ruinONLINE 0 0 0 c7t0d0ONLINE 0 0 0 errors: No known data errors There is still a process out there trying to run that destroy. It doesn't appear to be using much cpu time. local...@fsfs:~$ ps -ef | grep zfs localddb 7291 7228 0 15:10:56 pts/4 0:00 grep zfs root 7223 7101 0 14:18:27 pts/3 0:00 zfs destroy -r ruin/export Running 2008.11. local...@fsfs:~$ uname -a SunOS fsfs 5.11 snv_101b i86pc i386 i86pc Solaris Any suggestions? Eventually I'll kill the process by the gentlest way that works, I suppose (if it doesn't complete). -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS: unreliable for professional usage?
That does look like the issue being discussed. It's a little alarming that the bug was reported against snv54 and is still not fixed :( Does anyone know how to push for resolution on this? USB is pretty common, like it or not for storage purposes - especially amongst the laptop-using dev crowd that OpenSolaris apparently targets. On Thu, Feb 12, 2009 at 4:44 PM, bdebel...@intelesyscorp.com bdebel...@intelesyscorp.com wrote: Is this the crux of the problem? http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6424510 'For usb devices, the driver currently ignores DKIOCFLUSHWRITECACHE. This can cause catastrophic data corruption in the event of power loss, even for filesystems like ZFS that are designed to survive it. Dropping a flush-cache command is just as bad as dropping a write. It violates the interface that software relies on to use the device.' -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] strange 'too many errors' msg
I think you could try clearing the pool - however, consulting the fault management tools (fmdump and it's kin) might be smart first. It's possible this is an error in the controller. The output of 'cfgadm' might be of use also. On Wed, Feb 11, 2009 at 7:12 PM, Jens Elkner jel+...@cs.uni-magdeburg.de wrote: Hi, just found on a X4500 with S10u6: fmd: [ID 441519 daemon.error] SUNW-MSG-ID: ZFS-8000-GH, TYPE: Fault, VER: 1, SEVERITY: Major EVENT-TIME: Wed Feb 11 16:03:26 CET 2009 PLATFORM: Sun Fire X4500, CSN: 00:14:4F:20:E0:2C , HOSTNAME: peng SOURCE: zfs-diagnosis, REV: 1.0 EVENT-ID: 74e6f0ec-b1e7-e49b-8d71-dc1c9b68ad2b DESC: The number of checksum errors associated with a ZFS device exceeded acceptable levels. Refer to http://sun.com/msg/ZFS-8000-GH for more information. AUTO-RESPONSE: The device has been marked as degraded. An attempt will be made to activate a hot spare if available. IMPACT: Fault tolerance of the pool may be compromised. REC-ACTION: Run 'zpool status -x' and replace the bad device. zpool status -x ... mirror DEGRADED 0 0 0 spare DEGRADED 0 0 0 c6t6d0 DEGRADED 0 0 0 too many errors c4t0d0 ONLINE 0 0 0 c7t6d0ONLINE 0 0 0 ... spares c4t0d0 INUSE currently in use c4t4d0 AVAIL Strange thing is, that for more than 3 month there was no single error logged with any drive. IIRC, before u4 I've seen occasionaly a bad checksum error message, but this was obviously the result from the wellknown race condition of the marvell driver when havy writes took place. So I tend to interprete it as an false alarm and think about 'zpool ... clear c6t6d0'. What do you think. Is this a good idea? Regards, jel. BTW: zpool status -x msg refers to http://www.sun.com/msg/ZFS-8000-9P, the event to http://sun.com/msg/ZFS-8000-GH - little bit inconsistent I think. -- Otto-von-Guericke University http://www.cs.uni-magdeburg.de/ Department of Computer Science Geb. 29 R 027, Universitaetsplatz 2 39106 Magdeburg, Germany Tel: +49 391 67 12768 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS: unreliable for professional usage?
I'm sure it's very hard to write good error handling code for hardware events like this. I think, after skimming this thread (a pretty wild ride), we can at least decide that there is an RFE for a recovery tool for zfs - something to allow us to try to pull data from a failed pool. That seems like a reasonable tool to request/work on, no? On Thu, Feb 12, 2009 at 6:03 PM, Toby Thain t...@telegraphics.com.au wrote: On 12-Feb-09, at 3:02 PM, Tim wrote: On Thu, Feb 12, 2009 at 11:31 AM, David Dyer-Bennet d...@dd-b.net wrote: On Thu, February 12, 2009 10:10, Ross wrote: Of course, that does assume that devices are being truthful when they say that data has been committed, but a little data loss from badly designed hardware is I feel acceptable, so long as ZFS can have a go at recovering corrupted pools when it does happen, instead of giving up completely like it does now. Well; not acceptable as such. But I'd agree it's outside ZFS's purview. The blame for data lost due to hardware actively lying and not working to spec goes to the hardware vendor, not to ZFS. If ZFS could easily and reliably warn about such hardware I'd want it to, but the consensus seems to be that we don't have a reliable qualification procedure. In terms of upselling people to a Sun storage solution, having ZFS diagnose problems with their cheap hardware early is clearly desirable :-). Right, well I can't imagine it's impossible to write a small app that can test whether or not drives are honoring correctly by issuing a commit and immediately reading back to see if it was indeed committed or not. You do realise that this is not as easy as it looks? :) For one thing, the drive will simply serve the read from cache. It's hard to imagine a test that doesn't involve literally pulling plugs; even better, a purpose built hardware test harness. Nonetheless I hope that someone comes up with a brilliant test. But if the ZFS team hasn't found one yet... it looks grim :) --Toby Like a zfs test cXtX. Of course, then you can't just blame the hardware everytime something in zfs breaks ;) --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS snapshot splitting joinin
I believe Tim Foster's zfs backup service (very beta atm) has support for splitting zfs send backups. Might want to check that out and see about modifying it for your needs. On Thu, Feb 5, 2009 at 3:15 PM, Michael McKnight michael_mcknigh...@yahoo.com wrote: Hi everyone, I appreciate the discussion on the practicality of archiving ZFS sends, but right now I don't know of any other options. I'm a home user, so Enterprise-level solutions aren't available and as far as I know, tar, cpio, etc. don't capture ACL's and other low-level filesystem attributes. Plus, they are all susceptible to corruption while in storage, making recovery no more likely than with a zfs send. The checksumming capability is a key factor to me. I would rather not be able to restore the data than to unknowingly restore bad data. This is the biggest reason I started using ZFS to start with. Too many cases of invisible file corruption. Admittedly, it would be nicer if zfs recv would flag individual files with checksum problems rather than completely failing the restore. What I need is a complete snapshot of the filesystem (ie. ufsdump) and, correct me if I'm wrong, but zfs send/recv is the closest (only) thing we have. And I need to be able to break up this complete snapshot into pieces small enough to fit onto a DVD-DL. So far, using ZFS send/recv works great as long as the files aren't split. I have seen suggestions on using something like 7z (?) instead of split as an option. Does anyone else have any other ideas on how to successfully break up a send file and join it back together? Thanks again, Michael -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Introducing zilstat
I'm already using it. This could be really useful for my Windows roaming-profile application of ZFS/NFS/SMB On Fri, Jan 30, 2009 at 9:35 PM, Richard Elling richard.ell...@gmail.com wrote: For those who didn't follow down the thread this afternoon, I have posted a tool call zilstat which will help you to answer the question of whether a separate log might help your workload. Details start here: http://richardelling.blogspot.com/2009/01/zilstat.html Enjoy! -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool status -x strangeness
Maybe ZFS hasn't seen an error in a long enough time that it considers the pool healthy? You could try clearing the pool and then observing. On Wed, Jan 28, 2009 at 9:40 AM, Ben Miller mil...@eecis.udel.edu wrote: # zpool status -xv all pools are healthy Ben What does 'zpool status -xv' show? On Tue, Jan 27, 2009 at 8:01 AM, Ben Miller mil...@eecis.udel.edu wrote: I forgot the pool that's having problems was recreated recently so it's already at zfs version 3. I just did a 'zfs upgrade -a' for another pool, but some of those filesystems failed since they are busy and couldn't be unmounted. # zfs upgrade -a cannot unmount '/var/mysql': Device busy cannot unmount '/var/postfix': Device busy 6 filesystems upgraded 821 filesystems already at this version Ben -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Raidz1 faulted with single bad disk. Requesting assistance.
I guess you could try 'zpool import -f'. This is a pretty odd status, I think. I'm pretty sure raidz1 should survive a single disk failure. Perhaps a more knowledgeable list member can explain. On Sat, Jan 24, 2009 at 12:48 PM, Brad Hill b...@thosehills.com wrote: I've seen reports of a recent Seagate firmware update bricking drives again. What's the output of 'zpool import' from the LiveCD? It sounds like ore than 1 drive is dropping off. r...@opensolaris:~# zpool import pool: tank id: 16342816386332636568 state: FAULTED status: The pool was last accessed by another system. action: The pool cannot be imported due to damaged devices or data. The pool may be active on another system, but can be imported using the '-f' flag. see: http://www.sun.com/msg/ZFS-8000-EY config: tankFAULTED corrupted data raidz1DEGRADED c6t0d0 ONLINE c6t1d0 ONLINE c6t2d0 ONLINE c6t3d0 UNAVAIL cannot open c6t4d0 ONLINE pool: rpool id: 9891756864015178061 state: ONLINE status: The pool was last accessed by another system. action: The pool can be imported using its name or numeric identifier and the '-f' flag. see: http://www.sun.com/msg/ZFS-8000-EY config: rpool ONLINE c3d0s0ONLINE -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Unable to destory a pool
Can you share the output of 'uname -a' and the disk controller you are using? On Sun, Jan 25, 2009 at 6:24 PM, Ramesh Mudradi rameshm.ku...@gmail.com wrote: # zpool list NAME SIZE USED AVAILCAP HEALTH ALTROOT jira-app-zpool 272G 330K 272G 0% ONLINE - The following command hangs forever. If I reboot the box , zpool list shows online as I mentioned the output above. # zpool destroy -f jira-app-zpool How can get rid of this pool and any reference to it. bash-3.00# zpool status pool: jira-app-zpool state: UNAVAIL status: One or more devices are faultd in response to IO failures. action: Make sure the affected devices are connected, then run 'zpool clear'. see: http://www.sun.com/msg/ZFS-8000-HC scrub: none requested config: NAMESTATE READ WRITE CKSUM jira-app-zpool UNAVAIL 0 0 4 insufficient replicas c3t0d3FAULTED 0 0 4 experienced I/O failures errors: 2 data errors, use '-v' for a list bash-3.00# -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Replacing HDD in x4500
I'm not an authority, but on my 'vanilla' filer, using the same controller chipset as the thumper, I've been in really good shape since moving to zfs boot in 10/08 and doing 'zpool upgrade' and 'zfs upgrade' to all my mirrors (3 3-way). I'd been having similar troubles to yours in the past. My system is pretty puny next to yours, but it's been reliable now for slightly over a month. On Tue, Jan 27, 2009 at 12:19 AM, Jorgen Lundman lund...@gmo.jp wrote: The vendor wanted to come in and replace an HDD in the 2nd X4500, as it was constantly busy, and since our x4500 has always died miserably in the past when a HDD dies, they wanted to replace it before the HDD actually died. The usual was done, HDD replaced, resilvering started and ran for about 50 minutes. Then the system hung, same as always, all ZFS related commands would just hang and do nothing. System is otherwise fine and completely idle. The vendor for some reason decided to fsck root-fs, not sure why as it is mounted with logging, and also decided it would be best to do so from a CDRom boot. Anyway, that was 12 hours ago and the x4500 is still down. I think they have it at single-user prompt resilvering again. (I also noticed they'd decided to break the mirror of the root disks for some very strange reason). It still shows: raidz1 DEGRADED 0 0 0 c0t1d0ONLINE 0 0 0 replacing UNAVAIL 0 0 0 insufficient replicas c1t1d0s0/o OFFLINE 0 0 0 c1t1d0 UNAVAIL 0 0 0 cannot open So I am pretty sure it'll hang again sometime soon. What is interesting though is that this is on x4500-02, and all our previous troubles mailed to the list was regarding our first x4500. The hardware is all different, but identical. Solaris 10 5/08. Anyway, I think they want to boot CDrom to fsck root again for some reason, but since customers have been without their mail for 12 hours, they can go a little longer, I guess. What I was really wondering, has there been any progress or patches regarding the system always hanging whenever a HDD dies (or is replaced it seems). It really is rather frustrating. Lund -- Jorgen Lundman | lund...@lundman.net Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work) Shibuya-ku, Tokyo| +81 (0)90-5578-8500 (cell) Japan| +81 (0)3 -3375-1767 (home) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool status -x strangeness
What does 'zpool status -xv' show? On Tue, Jan 27, 2009 at 8:01 AM, Ben Miller mil...@eecis.udel.edu wrote: I forgot the pool that's having problems was recreated recently so it's already at zfs version 3. I just did a 'zfs upgrade -a' for another pool, but some of those filesystems failed since they are busy and couldn't be unmounted. # zfs upgrade -a cannot unmount '/var/mysql': Device busy cannot unmount '/var/postfix': Device busy 6 filesystems upgraded 821 filesystems already at this version Ben You can upgrade live. 'zfs upgrade' with no arguments shows you the zfs version status of filesystems present without upgrading. On Jan 24, 2009, at 10:19 AM, Ben Miller mil...@eecis.udel.edu wrote: We haven't done 'zfs upgrade ...' any. I'll give that a try the next time the system can be taken down. Ben A little gotcha that I found in my 10u6 update process was that 'zpool upgrade [poolname]' is not the same as 'zfs upgrade [poolname]/[filesystem(s)]' What does 'zfs upgrade' say? I'm not saying this is the source of your problem, but it's a detail that seemed to affect stability for me. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
