Re: [zfs-discuss] X4500 ILOM thinks disk 20 is faulted, ZFS thinks not.
Jason J. W. Williams wrote: Have any of y'all seen a condition where the ILOM considers a disk faulted (status is 3 instead of 1), but ZFS keeps writing to the disk and doesn't report any errors? I'm going to do a scrub tomorrow and see what comes back. I'm curious what caused the ILOM to fault the disk. Any advice is greatly appreciated. What does `iostat -E` tell you? I've experienced several times that ZFS is very fault tolerant - a bit too tolerant for my taste - when it comes to faulting a disk. I saw external FC drives with hundreds or even thousands of errors, even entire hanging loops or drives with hardware trouble, and neither ZFS nor /var/adm/messages reported a problem. So I prefer examining the iostat output over `zpool status` - but with the unattractive side effect that it's not possible to reset the error count which iostat reports without a reboot, so this method is not suitable for monitoring purposes. -- Ralf Ramge Senior Solaris Administrator, SCNA, SCSA Tel. +49-721-91374-3963 [EMAIL PROTECTED] - http://web.de/ 11 Internet AG Brauerstraße 48 76135 Karlsruhe Amtsgericht Montabaur HRB 6484 Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Andreas Gauger, Thomas Gottschlich, Matthias Greve, Robert Hoffmann, Norbert Lang, Achim Weiss Aufsichtsratsvorsitzender: Michael Scheeren ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] X4500 ILOM thinks disk 20 is faulted, ZFS thinks not.
Hey Guys, Have any of y'all seen a condition where the ILOM considers a disk faulted (status is 3 instead of 1), but ZFS keeps writing to the disk and doesn't report any errors? I'm going to do a scrub tomorrow and see what comes back. I'm curious what caused the ILOM to fault the disk. Any advice is greatly appreciated. Best Regards, Jason P.S. The system is running OpenSolaris Build 54. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] X4500 ILOM thinks disk 20 is faulted, ZFS thinks not.
Hi Ralf, Thank you for the suggestion. About half of the disks are reporting 1968-1969 in the Soft Errors field. All disks are reporting 1968 in the Illegal Request field. There don't appear to be any other errors; all other counters are 0. The Illegal Request count seems a little fishy...like iostat -E doesn't like the X4500 for some reason. Thank you again for your help. Best Regards, Jason On Dec 4, 2007 2:54 AM, Ralf Ramge [EMAIL PROTECTED] wrote: Jason J. W. Williams wrote: Have any of y'all seen a condition where the ILOM considers a disk faulted (status is 3 instead of 1), but ZFS keeps writing to the disk and doesn't report any errors? I'm going to do a scrub tomorrow and see what comes back. I'm curious what caused the ILOM to fault the disk. Any advice is greatly appreciated. What does `iostat -E` tell you? I've experienced several times that ZFS is very fault tolerant - a bit too tolerant for my taste - when it comes to faulting a disk. I saw external FC drives with hundreds or even thousands of errors, even entire hanging loops or drives with hardware trouble, and neither ZFS nor /var/adm/messages reported a problem. So I prefer examining the iostat output over `zpool status` - but with the unattractive side effect that it's not possible to reset the error count which iostat reports without a reboot, so this method is not suitable for monitoring purposes. -- Ralf Ramge Senior Solaris Administrator, SCNA, SCSA Tel. +49-721-91374-3963 [EMAIL PROTECTED] - http://web.de/ 11 Internet AG Brauerstraße 48 76135 Karlsruhe Amtsgericht Montabaur HRB 6484 Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Andreas Gauger, Thomas Gottschlich, Matthias Greve, Robert Hoffmann, Norbert Lang, Achim Weiss Aufsichtsratsvorsitzender: Michael Scheeren ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Yager on ZFS
Your response here appears to refer to a different post in this thread. I never said I was a typical consumer. Then it's unclear how your comment related to the material which you quoted (and hence to which it was apparently responding). If you look around photo forums, you'll see an interest the digital workflow which includes long term storage and archiving. A chunk of these users will opt for an external RAID box (10%? 20%?). I suspect ZFS will change that game in the future. In particular for someone doing lots of editing, snapshots can help recover from user error. Ah - so now the rationalization has changed to snapshot support. Unfortunately for ZFS, snapshot support is pretty commonly available (e.g., in Linux's LVM - and IIRC BSD's as well - if you're looking at open-source solutions) so anyone who actually found this feature important has had access to it for quite a while already. And my original comment which you quoted still obtains as far as typical consumers are concerned. - bill This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS write time performance question
And some results (for OLTP workload): http://przemol.blogspot.com/2007/08/zfs-vs-vxfs-vs-ufs -on-scsi-array.html While I was initially hardly surprised that ZFS offered only 11% - 15% of the throughput of UFS or VxFS, a quick glance at Filebench's OLTP workload seems to indicate that it's completely random-access in nature without any of the sequential-scan activity that can *really* give ZFS fits. The fact that you were using an underlying hardware RAID really shouldn't have affected these relationships, given that it was configured as RAID-10. It would be interesting to see your test results reconciled with a detailed description of the tests generated by the Kernel Performance Engineering group which are touted as indicating that ZFS performs comparably with other file systems in database use: I actually don't find that too hard to believe (without having put all that much thought into it) when it comes to straight OLTP without queries that might result in sequential scans, but your observations seem to suggest otherwise (and the little that I have been able to infer about the methodology used to generate some of the rosy-looking ZFS performance numbers does not inspire confidence in the real-world applicability of those internally-generated results). - bill This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] clones bound too tightly to its origin
Hallo all, while experimenting with zfs send and zfs receive mixed with cloning on receiver side I found the following... On server A there is a zpool with snapshots created on regular basis via cron. Server B get's updated by a zfs-send-ssh-zfs-receive command pipe. Sometimes I want to do some testing on server B without corrupting data on server A. For doing so I create a clone of the filesystem. Up to here everything is ok. As long as the clone filesystem is NOT busy, any further zfs-send-ssh-zfs-receive will work properly, updating my pool on B without desturbing my clone. But there are some long running test jobs on server B which keeps clone's filesystem busy. Meanwhile another zfs-send-ssh-zfs-receive command gets launched to copy new snapshot from A to B. If the receiving pool of a zfs-receive-command has busy clones, the receive command will fail. For some unknown reason the receive command tries a umount my cloned filesystem and fails with Device busy. The question is: why? Since the clone is (or should be) independent of its origin, zfs receive should not umount cloned data of older snapshots. If you want to reproduce this - I've attached simple test script. The script will bump out at last zfs-receive command. If you comment out the cd /mnt/copy, script will run as expected. Disclaimer: Before running my script, make sure you do not have zpools named copy or origin. Use this script only on a test machine! Cleanup with zpool destroy copy; zpool destroy origin; rm /var/tmp/zpool.* after running tests. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] clones bound too tightly to its origin
I forgot to mention: we are running Solaris 10 Update 4 (08/07)... - Andreas This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance with Oracle
So, if your array is something big like an HP XP12000, you wouldn't just make a zpool of one big LUN (LUSE volume), you'd split it in two and make a mirror when creating the zpool? If the array has redundancy built in, you're suggesting to add another layer of redundancy using ZFS on top of that? We're looking to use this in our environment. Just wanted some clarification. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] I screwed up my zpool
Why didn't this command just fail? # zpool add tank c4t0d0 invalid vdev specification use '-f' to override the following errors: mismatched replication level: pool uses raidz and new vdev is disk I did not use '-f' and yet my configuration was changed. That was unexpected behaviour. Thanks for the advice tho, I will proceed with recreating the zpool. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] clones bound too tightly to its origin
It seems my script got lost during edit/posting message. I'll try again attaching... - Andreas This message posted from opensolaris.org test-zfs-clone.sh Description: Bourne shell script ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] current status of zfs boot partition on Sparc
I haven't seen anything about this recently, or I have missed it. Can anyone share what the current status of ZFS boot partition on Sparc is? Thanks, Jerry K ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] current status of zfs boot partition on Sparc
It's currently planned for integration into Nevada in the build 82 or 83 time frame. Lori Jerry K wrote: I haven't seen anything about this recently, or I have missed it. Can anyone share what the current status of ZFS boot partition on Sparc is? Thanks, Jerry K ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] mounting a volume as zfs
I can't decide if this is a dumb question or not (so I'll try asking it). We have two Solaris machines (Solaris 08/07) one (x86) with a load of disk attached and one (sparc) without. I've configured a volume on the disk server and made that available via iscsi. Connected to that on the sparc and set it up as a zpool. Once we started using this in earnest we got problems with stabilty and performance which I'll come back to later. For now I'd just like to get the data on this volume on to a zfs filesystem on the x86 machine. What I can't work out is if I can just mount this directly or do I still need to access it via iSCSI? The volume concerned is shelf01/sap-spare-01 in the list below (shelf02-sap-spare-03 is a copy of that generated via zfs send | zfs receive). I can obviously grab the data by using the existing iscsi connection on the sparc and copying it back (using ssh/scp) but this seems daft as the data is having to make a complete trip round the network. I can't decide if this is something that obviously can't work or if I'm missing some access mechanism using /dev/zvols or similar. Paul issdata3# zfs list -o name,type NAME TYPE shelf01filesystem shelf01/pete2 volume shelf01/sap-spare-01 volume shelf01/[EMAIL PROTECTED]snapshot shelf02filesystem shelf02/sap-spare-02 filesystem shelf02/sap-spare-03 volume shelf02/[EMAIL PROTECTED]snapshot issdata3# zfs list NAMEUSED AVAIL REFER MOUNTPOINT shelf01 143G 6.57T 24.5K /shelf01 shelf01/pete2 72.8G 6.57T 72.8G - shelf01/sap-spare-01 69.7G 6.57T 69.7G - shelf01/[EMAIL PROTECTED] 6.13M - 69.7G - shelf0269.7G 6.64T 24.5K /shelf02 shelf02/sap-spare-02 11.0M 6.64T 11.0M /data/sap-spare/02 shelf02/sap-spare-03 69.7G 6.64T 69.7G - shelf02/[EMAIL PROTECTED] 0 - 69.7G - sap-spare# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT store012.92T 69.7G 2.85T 2% ONLINE - sap-spare# zfs list NAME USED AVAIL REFER MOUNTPOINT store01 69.7G 2.81T 24.5K none store01/bw40.4G 2.81T 40.4G /zone_mounts/bw store01/crm 6.53G 2.81T 6.53G /zone_mounts/crm store01/ep12.2G 2.81T 12.2G /zone_mounts/ep store01/r310.6G 2.81T 10.6G /zone_mounts/r3 store01/trex 30.5K 2.81T 30.5K /zone_mounts/trex This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Yager on ZFS
On 11/7/07, can you guess? [EMAIL PROTECTED] wrote: However, ZFS is not the *only* open-source approach which may allow that to happen, so the real question becomes just how it compares with equally inexpensive current and potential alternatives (and that would make for an interesting discussion that I'm not sure I have time to initiate tonight). - bill Hi bill, only a question: I'm an ex linux user migrated to solaris for zfs and its checksumming; you say there are other open-source alternatives but, for a linux end user, I'm aware only of Oracle btrfs (http://oss.oracle.com/projects/btrfs/), who is a Checksumming Copy on Write Filesystem not in a final state. what *real* alternatives are you referring to??? if I missed something tell me, and I'll happily stay with linux with my data checksummed and snapshotted. bye --- Stefano Spinucci This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] why are these three ZFS caches using so much kmem?
James C. McPherson wrote: Got an issue which is rather annoying to me - three of my ZFS caches are regularly using nearly 1/2 of the 1.09Gb of allocated kmem in my system ...[snip] Following suggestions from Andre and Rich that this was probably the ARC, I've implemented a 256Mb limit for my system's ARC, per the Solaris Internals wiki: * http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#ARCSIZE * set arc max to 256Mb set zfs:zfs_arc_max=0x1000 And my system now seems to be chugging along quite happily. thankyou, James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Yager on ZFS
On 11/7/07, can you guess? [EMAIL PROTECTED] wrote: However, ZFS is not the *only* open-source approach which may allow that to happen, so the real question becomes just how it compares with equally inexpensive current and potential alternatives (and that would make for an interesting discussion that I'm not sure I have time to initiate tonight). - bill Hi bill, only a question: I'm an ex linux user migrated to solaris for zfs and its checksumming; So the question is: do you really need that feature (please quantify that need if you think you do), or do you just like it because it makes you feel all warm and safe? Warm and safe is definitely a nice feeling, of course, but out in the real world of corporate purchasing it's just one feature out of many 'nice to haves' - and not necessarily the most important. In particular, if the *actual* risk reduction turns out to be relatively minor, that nice 'feeling' doesn't carry all that much weight. you say there are other open-source alternatives but, for a linux end user, I'm aware only of Oracle btrfs (http://oss.oracle.com/projects/btrfs/), who is a Checksumming Copy on Write Filesystem not in a final state. what *real* alternatives are you referring to??? As I said in the post to which you responded, I consider ZFS's ease of management to be more important (given that even in high-end installations storage management costs dwarf storage equipment costs) than its real but relatively marginal reliability edge, and that's the context in which I made my comment about alternatives (though even there if ZFS continues to require definition of mirror pairs and parity groups for redundancy that reduces its ease-of-management edge, as does its limitation to a single host system in terms of ease-of-scaling). Specifically, features like snapshots, disk scrubbing (to improve reliability by dramatically reducing the likelihood of encountering an unreadable sector during a RAID rebuild), and software RAID (to reduce hardware costs) have been available for some time in Linux and FreeBSD, and canned management aids would not be difficult to develop if they don't exist already. The dreaded 'write hole' in software RAID is a relatively minor exposure (since it only compromises data if a system crash or UPS failure - both rare events in an enterprise setting - sneaks in between a data write and the corresponding parity update and then, before the array has restored parity consistency in the background, a disk dies) - and that exposure can be reduced to seconds by a minuscule amount of NVRAM that remembers which writes were active (or to zero with somewhat more NVRAM to remember the updates themselves in an inexpensive hardware solution). The real question is usually what level of risk an enterprise storage user is willing to tolerate. At the paranoid end of the scale reside the users who will accept nothing less than z-series or Tandem-/Stratus-style end-to-end hardware checking from the processor traces on out - which rules out most environments that ZFS runs in (unless Sun's N-series telco products might fill the bill: I'm not very familiar with them). And once you get down into users of commodity processors, the risk level of using stable and robust file systems that lack ZFS's additional integrity checks is comparable to the risk inherent in the rest of the system (at least if the systems are carefully constructed, which should be a given in an enterprise setting) - so other open-source solutions are definitely in play there. All things being equal, of course users would opt for even marginally higher reliability - but all things are never equal. If using ZFS would require changing platforms or changing code, that's almost certainly a show-stopper for enterprise users. If using ZFS would compromise performance or require changes in management practices (e.g., to accommodate file-system-level quotas), those are at least significant impediments. In other words, ZFS has its pluses and minuses just as other open-source file systems do, and they *all* have the potential to start edging out expensive proprietary solutions in *some* applications (and in fact have already started to do so). When we move from 'current' to 'potential' alternatives, the scope for competition widens. Because it's certainly possible to create a file system that has all of ZFS's added reliability but runs faster, scales better, incorporates additional useful features, and is easier to manage. That discussion is the one that would take a lot of time to delve into adequately (and might be considered off topic for this forum - which is why I've tried to concentrate here on improvements that ZFS could actually incorporate without turning it upside down). - bill This message posted from opensolaris.org ___ zfs-discuss mailing list
[zfs-discuss] ZFS with Memory Sticks
OK, I've been putting off this question for a while now, but it eating at me, so I can't hold off any more. I have a nice 8 gig memory stick I've formated with the ZFS file system. Works great on all my Solaris PC's, but refuses to work on my Sparc processor. So I've formated it on my Sparc machine (Blade 2500), works great there now, but not on my PC's. Re-Formatted it on my PC, doesn't work on Sparc, and so on and so on. I thought it was a file system to go back and forth both architectures. So when will this compatibility be here, or if it's possible now, what is the secret? Paul ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
