Re: [zfs-discuss] Sun's flash offering(s)
Hi All, Currently, the ssd's used in 7000 series are stec's, ssd's used inside servers are intel Sent from a mobile device Mertol Ozyoney On 20.Nis.2009, at 06:09, Scott Laird sc...@sigkill.org wrote: On Sun, Apr 19, 2009 at 10:20 AM, David Magda dma...@ee.ryerson.ca wrote: Looking at the web site for Sun's SSD storage products, it looks like what's been offered is the so-called Logzilla: http://www.sun.com/storage/flash/specs.jsp You know, those specs look almost *identical* to the Intel X25-E. Is this actually the STEC device, or just a rebranded Intel SSD? Not that there's anything wrong with the Intel or anything, but if you were going to buy it it'd probably be dramatically cheaper buying it from someone other than Sun, if Sun's service contract, etc, wasn't important to you. Compare the URL above with this one: http://www.intel.com/design/flash/nand/extreme/index.htm Scott ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Can the new consumer NAS devices run OpenSolaris?
Re-surfacing an old thread. I was wondering myself if there are any home-use commercial NAS devices with zfs. I did find that there is Thecus 7700. But, it appears to come with Linux, and use ZFS in FUSE, but I (perhaps unjustly) don't feel comfortable with :) Perhaps we will start to see more home NAS devices with zfs options, or at least to be able to run EON ? Joe S wrote: In the last few weeks, I've seen a number of new NAS devices released from companies like HP, QNAP, VIA, Lacie, Buffalo, Iomega, Cisco/Linksys, etc. Most of these are powered by Intel Celeron, Intel Atom, AMD Sempron, Marvell Orion, or Via C7 chips. I've also noticed that most allow a maximum of 1 or 2 GB of RAM. Is it likely that any of these will run OpenSolaris? Has anyone else tried? http://www.via.com.tw/en/products/embedded/nsd7800/ http://www.hp.com/united-states/digitalentertainment/mediasmart/serverdemo/index-noflash.html http://www.qnap.com/pro_detail_feature.asp?p_id=108 I prefer one of these instead of the huge PC I have at home. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Jorgen Lundman | lund...@lundman.net Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work) Shibuya-ku, Tokyo| +81 (0)90-5578-8500 (cell) Japan| +81 (0)3 -3375-1767 (home) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [on-discuss] Reliability at power failure?
dick hoogendijk wrote: Sorry Uwe, but the answer is yes. Assuming that your hardware is in order. I've read quite some msgs from you here recently and all of them make me think you're no fan of zfs at all. Why don't you quit using it and focus a little more on installing SunStudio I would really like to NOT chase people away from ZFS for any reason. There's no need. ZFS is currently a little too expert-friendly. I'm used to ZFS, so when it shows me messages, I know what it's saying. But when I read them a second time, I always wonder if we could word them to be more approachable without losing the precision. I would like to see alternate wordings suggested in RFEs, since I think some folks had good suggestions. As an example of wording that needs an upgrade: errors: Permanent errors have been detected in the following files: 0xa6:0x4f002 Could we not offer a clue that this was in metadata, even if it is darned hard to print a meaningful path name? Obligatory positive message: I was rewiring my monitors yesterday to get them all on a switchable power bar, and bumped a power switch briefly. The old dual Opteron machine hosting my storage pool did not power up again after that. I had an external Firewire case the pool had been destined for, and so I removed the drives and put them in the external case, and plugged the case into my SunBlade 2500. 'zpool import -f' went nicely, and I didn't lose a thing. I don't think any other filesystem or OS would make a recovery operation like this any easier. Oh yeah, this was after a mostly-effortless ZFS-accelerated Live Upgrade from snv_91 to snv_112 (almost a year) on another box. Rob T ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] What causes slow performance under load?
Tim wrote: On Sun, Apr 19, 2009 at 6:47 PM, Richard Elling richard.ell...@gmail.com mailto:richard.ell...@gmail.com wrote: I see no evidence of an I/O or file system bottleneck here. While the service times are a little higher than I expect, I don't get worried until the %busy is high and actv is high and asvc_t is high(er). I think your problem is elsewhere. NB when looking at ZFS, a 1 second interval for iostat is too small to be useful. 10 seconds is generally better, especially for older releases of ZFS (anything on Solaris 10). shameless plug ZFS consulting available at http://www.richardelling.com /shamelss plug -- richard So does that mean you don't work for Sun anymore...? I describe it as free of the shackles of the corporate jail, I can now recognize and act upon any opportunity I find interesting. With Sun being bought by Oracle, I have a feeling there will be plenty of opportunity... -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Update: sharenfs settings ignored
Crossposted since I think there may be zfs folks that are new to using the direct integration with nfs and may be confounded as I was. The problem as outlined exists as long as the client machine is not referenced in any kind of name resolution service. It turns out that if I can do a reverse lookup from the DNS server for the client IP address nfs connections are permitted, or if the IP address is listed in /etc/hosts. it doesn't matter what name you give it, just that the address resolves to a name and it will permit access. Can anyone explain this behaviour? It's manageable as long as you know this is the case, but it strikes me as a non obvious dependency since the subnet declaration should be sufficient to permit access (or at least it would appear to be the case from reading the documentation). Cheers, Erik Begin forwarded message: From: erik.ableson eable...@mac.com Date: 17 avril 2009 13:15:21 HAEC To: zfs-discuss@opensolaris.org Subject: [zfs-discuss] sharenfs settings ignored Hi there, I'm working on a new OS 2008.11 setup here and running into a few issues with the nfs integration. Notably, it appears that subnet values attributed to sharenfs are ignored and gives back a permission denied for all connection attempts. I have another environment where permission is assigned by FQDN which works fine, but I don't want to have to manage individual connections for server farms. Currently the server is running in a dedicated subnet (192.168.100.0/24) and the machines that will require access are running in two other subnets (192.168.0.0/24 192.168.254.0/24- ESX). The client machines are ESX Server, Mac OS X, Linux. From what I've been able to gather, I should be able to set specific permissions in CIDR syntax with the @ prefix) in the sharenfs value. I've tried a dozen different variants with no success. The one that I think should work is : sharenfs=...@192.168.0.0/24:@192.168.254.0/24,ro...@192.168.254.0/24 giving access to the client machines as well as giving root access to the ESX servers. Every connection attempt returns permission denied to the client. Trying with just a single subnet returns the same error. sharenfs=...@192.168.254.0/24,ro...@192.168.254.0/24 I've tried all of the following variants (and many others) with no success : sharenfs=on sharenfs=rw sharenfs=rw,anon=0 sharenfs=...@192.168.0.0/16 I did check tp make sure that the nfs server is running, :-) Everything looks fine from the sharemgr perspective: sharemgr show -vx zfs ?xml version=1.0? sharecfg group name=zfs state=enabled zfs=true group name=n01p01/nfs01 state=enabled zfs=true changed=true optionset type=nfs/ security type=nfs sectype=sys option type=rw value=@192.168.0.0/24:@192.168.254.0/24/ option type=root value=@192.168.254.0/24/ /security share path=/n01p01/nfs01 type=transient shared=true shareopts-nfs=sec=sys,r...@192.168.0.0/24:@192.168.254.0/24,ro...@192.168.254.0 /24/ /group /group /sharecfg From the client side of the house it looks fine: showmount -e 192.168.100.113 Exports list on 192.168.100.113: /n01p01/nfs01 @192.168.254.0/24 @192.168.0.0/24 Time to file a bug report? Or is there already one for this issue? Searching nfs subnet on defect.opensolaris.org returns nothing. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] simulating directio on zfs?
I had to let this go and get on with testing DB2 on Solaris. I had to abandon zfs on local discs in x64 Solaris 10 5/08. The situation was that: * DB2 buffer pools occupied up to 90% of 32GB RAM on each host * DB2 cached the entire database in its buffer pools o having the file system repeat this was not helpful * running high-load DB2 tests for 2 weeks showed 100% file-system writes and practically no reads Having database tables on zfs meant launching applications took minutes instead of sub-second. The test configuration I ended up with was: * transaction logs worked well to zfs on SAN with compression (nearly 1 TB of logs) * database tables worked well with ufs directio to multiple SCSI discs on each of 4 hosts (using DB2 database partitioning feature) I refer to DIRECTIO only as this already provides a reasonable set of hints to the OS: * reads and writes need not be cached * write() should not return until data is in non-volatile storage o DB2 has multiple concurrent write() threads to optimize this strategy * I/O will usually be in whole blocks aligned on block boundaries As an aside: It should be possible to state to zfs that a device cache is non-volatile (e.g. on SAN) and does not need flushing. Otherwise SAN must be configured to ignore all cache flush commands. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] What causes slow performance under load?
On Mon, Apr 20, 2009 at 11:35 AM, Richard Elling richard.ell...@gmail.comwrote: Tim wrote: On Sun, Apr 19, 2009 at 6:47 PM, Richard Elling richard.ell...@gmail.commailto: richard.ell...@gmail.com wrote: I see no evidence of an I/O or file system bottleneck here. While the service times are a little higher than I expect, I don't get worried until the %busy is high and actv is high and asvc_t is high(er). I think your problem is elsewhere. NB when looking at ZFS, a 1 second interval for iostat is too small to be useful. 10 seconds is generally better, especially for older releases of ZFS (anything on Solaris 10). shameless plug ZFS consulting available at http://www.richardelling.com /shamelss plug -- richard So does that mean you don't work for Sun anymore...? I describe it as free of the shackles of the corporate jail, I can now recognize and act upon any opportunity I find interesting. With Sun being bought by Oracle, I have a feeling there will be plenty of opportunity... -- richard LOL, fair enough :) Sorry for the intrusion if you will. I just noticed the @gmail instead of the @sun (perhaps I'm a bit slow) and was a bit taken aback to see someone so involved in zfs was no longer with Sun. I guess whatever it takes to make the books look good. Oracle: It should be an interesting ride to say the least. I guess we'll see just how much they love linux... either zfs et. all will become GPL, or we'll see their true colors. I'm secretly hoping for the latter (as long as they keep it open sourced). --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] simulating directio on zfs?
Andrew Robb wrote: I had to let this go and get on with testing DB2 on Solaris. I had to abandon zfs on local discs in x64 Solaris 10 5/08. This version does not have the modern write throttle code, which should explain much of what you experience. The fix is available in Solaris 10 10/08. For more info, see Roch's excellent blog http://blogs.sun.com/roch/entry/the_new_zfs_write_throttle One CR to reference is http://bugs.opensolaris.org/view_bug.do?bug_id=6429205 IMHO, if you are trying to make performance measurements on such old releases, then you are at great risk of wasting your time. You would be better served to look at more recent releases, within the constraints of your business, of course. -- richard The situation was that: * DB2 buffer pools occupied up to 90% of 32GB RAM on each host * DB2 cached the entire database in its buffer pools o having the file system repeat this was not helpful * running high-load DB2 tests for 2 weeks showed 100% file-system writes and practically no reads Having database tables on zfs meant launching applications took minutes instead of sub-second. The test configuration I ended up with was: * transaction logs worked well to zfs on SAN with compression (nearly 1 TB of logs) * database tables worked well with ufs directio to multiple SCSI discs on each of 4 hosts (using DB2 database partitioning feature) I refer to DIRECTIO only as this already provides a reasonable set of hints to the OS: * reads and writes need not be cached * write() should not return until data is in non-volatile storage o DB2 has multiple concurrent write() threads to optimize this strategy * I/O will usually be in whole blocks aligned on block boundaries As an aside: It should be possible to state to zfs that a device cache is non-volatile (e.g. on SAN) and does not need flushing. Otherwise SAN must be configured to ignore all cache flush commands. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] What causes slow performance under load?
On Mon, 20 Apr 2009, Richard Elling wrote: I describe it as free of the shackles of the corporate jail, I can now recognize and act upon any opportunity I find interesting. With Sun being bought by Oracle, I have a feeling there will be plenty of opportunity... Is this a forward-looking statement? Are you planning on taking your consulting company public soon so we can all invest in it? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] simulating directio on zfs?
Andrew Robb wrote: As an aside: It should be possible to state to zfs that a device cache is non-volatile (e.g. on SAN) and does not need flushing. Otherwise SAN must be configured to ignore all cache flush commands. ZFS already does the right thing. It sends a flush volatile cache command. If your storage array misbehaves and flushes non-volatile cache when it receives this command, get your storage vendor to fix their code. Disabling cache flushes entirely is just a hack to work around broken storage controller code. -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Sun Flash Modules
re == Richard Elling richard.ell...@gmail.com writes: re The win is nonvolatile main memory. When we get this on a re large, fast scale (and it will happen in our lifetime :-) then re we can begin to forget about file systems, with an interim re step through ramdisks. yeah I still think an unrealized and very cheap winning design would be to transform part of the main SDRAM into novolatile memory by adding a hardware watchdog device that backs it up to FLASH (and writing a software driver to handle the restore on boot). It's not hot-pluggable, but at least it's pluggable: with the power off you could move the flash module from one machine to another. The companies that write motherboard BIOS seem way too incompetent to manage this reliably enough to be useful, but for some high-end botique server maybe one could pull it off. but as for not needing filesystems, I can't imagine it. The smalltalk zealots like to talk about their persistence layer, but when you ask them, ``how do you upgrade the software while keeping the old data?'' they hem and haw and say ``it's not really *THAT* bad,'' but I suspect the reason DabbleDB is only available hosted isn't just revenue model---I bet they couldn't safely have customers doing their own software upgrades. I bet those guys do all upgrades with the Senior Wizard present and the debugger attached, and use convuluted schemes of snapshots and parallel development environments to supervise the whole delicate cutover. We'll still need snapshots, clones, backup/replication tools, ACL's MAC-labels zones, byterange locking, recovery/verification tools for spotting bugs in the NeoFilesystem code itself, u.s.w. I think the tree-of-bytestreams metaphor might end up enduring, but squeezing maximal performance out of a novolatile device that one can access without a disk driver, sometimes without even a syscall, will need new userland API's, a thick subtle library that can cooperate with other untrusted copies of itself, and a strong focus on ``zero copy''. pgpG7hJLu96dV.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] What causes slow performance under load?
On Mon, 20 Apr 2009, Tim wrote: Oracle: It should be an interesting ride to say the least. I guess we'll see just how much they love linux... either zfs et. all will become GPL, or we'll see their true colors. I'm secretly hoping for the latter (as long as they keep it open sourced). I don't think that GPL would be very wise, although a dual-license may be ok. Linux would need GPLv2, which is now out of date. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] What causes slow performance under load?
On Mon, Apr 20 at 14:19, Bob Friesenhahn wrote: On Mon, 20 Apr 2009, Tim wrote: Oracle: It should be an interesting ride to say the least. I guess we'll see just how much they love linux... either zfs et. all will become GPL, or we'll see their true colors. I'm secretly hoping for the latter (as long as they keep it open sourced). I don't think that GPL would be very wise, although a dual-license may be ok. Linux would need GPLv2, which is now out of date. GPL v2 may not be the most recent version, but a lot of people prefer GPLv2 to GPLv3, in the same way that some people might prefer Solaris 8 to Solaris 10, or Linux 2.4 kernels to the 2.6 series. I don't know who they are, but they certainly exist. -- Eric D. Mudama edmud...@mail.bounceswoosh.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] simulating directio on zfs?
Andrew Robb wrote: Richard Elling wrote: Andrew Robb wrote: I had to let this go and get on with testing DB2 on Solaris. I had to abandon zfs on local discs in x64 Solaris 10 5/08. This version does not have the modern write throttle code, which should explain much of what you experience. The fix is available in Solaris 10 10/08. For more info, see Roch's excellent blog http://blogs.sun.com/roch/entry/the_new_zfs_write_throttle One CR to reference is http://bugs.opensolaris.org/view_bug.do?bug_id=6429205 IMHO, if you are trying to make performance measurements on such old releases, then you are at great risk of wasting your time. You would be better served to look at more recent releases, within the constraints of your business, of course. -- richard This still misses the BIG point - DIRECTIO primarily tries to avoid data entering the file system cache (the database already caches it in its own much larger buffer pools). For this big-iron cluster, once written, the table data is only read back from file if the database is restarted. I suppose the same is also true of transaction logs, which are only replayed as part of data recovery. Typically, a database will be fastest if it avoids the file system altogether. However, this is difficult to manage and we will benefit greatly if file systems are nearly as fast as raw devices. There are a lot of misunderstandings surrounding directio. UFS directio offers the following 4 features: 1. no buffer cache (ZFS: primarycache property) 2. concurrent I/O (ZFS: concurrent by design) 3. async I/O code path (ZFS: more modern code path) 4. long urban myth history (ZFS: forgetaboutit ;-) The following pointers might be useful for you. http://blogs.sun.com/bobs/entry/one_i_o_two_i http://blogs.sun.com/roch/entry/people_ask_where_are_we What is missing from the above (note to self: blog this :-) is that you can limit the size of the buffer cache and control what sort of data is cached via the primarycache parameter. It really doesn't make a lot of sense to have zero buffer cache since *any* disk IO is going to be much, much, much more painful than any bcopy/memcpy. If you really don't want the ARC resizing on your behalf, then you can cap it, as we describe in the Evil Tuning Guide. http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Limiting_the_ARC_Cache But I think you'll find that the write throttle change is the big win and that primarycache gives you fine control of cache behaviour. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] What causes slow performance under load?
On Mon, Apr 20, 2009 at 01:04:59PM -0700, Eric D. Mudama wrote: On Mon, Apr 20 at 14:19, Bob Friesenhahn wrote: On Mon, 20 Apr 2009, Tim wrote: Oracle: It should be an interesting ride to say the least. I guess we'll see just how much they love linux... either zfs et. all will become GPL, or we'll see their true colors. I'm secretly hoping for the latter (as long as they keep it open sourced). I don't think that GPL would be very wise, although a dual-license may be ok. Linux would need GPLv2, which is now out of date. GPL v2 may not be the most recent version, but a lot of people prefer GPLv2 to GPLv3, in the same way that some people might prefer Solaris 8 to Solaris 10, or Linux 2.4 kernels to the 2.6 series. I don't know who they are, but they certainly exist. I wouldn't say GPLv2 is out of date. In fact, I don't think it'll ever go away as a lot of people see it as being more free than GPLv3. So, yes, GPLv3 has a higher version number, but it hardly obsoletes GPLv2 :-) (I think I'm basically agreeing with what you said here) Ray ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] What causes slow performance under load?
On Mon, 20 Apr 2009, Eric D. Mudama wrote: GPL v2 may not be the most recent version, but a lot of people prefer GPLv2 to GPLv3, in the same way that some people might prefer Solaris 8 to Solaris 10, or Linux 2.4 kernels to the 2.6 series. To be more clear, standard GPL provides the option for the user to use any later version. The Linux kernel uses a modified verison of GPLv2 which removes that option since they could not control the future of GPL. GPLv2 and GPLv3 are only similar in general intent. One prints in a page or two while the other requires a book. Due to this, ZFS would need to be licensed using GPLv2 in order to be included in the the Linux kernel. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] StorageTek 2540 performance radically changed
Hello Bob, Wednesday, April 15, 2009, 1:01:02 AM, you wrote: BF Today I updated the firmware on my StorageTek 2540 to the latest BF recommended version and am seeing radically difference performance BF when testing with iozone than I did in February of 2008. I am using BF Solaris 10 U5 with all the latest patches. BF This is the performance achieved (on a 32GB file) in February last BF year: BFKB reclen write rewritereadreread BF 33554432 64 279863 167138 458807 449817 BF 33554432 128 265099 250903 455623 460668 BF 33554432 256 265616 259599 451944 448061 BF 33554432 512 278530 294589 522930 471253 BF This is the new performance: BFKB reclen write rewritereadreread BF 33554432 64 76688 27870 552106 555438 BF 33554432 128 103120 369527 538206 555049 BF 33554432 256 355237 366563 534333 553660 BF 33554432 512 379515 364515 535635 553940 BF When using the 64KB record length, the service times are terrible. At BF first I thought that my drive array must be broken but now it seems BF like a change in the ZFS caching behavior (i.e. caching gone!): BF extended device statistics BF devicer/sw/s kr/s kw/s wait actv svc_t %w %b BF sd0 0.00.00.00.0 0.0 0.00.0 0 0 BF sd1 0.00.00.00.0 0.0 0.00.0 0 0 BF sd2 1.30.36.82.0 0.0 0.01.7 0 0 BF sd10 0.0 99.30.0 12698.3 0.0 32.2 324.5 0 97 BF sd11 0.3 105.9 38.4 12753.3 0.0 31.8 299.9 0 99 BF sd12 0.0 100.20.0 12095.9 0.0 26.4 263.8 0 82 BF sd13 0.0 102.30.0 12959.7 0.0 31.0 303.4 0 94 BF sd14 0.1 97.2 12.8 12291.8 0.0 30.4 312.0 0 92 BF sd15 0.0 99.70.0 12057.5 0.0 26.0 260.8 0 80 BF sd16 0.1 98.8 12.8 12634.3 0.0 31.9 322.1 0 96 BF sd17 0.0 99.00.0 12522.2 0.0 30.9 312.0 0 94 BF sd18 0.2 102.1 25.6 12934.1 0.0 29.7 290.4 0 90 BF sd19 0.0 103.40.0 12486.3 0.0 32.0 309.1 0 97 BF sd20 0.0 105.00.0 12678.3 0.0 32.1 305.6 0 98 BF sd21 0.1 103.9 12.8 12501.7 0.0 31.2 299.6 0 96 BF sd22 0.00.00.00.0 0.0 0.00.0 0 0 BF sd23 0.00.00.00.0 0.0 0.00.0 0 0 BF sd28 0.00.00.00.0 0.0 0.00.0 0 0 BF sd29 0.00.00.00.0 0.0 0.00.0 0 0 BF nfs1 0.00.00.00.0 0.0 0.00.0 0 0 BF Notice that the peak performance with large block writes is much BF better than it was before but the peak performance with smaller writes BF is much worse. When doing the smaller writes, the performance meter BF shows little blips every 10 seconds or so. BF One change is that I had applied a firmware tweak from Joel Miller BF (apparently no longer at Sun) to tell the array to ignore cache sync BF commands (i.e. don't wait for disk). This updated firmware seems BF totally different so it is unlikely that the firmware tweak will work. Well, you need to disable cache flushes on zfs side then (or make a firmware change work) and it will make a difference. -- Best regards, Robert Milkowski http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] StorageTek 2540 performance radically changed
On Tue, 21 Apr 2009, Robert Milkowski wrote: BF One change is that I had applied a firmware tweak from Joel Miller BF (apparently no longer at Sun) to tell the array to ignore cache sync BF commands (i.e. don't wait for disk). This updated firmware seems BF totally different so it is unlikely that the firmware tweak will work. Well, you need to disable cache flushes on zfs side then (or make a firmware change work) and it will make a difference. Based on results obtained when I re-ran the benchmark, it seems that these various tweaks are either no longer needed, or the existing tweaks were carried forward from the older firmware. I don't know what was going on the first time I ran the benchmark and saw odd performance with 16GB files. Large file writes are now almost wire speed given that I have two 4 Gbit FC optical links and am using mirroring with the mirror pairs carefully split across the two controllers. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] StorageTek 2540 performance radically changed
On 4/20/2009 7:26 PM, Robert Milkowski wrote: Well, you need to disable cache flushes on zfs side then (or make a firmware change work) and it will make a difference. If you're running recent OpenSolaris/Solaris/SX builds you shouldn't have to disable cache flushing on the array. The driver stack should set the correct modes. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] StorageTek 2540 performance radically changed
On Mon, 20 Apr 2009, Torrey McMahon wrote: On 4/20/2009 7:26 PM, Robert Milkowski wrote: Well, you need to disable cache flushes on zfs side then (or make a firmware change work) and it will make a difference. If you're running recent OpenSolaris/Solaris/SX builds you shouldn't have to disable cache flushing on the array. The driver stack should set the correct modes. Whatever Sun did with this new firmware (and of course the zfs enhancements) did wonderful things. This is the type of performance I am now seeing from the six mirror pairs: Synchronous random writes with 8k blocks and 8 writers: 3708.89 ops/sec Large file write: 359MB/second Large file read: 550MB/second All of which is much better than before. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Can the new consumer NAS devices run OpenSolaris?
I've gotten Nexenta installed onto a USB stick on a SS4200-E. To get it install required a PCI-E flex adapter. If you can reconfig EON for boot on a USB stick and serial console it might be possible. I've got two SS4200 and I might try EON on the second. Nicholas On Mon, Apr 20, 2009 at 8:39 PM, Jorgen Lundman lund...@gmo.jp wrote: Re-surfacing an old thread. I was wondering myself if there are any home-use commercial NAS devices with zfs. I did find that there is Thecus 7700. But, it appears to come with Linux, and use ZFS in FUSE, but I (perhaps unjustly) don't feel comfortable with :) Perhaps we will start to see more home NAS devices with zfs options, or at least to be able to run EON ? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] What causes slow performance under load?
On Sat, Apr 18, 2009 at 04:27:55PM -0500, Gary Mills wrote: We have an IMAP server with ZFS for mailbox storage that has recently become extremely slow on most weekday mornings and afternoons. When one of these incidents happens, the number of processes increases, the load average increases, but ZFS I/O bandwidth decreases. Users notice very slow response to IMAP requests. On the server, even `ps' becomes slow. After I moved a couple of Cyrus databases from ZFS to UFS on Sunday morning, the server seemed to run quite nicely. One of these databases is memory-mapped by all of the lmtpd and pop3d processes. The other is opened by all the lmtpd processes. Both were quite active, with many small writes, so I assumed they'd be better on UFS. All of the IMAP mailboxes were still on ZFS. However, this morning, things went from bad to worse. All writes to the ZFS filesystems stopped completely. Look at this: $ zpool iostat 5 5 capacity operationsbandwidth pool used avail read write read write -- - - - - - - space 1.04T 975G 86 67 4.53M 2.57M space 1.04T 975G 5 0 159K 0 space 1.04T 975G 7 0 337K 0 space 1.04T 975G 3 0 179K 0 space 1.04T 975G 4 0 167K 0 `fsstat' told me that there was both writes and memory-mapped I/O to UFS, but nothing to ZFS. At the same time, the `ps' command would hang and could not be interrupted. `truss' on `ps' looked like this, but it eventually also stopped and not be interrupted. open(/proc/6359/psinfo, O_RDONLY) = 4 read(4, 02\0\0\0\0\0\001\0\018D7.., 416) = 416 close(4)= 0 open(/proc/12782/psinfo, O_RDONLY)= 4 read(4, 02\0\0\0\0\0\001\0\0 1EE.., 416) = 416 close(4)= 0 What could cause this sort of behavior? It happened three times today! -- -Gary Mills--Unix Support--U of M Academic Computing and Networking- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
