Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
Richard Elling wrote: On Dec 25, 2009, at 4:15 PM, Erik Trimble wrote: I haven't seen this mentioned before, but the OCZ Vertex Turbo is still an MLC-based SSD, and is /substantially/ inferior to an Intel X25-E in terms of random write performance, which is what a ZIL device does almost exclusively in the case of NFS traffic. ZIL traffic tends to be sequential on separate logs. But may be of different sizes. -- richard Really? Now that I think about it, that seems to make sense - I was assuming that each NFS write would be relatively small, but that's likely not a valid general assumption. I'd still think that the MLC-nature of the Vertex isn't optimal for write-heavy applications like here, even with a modest SDRAM cache on the SSD. In fact, I think that the Vertex's sustained random write IOPs performance is actually inferior to a 15k SAS drive. I read a benchmark report yesterday that might be interesting. It seems that there is a market for modest sized SSDs, which would be perfect for separate logs + OS for servers. http://benchmarkreviews.com/index.php?option=com_contenttask=viewid=392Itemid=60 -- richard I'm still hoping that vendors realize that there definitely is a (very large) market for ~20GB high write IOPS SSDs.I like my 18GB Zeus SSD, but it sure would be nice to be able to pay $2/Gb for it, instead of 10x that now... -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
On Dec 25, 2009, at 3:01 PM, Jeroen Roodhart wrote: -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Hi Freddie, list, Option 4 is to re-do your pool, using fewer disks per raidz2 vdev, giving more vdevs to the pool, and thus increasing the IOps for the whole pool. 14 disks in a single raidz2 vdev is going to give horrible IO, regardless of how fast the individual disks are. Redoing it with 6-disk raidz2 vdevs, or even 8-drive raidz2 vdevs will give you much better throughput. We are aware of the configuration being possibly suboptimal. However, before we had the SSDs, we did test earlier with 6x7 Z2 and even 2way mirrorset setups. These gave better IOPS but not significantly enough improvement (I would expect roughly a bit more than double the performance in 14x3 vs 6x7) . In the end it is indeed a choice between performance, space and security. Our hope is that the SSD slogs serialise the data flow enough to make this work. But you have a fair point and we will also look into the combination of SSDs and pool-configurations. For your benchmark, there will not be a significant difference for any combination of HDDs. They all have at least 4 ms of write latency. Going from 10 ms down to 4 ms will not be nearly as noticeable as going from 10 ms to 0.01 ms :-) -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Hi Freddie, list, Option 4 is to re-do your pool, using fewer disks per raidz2 vdev, giving more vdevs to the pool, and thus increasing the IOps for the whole pool. 14 disks in a single raidz2 vdev is going to give horrible IO, regardless of how fast the individual disks are. Redoing it with 6-disk raidz2 vdevs, or even 8-drive raidz2 vdevs will give you much better throughput. We are aware of the configuration being possibly suboptimal. However, before we had the SSDs, we did test earlier with 6x7 Z2 and even 2way mirrorset setups. These gave better IOPS but not significantly enough improvement (I would expect roughly a bit more than double the performance in 14x3 vs 6x7) . In the end it is indeed a choice between performance, space and security. Our hope is that the SSD slogs serialise the data flow enough to make this work. But you have a fair point and we will also look into the combination of SSDs and pool-configurations. Also, possibly the Vortex Turbo SSDs aren't as good latency wise as I expected. Maybe the Sun SSDs will do a lot better. We will find this out when they arrive (due somewhere in february). With kind regards, Jeroen - -- Jeroen Roodhart IT Consultant University of Amsterdam j.r.roodh...@uva.nl Informatiseringscentrum Tel. 020 525 7203 - -- See http://www.science.uva.nl/~jeroen for openPGP public key -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFLNURP37AP1zFtDU0RA0QqAKDg/hr68JPjLvc0gaOmCe4RxPXY3QCg1G+g e6BTEKqq6QxpePonnn54fOo= =JQcb -END PGP SIGNATURE- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
Jeroen Roodhart wrote: -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Hi Freddie, list, Option 4 is to re-do your pool, using fewer disks per raidz2 vdev, giving more vdevs to the pool, and thus increasing the IOps for the whole pool. 14 disks in a single raidz2 vdev is going to give horrible IO, regardless of how fast the individual disks are. Redoing it with 6-disk raidz2 vdevs, or even 8-drive raidz2 vdevs will give you much better throughput. We are aware of the configuration being possibly suboptimal. However, before we had the SSDs, we did test earlier with 6x7 Z2 and even 2way mirrorset setups. These gave better IOPS but not significantly enough improvement (I would expect roughly a bit more than double the performance in 14x3 vs 6x7) . In the end it is indeed a choice between performance, space and security. Our hope is that the SSD slogs serialise the data flow enough to make this work. But you have a fair point and we will also look into the combination of SSDs and pool-configurations. Also, possibly the Vortex Turbo SSDs aren't as good latency wise as I expected. Maybe the Sun SSDs will do a lot better. We will find this out when they arrive (due somewhere in february). With kind regards, Jeroen - I haven't seen this mentioned before, but the OCZ Vertex Turbo is still an MLC-based SSD, and is /substantially/ inferior to an Intel X25-E in terms of random write performance, which is what a ZIL device does almost exclusively in the case of NFS traffic. In fact, I think that the Vertex's sustained random write IOPs performance is actually inferior to a 15k SAS drive. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
On Dec 25, 2009, at 4:15 PM, Erik Trimble wrote: I haven't seen this mentioned before, but the OCZ Vertex Turbo is still an MLC-based SSD, and is /substantially/ inferior to an Intel X25-E in terms of random write performance, which is what a ZIL device does almost exclusively in the case of NFS traffic. ZIL traffic tends to be sequential on separate logs. But may be of different sizes. -- richard In fact, I think that the Vertex's sustained random write IOPs performance is actually inferior to a 15k SAS drive. I read a benchmark report yesterday that might be interesting. It seems that there is a market for modest sized SSDs, which would be perfect for separate logs + OS for servers. http://benchmarkreviews.com/index.php?option=com_contenttask=viewid=392Itemid=60 -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
On Dec 25, 2009, at 6:01 PM, Jeroen Roodhart j.r.roodh...@uva.nl wrote: -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Hi Freddie, list, Option 4 is to re-do your pool, using fewer disks per raidz2 vdev, giving more vdevs to the pool, and thus increasing the IOps for the whole pool. 14 disks in a single raidz2 vdev is going to give horrible IO, regardless of how fast the individual disks are. Redoing it with 6-disk raidz2 vdevs, or even 8-drive raidz2 vdevs will give you much better throughput. We are aware of the configuration being possibly suboptimal. However, before we had the SSDs, we did test earlier with 6x7 Z2 and even 2way mirrorset setups. These gave better IOPS but not significantly enough improvement (I would expect roughly a bit more than double the performance in 14x3 vs 6x7) . In the end it is indeed a choice between performance, space and security. Our hope is that the SSD slogs serialise the data flow enough to make this work. But you have a fair point and we will also look into the combination of SSDs and pool-configurations. Also, possibly the Vortex Turbo SSDs aren't as good latency wise as I expected. Maybe the Sun SSDs will do a lot better. We will find this out when they arrive (due somewhere in february). Understand too that every write is striped across the vdev which means the vdev has the IOPS of a single drive. A 14 drive pool of mirrors would provide 7 times the IOPS then a 14 drive raidz2/3 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Mattias Pantzare wrote: The ZIL is _not_ optional as the log is in UFS. Right, thanks (also to Richard and Daniel) for the explanation. I was afraid this was to good to be true, nice to see it stated this clearly though. That would leave us with three options; 1) Deal with it and accept performance as it is. 2) Find a way to speed things up further for this workload 3) Stop trying to use ZFS for this workload Option 1 is not going to be feasible, so we're left with 2 and 3. We will have to do some more benchmarks in the new year. Maybe if trying different NFS wsize-s results in different figures. Also we'll look at UFS on the Thor, although I am not looking forward to handle large amounts of data on anything other than ZFS. Spoiled for life probably :) In the mean time, if any of you would have time to look at our iozone data and spotted glaring mistakes, we would definitely appreciate your comments. Thanks for your help, With kind regards, Jeroen - -- Jeroen Roodhart IT Consultant University of Amsterdam j.r.roodh...@uva.nl Informatiseringscentrum Tel. 020 525 7203 - -- See http://www.science.uva.nl/~jeroen for openPGP public key -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD4DBQFLMynd37AP1zFtDU0RA9dNAJdhfwGH7Dj3cXBCX3MS/zTaV/c+AKCozNJn kmxtdS9Vu/sM/icXTE0hsA== =fP4d -END PGP SIGNATURE- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Jeroen Roodhart wrote: Questions: 1. Client wsize? We usually set these to 342768 but this was tested with CenOS defaults: 8192 (were doing this over NFSv3) Is stand corrected here. Looking at proc/mounts I see we are in fact using different values: ... 10.0.0.1:/mypool/test_FS /nfs nfs rw,vers=3,rsize=1048576,wsize=1048576,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.0.0.1 0 0 So wsize was 1048576 during the iozone tests. That'll teach me to rely on manuals :) So repeating these tests with different wsizes seems to be a smart thing to do. With kind regards, Jeroen - -- Jeroen Roodhart IT Consultant University of Amsterdam j.r.roodh...@uva.nl Informatiseringscentrum Tel. 020 525 7203 - -- See http://www.science.uva.nl/~jeroen for openPGP public key -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFLMy0737AP1zFtDU0RA5OmAKDtt7LMRu/a36PBa+Fg5vL2pDKdGACdFIkU fWkmBPJvNwoQpm2A4Y3SorA= =gaX7 -END PGP SIGNATURE- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
On Dec 24, 2009, at 12:44 AM, Jeroen Roodhart wrote: -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Mattias Pantzare wrote: The ZIL is _not_ optional as the log is in UFS. Right, thanks (also to Richard and Daniel) for the explanation. I was afraid this was to good to be true, nice to see it stated this clearly though. That would leave us with three options; 1) Deal with it and accept performance as it is. 2) Find a way to speed things up further for this workload 3) Stop trying to use ZFS for this workload Option 1 is not going to be feasible, so we're left with 2 and 3. We will have to do some more benchmarks in the new year. Maybe if trying different NFS wsize-s results in different figures. Also we'll look at UFS on the Thor, although I am not looking forward to handle large amounts of data on anything other than ZFS. Spoiled for life probably :) UFS is really not an option. The absolute maximum possible size for a UFS file system is 16 TB, but 1 TB is the practical maximum. For a thumper or thor, you'd be crazy trying to manage 24 UFS file systems. Not a good return for the effort. In the mean time, if any of you would have time to look at our iozone data and spotted glaring mistakes, we would definitely appreciate your comments. How about posting the data somewhere we can see it? -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
[revisiting the OP] On Dec 23, 2009, at 8:27 AM, Auke Folkerts wrote: Hello, We have performed several tests to measure the performance using SSD drives for the ZIL. Tests are performed using a X4540 Thor with a zpool consisting of 3 14-disk RaidZ2 vdevs. This fileserver is connected to a Centos 5.4 machine which mounts a filesystem on the zpool via NFS, over a dedicated, direct, 1Gb ethernet link. The issue we're trying to resolve by using SSD's, is the much- discussed slow NFS performance when using synchronous IO. Unfortunately, asynchronous IO is not possible, since the Solaris NFS server is synchronous by default, and the linux clients are unable to request asynchronous NFS traffic. The SSD devices we've used are OCZ Vertex Turbo 30Gb disks. Data was gathered using iozone from the centos machine: (iozone -c -e -i 0 -i 1 -i 2 -o -a). For the archives, these options are interpreted as: -atest a number file and I/O sizes, 64 KB to 512 MB and 4 KB to 16 MB respectively. -cmeasure close() timing (note: Solaris file systems enforce sync-on-close, some Linux file systems make this optional thereby risking data) -e include flush in timing calculations (note: before you measure this, make sure your NFS server actually respects syncs, Solaris does) -i 0 run write/rewrite test -i 1 run read/re-read test (note: this tests server side read cache, and is probably not a very useful test unless both your client and server is memory constrained) -i 2 run random read/write test -owrites are synchronously written to the disk. Files are opened with the O_SYNC flag. This would make sense if your workload is a database, like Oracle, which opens its datafiles O_DSYNC. It is not at all representative of normal file system use. OK, so you are interested in finding out how fast sync writes can be written on a Solaris file system and NFS service that ensures sync writes are flushed to media. To fully understand how this is impacted by ZFS, you need to understand how the ZIL works and the impact of the logbias setting. This is why I asked about wsize, because by default, that is automatically adjusted by an algorithm that you can override with logbias settings. For tests that do a number of different I/O sizes, you should be able to see the knee in the curve as a result of this algorithm. In short, you can optimize for latency or bandwidth. Don't expect good latency when you are optimized for bandwidth. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Hi Richard, Richard Elling wrote: How about posting the data somewhere we can see it? As stated in an earlier posting it should be accessible at: http://init.science.uva.nl/~jeroen/solaris11_iozone_nfs2zfs Happy holidays! ~Jeroen - -- Jeroen Roodhart IT Consultant University of Amsterdam j.r.roodh...@uva.nl Informatiseringscentrum Tel. 020 525 7203 - -- See http://www.science.uva.nl/~jeroen for openPGP public key -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFLM9Pf37AP1zFtDU0RA0ZxAJ9IJmn8fc6sBWi2KG7sVlL8RLiZ0wCbBcbL 4+5IT9mxhgIqRm+j5Mx9Kqk= =nKUC -END PGP SIGNATURE- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
Mattias Pantzare wrote: That would leave us with three options; 1) Deal with it and accept performance as it is. 2) Find a way to speed things up further for this workload 3) Stop trying to use ZFS for this workload Option 4 is to re-do your pool, using fewer disks per raidz2 vdev, giving more vdevs to the pool, and thus increasing the IOps for the whole pool. 14 disks in a single raidz2 vdev is going to give horrible IO, regardless of how fast the individual disks are. Redoing it with 6-disk raidz2 vdevs, or even 8-drive raidz2 vdevs will give you much better throughput. Freddie -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
On Dec 24, 2009, at 5:34 PM, Freddie Cash wrote: Mattias Pantzare wrote: That would leave us with three options; 1) Deal with it and accept performance as it is. 2) Find a way to speed things up further for this workload 3) Stop trying to use ZFS for this workload Option 4 is to re-do your pool, using fewer disks per raidz2 vdev, giving more vdevs to the pool, and thus increasing the IOps for the whole pool. 14 disks in a single raidz2 vdev is going to give horrible IO, regardless of how fast the individual disks are. Redoing it with 6-disk raidz2 vdevs, or even 8-drive raidz2 vdevs will give you much better throughput. At this point it is useful to know that if you do not have a separate log, then the ZIL uses the pool and its data protection scheme. In other words, each ZIL write will be a raidz2 stripe with its associated performance. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
Hello, We have performed several tests to measure the performance using SSD drives for the ZIL. Tests are performed using a X4540 Thor with a zpool consisting of 3 14-disk RaidZ2 vdevs. This fileserver is connected to a Centos 5.4 machine which mounts a filesystem on the zpool via NFS, over a dedicated, direct, 1Gb ethernet link. The issue we're trying to resolve by using SSD's, is the much-discussed slow NFS performance when using synchronous IO. Unfortunately, asynchronous IO is not possible, since the Solaris NFS server is synchronous by default, and the linux clients are unable to request asynchronous NFS traffic. The SSD devices we've used are OCZ Vertex Turbo 30Gb disks. Data was gathered using iozone from the centos machine: (iozone -c -e -i 0 -i 1 -i 2 -o -a). The raw data as well as the graphs that I created are available on request, should people be interested. Since we are interested in using the Thor as an NFS file server for homedirectories, we are mostly concerned about random write performance. We have made the following observations: - Using SSD devices as ZIL logs yields a factor 2 improvement in throughput when using a recordsize = 128k, in comparison to using the internal ZIL devices of the pool (ie. not setting up slog devices). - With recordsizes of 1MB and up, having the ZIL reside on the raw disks of pool (no separate slog devices) outperforms using SSD's as a slog device. - Disabling the ZIL altogether yields significantly better performance (at least a factor 10). We had hoped that using SSD's would yield better performance. It is possible we will see an improvement with Intel X25-E series SSD's, but those haven't arrived yet so we can't test that. An alternative test we performed was extracting a 138Mb tarfile consisting of ~2000 small files. With the ZIL disabled, extracting the file took 4 seconds. With the ZIL enabled, but with no specific slog devices in the pool (thus using the disks in the pool), extraction took 72seconds. Using the SSD's as log devices, the time required was reduced to 34 seconds. This corresponds to the ~factor 2 improvement we noticed using our iozone benchmark. For this specific workload, we noticed no difference in using 1 or 2 (striped) slog SSD's. At the bottom line, lets end up with a few specific questions: 1. Is this performance using SSD's as expected? Can we expect better performance using Intel X25-E SSD's? 2. Disabling the ZIL looks like a serious option, after these performance benchmarks. I would expect to see Disabling the ZIL as an officially supported option, given that we all have used UFS for years, which is no better in terms of reliability. Is there an Official Sun Response to this? with kind regards, Auke Folkerts University of Amsterdam pgpdRrzeL41WP.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
Some questions below... On Dec 23, 2009, at 8:27 AM, Auke Folkerts wrote: Hello, We have performed several tests to measure the performance using SSD drives for the ZIL. Tests are performed using a X4540 Thor with a zpool consisting of 3 14-disk RaidZ2 vdevs. This fileserver is connected to a Centos 5.4 machine which mounts a filesystem on the zpool via NFS, over a dedicated, direct, 1Gb ethernet link. The issue we're trying to resolve by using SSD's, is the much- discussed slow NFS performance when using synchronous IO. Unfortunately, asynchronous IO is not possible, since the Solaris NFS server is synchronous by default, and the linux clients are unable to request asynchronous NFS traffic. The SSD devices we've used are OCZ Vertex Turbo 30Gb disks. Data was gathered using iozone from the centos machine: (iozone -c -e -i 0 -i 1 -i 2 -o -a). The raw data as well as the graphs that I created are available on request, should people be interested. Yes, can you post somewhere? Since we are interested in using the Thor as an NFS file server for homedirectories, we are mostly concerned about random write performance. We have made the following observations: - Using SSD devices as ZIL logs yields a factor 2 improvement in throughput when using a recordsize = 128k, in comparison to using the internal ZIL devices of the pool (ie. not setting up slog devices). - With recordsizes of 1MB and up, having the ZIL reside on the raw disks of pool (no separate slog devices) outperforms using SSD's as a slog device. - Disabling the ZIL altogether yields significantly better performance (at least a factor 10). We had hoped that using SSD's would yield better performance. It is possible we will see an improvement with Intel X25-E series SSD's, but those haven't arrived yet so we can't test that. An alternative test we performed was extracting a 138Mb tarfile consisting of ~2000 small files. With the ZIL disabled, extracting the file took 4 seconds. With the ZIL enabled, but with no specific slog devices in the pool (thus using the disks in the pool), extraction took 72seconds. Using the SSD's as log devices, the time required was reduced to 34 seconds. This corresponds to the ~factor 2 improvement we noticed using our iozone benchmark. For this specific workload, we noticed no difference in using 1 or 2 (striped) slog SSD's. Questions: 1. Client wsize? 2. Client NFS version? 3. logbias settings? 4. Did you test with a Solaris NFS client? If not, why not? At the bottom line, lets end up with a few specific questions: 1. Is this performance using SSD's as expected? Can we expect better performance using Intel X25-E SSD's? 2. Disabling the ZIL looks like a serious option, after these performance benchmarks. I would expect to see Disabling the ZIL as an officially supported option, given that we all have used UFS for years, which is no better in terms of reliability. Is there an Official Sun Response to this? UFS is a totally different issue, sync writes are always sync'ed. I don't work for Sun, but it would be unusual for a company to accept willful negligence as a policy. Ambulance chasing lawyers love that kind of thing. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benchmarks results for ZFS + NFS, using SSD's as slog devices (ZIL)
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Hi Richard, ZFS-discuss. Message: 2 Date: Wed, 23 Dec 2009 09:49:18 -0800 From: Richard Elling richard.ell...@gmail.com To: Auke Folkerts folke...@science.uva.nl Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] Benchmarks results for ZFS + NFS,using SSD's as slog devices (ZIL) Message-ID: 40070921-f894-4146-9e4c-7570d52c8...@gmail.com Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Some questions below... On Dec 23, 2009, at 8:27 AM, Auke Folkerts wrote: Filling in for Auke here, The raw data as well as the graphs that I created are available on request, should people be interested. Yes, can you post somewhere? I've put the results here, tests are run under nv129: http://www.science.uva.nl/~jeroen/solaris11_iozone_nfs2zfs Original measurements (with iozone headers) are in: http://www.science.uva.nl/~jeroen/solaris11_iozone_nfs2zfs/originals/ Questions: 1. Client wsize? We usually set these to 342768 but this was tested with CenOS defaults: 8192 (were doing this over NFSv3) 2. Client NFS version? NFSv3 (earlier tests show about 15% improvement using v4, but we still use v3 in production). 3. logbias settings? Throughput for runs labeled throughput otherwise defaults. 4. Did you test with a Solaris NFS client? If not, why not? We didn't, because our production environment consists of Solaris servers and Linux/MS Windows clients. UFS is a totally different issue, sync writes are always sync'ed. I don't work for Sun, but it would be unusual for a company to accept willful negligence as a policy. Ambulance chasing lawyers love that kind of thing. The Thor replaces a geriatric Enterprise system running Solaris 8 over UFS. For these workloads it beat the pants out of our current setup and somehow the but you're safer now argument doesn't go over very well :) We are under the impression that a setup that server NFS over UFS has the same assurance level than a setup using ZFS without ZIL. Is this impression false? If it isn't then offering a tradeoff between same assurance level as you are used to with better performance or better assurance level but for random-IO significant performance hits doesn't seem too wrong to me. In the first case you still have the ZFS guarantees once data is on disk... Thanks in advance for your insights, With kind regards, Jeroen - -- Jeroen Roodhart IT Consultant University of Amsterdam j.r.roodh...@uva.nl Informatiseringscentrum Tel. 020 525 7203 - -- See http://www.science.uva.nl/~jeroen for openPGP public key -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFLMqKT37AP1zFtDU0RAxeCAKDcglo2n0Q8Sx0tGyzx+MEGJt90TwCfWm59 JbGdTavhenqSrQEtGUvPZaw= =K25S -END PGP SIGNATURE- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
