Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-22 Thread Chad Cantwell
Hi Garrett, Since my problem did turn out to be a debug kernel on my compilations, I booted back into the Nexanta 3 RC2 CD and let a scrub run for about half an hour to see if I just hadn't waited long enough the first time around. It never made it past 159 MB/s. I finally rebooted into my 145

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-21 Thread Richard Lowe
I built in the normal fashion, with the CBE compilers (cc: Sun C 5.9 SunOS_i386 Patch 124868-10 2009/04/30), and 12u1 lint. I'm not subscribed to zfs-discuss, but have you established whether the problematic build is DEBUG? (the bits I uploaded were non-DEBUG). -- Rich Haudy Kazemi wrote:

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-21 Thread Garrett D'Amore
On Wed, 2010-07-21 at 02:21 -0400, Richard Lowe wrote: I built in the normal fashion, with the CBE compilers (cc: Sun C 5.9 SunOS_i386 Patch 124868-10 2009/04/30), and 12u1 lint. I'm not subscribed to zfs-discuss, but have you established whether the problematic build is DEBUG? (the bits I

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-21 Thread Chad Cantwell
Hi, My bits were originally debug because I didn't know any better. I thought I had then recompiled without debug to test again, but I didn't realize until just now the packages end up in a different directory (nightly vs nightly-nd) so I believe after compiling non-debug I just reinstalled

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-21 Thread Chad Cantwell
It does seem to be faster now that I really installed the non-debug bits. I let it resume a scrub after reboot, and while it's not as fast as it usually is (280 - 300 MB/s vs 500+) I assume it's just presently checking a part of the filesystem currently with smaller files thus reducing the

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Chad Cantwell
On Mon, Jul 19, 2010 at 07:01:54PM -0700, Chad Cantwell wrote: On Tue, Jul 20, 2010 at 10:54:44AM +1000, James C. McPherson wrote: On 20/07/10 10:40 AM, Chad Cantwell wrote: fyi, everyone, I have some more info here. in short, rich lowe's 142 works correctly (fast) on my hardware, while

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Robert Milkowski
On 20/07/2010 07:59, Chad Cantwell wrote: I've just compiled and booted into snv_142, and I experienced the same slow dd and scrubbing as I did with my 142 and 143 compilations and with the Nexanta 3 RC2 CD. So, this would seem to indicate a build environment/process flaw rather than a

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Roy Sigurd Karlsbakk
I'm surprised you're even getting 400MB/s on the fast configurations, with only 16 drives in a Raidz3 configuration. To me, 16 drives in Raidz3 (single Vdev) would do about 150MB/sec, as your slow speeds suggest. That'll be for random i/o. His i/o here is sequential, so the i/o is spread

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Chad Cantwell
Yes, I think this might have been it. I missed the NIGHTLY_OPTIONS variable in opensolaris and I think it was compiling a debug build. I'm not sure what the ramifications are of this or how much slower a debug build should be, but I'm recompiling a release build now so hopefully all will be

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Chad Cantwell
No, this wasn't it. A non debug build with the same NIGHTLY_OPTIONS at Rich Lowe's 142 build is still very slow... On Tue, Jul 20, 2010 at 09:52:10AM -0700, Chad Cantwell wrote: Yes, I think this might have been it. I missed the NIGHTLY_OPTIONS variable in opensolaris and I think it was

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Brent Jones
On Tue, Jul 20, 2010 at 10:29 AM, Chad Cantwell c...@iomail.org wrote: No, this wasn't it.  A non debug build with the same NIGHTLY_OPTIONS at Rich Lowe's 142 build is still very slow... On Tue, Jul 20, 2010 at 09:52:10AM -0700, Chad Cantwell wrote: Yes, I think this might have been it.  I

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Chad Cantwell
On Tue, Jul 20, 2010 at 10:45:58AM -0700, Brent Jones wrote: On Tue, Jul 20, 2010 at 10:29 AM, Chad Cantwell c...@iomail.org wrote: No, this wasn't it.  A non debug build with the same NIGHTLY_OPTIONS at Rich Lowe's 142 build is still very slow... On Tue, Jul 20, 2010 at 09:52:10AM -0700,

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Garrett D'Amore
So the next question is, lets figure out what richlowe did differently. ;-) - Garrett ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Marcelo H Majczak
If I can help narrow the variables, I compiled both 137 and 144 (137 is minimum req. to build 144) using the same recommended compiler and lint, nightly options etc. 137 works fine but 144 suffer the slowness reported. System wise, I'm using only the 32bit non-debug version in an old

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Bill Sommerfeld
On 07/20/10 14:10, Marcelo H Majczak wrote: It also seems to be issuing a lot more writing to rpool, though I can't tell what. In my case it causes a lot of read contention since my rpool is a USB flash device with no cache. iostat says something like up to 10w/20r per second. Up to 137 the

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Bill Sommerfeld
On 07/20/10 14:10, Marcelo H Majczak wrote: It also seems to be issuing a lot more writing to rpool, though I can't tell what. In my case it causes a lot of read contention since my rpool is a USB flash device with no cache. iostat says something like up to 10w/20r per second. Up to 137 the

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Garrett D'Amore
Your config makes me think this is an atypical ZFS configuration. As a result, I'm not as concerned. But I think the multithread/concurrency may be the biggest concern here. Perhaps the compilers are doing something different that causes significant cache issues. (Perhaps the compilers

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-20 Thread Haudy Kazemi
Could it somehow not be compiling 64-bit support? -- Brent Jones I thought about that but it says when it boots up that it is 64-bit, and I'm able to run 64-bit binaries. I wonder if it's compiling for the wrong processor optomization though? Maybe if it is missing some of the newer

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-19 Thread Chad Cantwell
fyi, everyone, I have some more info here. in short, rich lowe's 142 works correctly (fast) on my hardware, while both my compilations (snv 143, snv 144) and also the nexanta 3 rc2 kernel (134 with backports) are horribly slow. I finally got around to trying rich lowe's snv 142 compilation in

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-19 Thread James C. McPherson
On 20/07/10 10:40 AM, Chad Cantwell wrote: fyi, everyone, I have some more info here. in short, rich lowe's 142 works correctly (fast) on my hardware, while both my compilations (snv 143, snv 144) and also the nexanta 3 rc2 kernel (134 with backports) are horribly slow. I finally got around to

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-19 Thread Garrett D'Amore
On Mon, 2010-07-19 at 17:40 -0700, Chad Cantwell wrote: fyi, everyone, I have some more info here. in short, rich lowe's 142 works correctly (fast) on my hardware, while both my compilations (snv 143, snv 144) and also the nexanta 3 rc2 kernel (134 with backports) are horribly slow. The idea

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-19 Thread Chad Cantwell
On Tue, Jul 20, 2010 at 10:54:44AM +1000, James C. McPherson wrote: On 20/07/10 10:40 AM, Chad Cantwell wrote: fyi, everyone, I have some more info here. in short, rich lowe's 142 works correctly (fast) on my hardware, while both my compilations (snv 143, snv 144) and also the nexanta 3 rc2

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-19 Thread Chad Cantwell
On Mon, Jul 19, 2010 at 06:00:04PM -0700, Brent Jones wrote: On Mon, Jul 19, 2010 at 5:40 PM, Chad Cantwell c...@iomail.org wrote: fyi, everyone, I have some more info here.  in short, rich lowe's 142 works correctly (fast) on my hardware, while both my compilations (snv 143, snv 144)

[zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-06 Thread Chad Cantwell
Hi all, I've noticed something strange in the throughput in my zpool between different snv builds, and I'm not sure if it's an inherent difference in the build or a kernel parameter that is different in the builds. I've setup two similiar machines and this happens with both of them. Each system