Re: ZFS performance degradation over time

2010-01-24 Thread Miroslav Lachman
Garrett Moore wrote: I've been watching my memory usage and I have no idea what is consuming memory as 'Active'. Last night I had around 6500MB 'Active' again, 1500MB Wired, no inact, ~30MB buf, no free, and ~100MB swap used. My performance copying ZFS-ZFS was again slow (1MB/s). I tried

Re: ZFS performance degradation over time

2010-01-24 Thread Miroslav Lachman
Miroslav Lachman wrote: [...] Last night I tried ZFS with pool on iSCSI connected Dell MD3000i and I was suprised by too low speed of simple cp -a command (copying from UFS partition to ZFS) The write speed was about 2MB/s only. After looking in to ARC stuff, I realized some weird values:

Re: ZFS performance degradation over time

2010-01-23 Thread Jonathan
On 1/19/2010 12:01 PM, Jeremy Chadwick wrote: On Tue, Jan 19, 2010 at 11:40:50AM -0500, Garrett Moore wrote: I've been watching my memory usage and I have no idea what is consuming memory as 'Active'. Last night I had around 6500MB 'Active' again, 1500MB Wired, no inact, ~30MB buf, no free,

Re: ZFS performance degradation over time

2010-01-23 Thread Erwin Lansing
On Wed, Jan 20, 2010 at 11:47:30AM +0100, Alexander Leidinger wrote: Quoting Jeremy Chadwick free...@jdc.parodius.com (from Tue, 19 Jan 2010 09:01:01 -0800): I've two recommendations: 1) Have you considered upgrading to RELENG_8 (e.g. 8.0-STABLE) instead of sticking with

Re: ZFS performance degradation over time

2010-01-22 Thread Alexander Leidinger
Quoting Jeremy Chadwick free...@jdc.parodius.com (from Tue, 19 Jan 2010 09:01:01 -0800): On Tue, Jan 19, 2010 at 11:40:50AM -0500, Garrett Moore wrote: I've been watching my memory usage and I have no idea what is consuming memory as 'Active'. Last night I had around 6500MB 'Active'

Re: ZFS performance degradation over time

2010-01-19 Thread Garrett Moore
I've been watching my memory usage and I have no idea what is consuming memory as 'Active'. Last night I had around 6500MB 'Active' again, 1500MB Wired, no inact, ~30MB buf, no free, and ~100MB swap used. My performance copying ZFS-ZFS was again slow (1MB/s). I tried killing rTorrent and no

Re: ZFS performance degradation over time

2010-01-19 Thread Jeremy Chadwick
On Tue, Jan 19, 2010 at 11:40:50AM -0500, Garrett Moore wrote: I've been watching my memory usage and I have no idea what is consuming memory as 'Active'. Last night I had around 6500MB 'Active' again, 1500MB Wired, no inact, ~30MB buf, no free, and ~100MB swap used. My performance copying

Re: ZFS performance degradation over time

2010-01-18 Thread Norbert Papke
On January 17, 2010, Garrett Moore wrote: I upgraded my system to 8GB of ram to see if that would help. It hasn't made much of a difference. After having rTorrent running for a while, my performance again tanked. Around 6.5GB of memory was showing as 'Active' according to top. 6.5GB of

Re: ZFS performance degradation over time

2010-01-17 Thread Garrett Moore
I upgraded my system to 8GB of ram to see if that would help. It hasn't made much of a difference. After having rTorrent running for a while, my performance again tanked. Around 6.5GB of memory was showing as 'Active' according to top. Copying a file from the zpool to another location on the zpool

Re: ZFS performance degradation over time

2010-01-09 Thread Alexander Leidinger
On Fri, 8 Jan 2010 15:05:06 -0800 Jeremy Chadwick free...@jdc.parodius.com wrote: All that said -- I know what the OP is referring to, as I've seen it myself (on RELENG_7, and possibly early releases of 8.0). The only way to relieve the pain, AFAIK, is to reboot. I assume the problem is that

Re: ZFS performance degradation over time

2010-01-09 Thread Ivan Voras
Andrew Snow wrote: Ivan Voras wrote: It is true that ZFS in theory doesn't do very well with random writes of any kind - the kind that torrent clients do should actually be the worst case for ZFS, *but*, this very much depends on the actual workload. ZFS has aggressive read-ahead for

Re: ZFS performance degradation over time

2010-01-09 Thread Wiktor Niesiobedzki
2010/1/8 Artem Belevich fbsdl...@src.cx: Keep an eye on ARC size and on active/inactive/cache/free memory lists: I can share a munin plugin for monitoring some of ARC L1/L2 statistics, as well as memory decomposition. It still WIP (and I'm sure, that not all statistics are properly gathered),

Re: ZFS performance degradation over time

2010-01-09 Thread Garrett Moore
After being up for a few days and having good performance, I turned on rTorrent and downloaded two 700MB files. After these torrents completed, my performance has tanked again. My combined read/write speed to the array won't exceed about 25MB/s. It seems that after having downloaded a few

Re: ZFS performance degradation over time

2010-01-09 Thread Jeremy Chadwick
On Sat, Jan 09, 2010 at 07:40:53PM -0500, Garrett Moore wrote: If no-one has any questions, I'll try Artem's suggestion of wasting a bunch of memory in Perl/Python and forcing some memory to be swapped out. (I don't want to do it yet in case someone wants a specific number before I do that).

Re: ZFS performance degradation over time

2010-01-09 Thread Garrett Moore
Sure: [r...@leviathan ~]# sysctl kstat.zfs.misc.arcstats kstat.zfs.misc.arcstats.hits: 32092629 kstat.zfs.misc.arcstats.misses: 1064835 kstat.zfs.misc.arcstats.demand_data_hits: 30542262 kstat.zfs.misc.arcstats.demand_data_misses: 848959 kstat.zfs.misc.arcstats.demand_metadata_hits: 1550367

Re: ZFS performance degradation over time

2010-01-08 Thread Garrett Moore
No, I haven't isolated the cause to only be uptime related. In my original email I mentioned that as suggested by someone in the thread, it's probably not directly related to system uptime, but instead related to usage - the more usage, the worse the performance. I've been starting my system with

Re: ZFS performance degradation over time

2010-01-08 Thread Artem Belevich
Keep an eye on ARC size and on active/inactive/cache/free memory lists: sysctl kstat.zfs.misc.arcstats.size sysctl vm.stats.vm.v_inactive_count sysctl vm.stats.vm.v_active_count sysctl vm.stats.vm.v_cache_count sysctl vm.stats.vm.v_cache_count ZFS performance does degrade a lot if ARC becomes

Re: ZFS performance degradation over time

2010-01-08 Thread Jonathan Noack
On Fri, January 8, 2010 11:31, Garrett Moore wrote: No, I haven't isolated the cause to only be uptime related. In my original email I mentioned that as suggested by someone in the thread, it's probably not directly related to system uptime, but instead related to usage - the more usage, the

Re: ZFS performance degradation over time

2010-01-08 Thread Jeremy Chadwick
On Fri, Jan 08, 2010 at 04:23:03PM -0500, Jonathan Noack wrote: On Fri, January 8, 2010 11:31, Garrett Moore wrote: No, I haven't isolated the cause to only be uptime related. In my original email I mentioned that as suggested by someone in the thread, it's probably not directly related

Re: ZFS performance degradation over time

2010-01-08 Thread Jeremy Chadwick
On Fri, Jan 08, 2010 at 03:05:06PM -0800, Jeremy Chadwick wrote: ... You can also explicitly enable prefetch by setting the value to 1, and this trumps the how-much-usable-RAM check. This should have read 'setting the value to 0'; sorry. -- | Jeremy Chadwick

Re: ZFS performance degradation over time

2010-01-08 Thread Andrew Snow
Ivan Voras wrote: It is true that ZFS in theory doesn't do very well with random writes of any kind - the kind that torrent clients do should actually be the worst case for ZFS, *but*, this very much depends on the actual workload. ZFS has aggressive read-ahead for sequential read-aheads, so

Re: ZFS performance degradation over time

2010-01-06 Thread Ivan Voras
On 3.1.2010 17:42, Garrett Moore wrote: I'm having problems with ZFS performance. When my system comes up, read/write speeds are excellent (testing with dd if=/dev/zero of=/tank/bigfile and dd if=/tank/bigfile of=/dev/null); I get at least 100MB/s on both reads and writes, and I'm happy with

ZFS performance degradation over time

2010-01-03 Thread Garrett Moore
Hello all, I posted a thread about ZFS performance issues in the General section of the forums and there were other people having the same issue, so I thought that it might be helpful to send the details to the mailing list as well. The thread can be found at:

Re: ZFS performance degradation over time

2010-01-03 Thread Richard Todd
Garrett Moore garrettmo...@gmail.com writes: I'm having problems with ZFS performance. When my system comes up, read/write speeds are excellent (testing with dd if=/dev/zero of=/tank/bigfile and dd if=/tank/bigfile of=/dev/null); I get at least 100MB/s on both reads and writes, and I'm happy