Re: [zfs-discuss] gzip compression throttles system?

2007-05-03 Thread Dale Ghent
On May 2, 2007, at 10:36 PM, Ian Collins wrote: The files are between 15 and 50MB. It's worth pointing out that .wav files only compress by a few percent. Not entirely related to your maxed CPU problem, but gzip on PCM audio isn't, as you point out, going to earn you much of a

Re: [zfs-discuss] gzip compression throttles system?

2007-05-03 Thread Ian Collins
Dale Ghent wrote: On May 2, 2007, at 10:36 PM, Ian Collins wrote: The files are between 15 and 50MB. It's worth pointing out that .wav files only compress by a few percent. Not entirely related to your maxed CPU problem, but I don't think it was a maxed CPU problem, only one core was

[zfs-discuss] iscsitadm local_name in ZFS

2007-05-03 Thread cedric briner
hello dear community, Is there a way to have a ``local_name'' as define in iscsitadm.1m when you verbshareiscsi/verb a zvol. This way, it will give even easier way to identify an device through IQN. Ced. -- Cedric BRINER Geneva - Switzerland ___

Re: [zfs-discuss] gzip compression throttles system?

2007-05-03 Thread Wee Yeh Tan
Ian, On 5/3/07, Ian Collins [EMAIL PROTECTED] wrote: I don't think it was a maxed CPU problem, only one core was loaded and the prstat numbers I could get (the reporting period was erratic) didn't show anything nasty. Do you have the output of 'mpstat 5'? -- Just me, Wire ... Blog:

[zfs-discuss] Re: gzip compression throttles system?

2007-05-03 Thread Jürgen Keil
I just had a quick play with gzip compression on a filesystem and the result was the machine grinding to a halt while copying some large (.wav) files to it from another filesystem in the same pool. The system became very unresponsive, taking several seconds to echo keystrokes. The box is a

Re: [zfs-discuss] Re: gzip compression throttles system?

2007-05-03 Thread Darren J Moffat
Jürgen Keil wrote: A kernel profile seems to show that the kernel is busy with gzip'ing (and busy with computing SHA1 hashes?): # lockstat -kIW -D 20 sleep 20 Profiling interrupt: 3882 events in 20.021 seconds (194 events/sec) Count indv cuml rcnt nsec Hottest CPU+PILCaller

[zfs-discuss] Re: Re: gzip compression throttles system?

2007-05-03 Thread Jürgen Keil
The reason you are busy computing SHA1 hashes is you are using /dev/urandom. The implementation of drv/random uses SHA1 for mixing, actually strictly speaking it is the swrand provider that does that part. Ahh, ok. So, instead of using dd reading from /dev/urandom all the time, I've now

Re: [zfs-discuss] Re: Re: gzip compression throttles system?

2007-05-03 Thread Frank Hofmann
I'm not quite sure what this test should show ? Compressing random data is the perfect way to generate heat. After all, compression working relies on input entropy being low. But good random generators are characterized by the opposite - output entropy being high. Even a good compressor, if

Re: [zfs-discuss] Re: Re: gzip compression throttles system?

2007-05-03 Thread Rayson Ho
On 5/3/07, Frank Hofmann [EMAIL PROTECTED] wrote: I'm not quite sure what this test should show ? I didn't try the test myself... but I think what it shows is a possible problem that turning compression can hang a machine. Rayson Compressing random data is the perfect way to generate

Re: [zfs-discuss] Re: Re: gzip compression throttles system?

2007-05-03 Thread Roch Bourbonnais
with recent bits ZFS compression is now handled concurrently with many CPUs working on different records. So this load will burn more CPUs and acheive it's results (compression) faster. So the observed pauses should be consistent with that of a load generating high system time. The

Re: [zfs-discuss] Re: Re: gzip compression throttles system?

2007-05-03 Thread Wade . Stuart
[EMAIL PROTECTED] wrote on 05/03/2007 11:35:24 AM: with recent bits ZFS compression is now handled concurrently with many CPUs working on different records. So this load will burn more CPUs and acheive it's results (compression) faster. So the observed pauses should be consistent

Re: [zfs-discuss] Re: Re: gzip compression throttles system?

2007-05-03 Thread Ian Collins
Roch Bourbonnais wrote: with recent bits ZFS compression is now handled concurrently with many CPUs working on different records. So this load will burn more CPUs and acheive it's results (compression) faster. Would changing (selecting a smaller) filesystem record size have any effect? So

Re: [zfs-discuss] Re: Re: gzip compression throttles system?

2007-05-03 Thread johansen-osdev
A couple more questions here. [mpstat] CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 00 0 3109 3616 316 1965 17 48 45 2450 85 0 15 10 0 3127 3797 592 2174 17 63 46 1760 84 0 15 CPU minf mjf xcal

[zfs-discuss] ZFS vs UFS2 overhead and may be a bug?

2007-05-03 Thread Bakul Shah
[originally reported for ZFS on FreeBSD but Pawel Jakub Dawid says this problem also exists on Solaris hence this email.] Summary: on ZFS, overhead for reading a hole seems far worse than actual reading from a disk. Small buffers are used to make this overhead more visible. I ran the following

Re: [zfs-discuss] gzip compression throttles system?

2007-05-03 Thread Erblichs
Ian Collins, My two free cents.. If the gzip was in application space, most gzip's implementations support (maybe a new compile) a less extensive/expensive deflation that would consume fewer CPU cycles. Secondly, if the file objects are being written