Read_Write buffers for dd WAS: little cp diff
Moving this to m...@... Would part of this discussion usefully related to such issues like using 'dd' for diskwipes/copies/reformatting and slow data movement speeds? There are times when I am wiping (for reuse) hard disks using 'dd' and I set the BlockSize to 512 (like 1M or so sometimes) and the transfer speeds are quite a lot slower than for using 'dd' on some other Operating systems. (Linux or Windows) Mind you, for a lot of this, I am using oBSD RamDISK, so I am not anticipating a full-fledged OS support for the ATA or SCSI or USB2 platforms. But for those systems where I am using -stable or -current, the speeds are still comparably slow. I concur with Theo's point on portability and making a sysctl for kernel is hazardous, but what am I seeing in the above for 'dd' that would be causing the poor performance? (* BTW, I am using if=/dev/zero for the baseline, other if=/...'es may have lower performance as an input for compare*) Just my 2 cents. -sean Subject: Re: little cp diff 2010/2/8 Theo de Raadt dera...@cvs.openbsd.org: For those of you who asked why cp needs to be portable, come on. You've got it all wrong. If cp isn't written in a portable fashion, then what is the point of doing anything else in a portable fashion. This is good and reasonable answer. So I think we should stop discussion. antonvm _
Re: Read_Write buffers for dd WAS: little cp diff
On Mon, Feb 08, 2010 at 09:06:21AM -0500, Sean Kennedy wrote: Moving this to m...@... Would part of this discussion usefully related to such issues like using 'dd' for diskwipes/copies/reformatting and slow data movement speeds? There are times when I am wiping (for reuse) hard disks using 'dd' and I set the BlockSize to 512 (like 1M or so sometimes) and the transfer speeds are quite a lot slower than for using 'dd' on some other Operating systems. (Linux or Windows) Mind you, for a lot of this, I am using oBSD RamDISK, so I am not anticipating a full-fledged OS support for the ATA or SCSI or USB2 platforms. But for those systems where I am using -stable or -current, the speeds are still comparably slow. I concur with Theo's point on portability and making a sysctl for kernel is hazardous, but what am I seeing in the above for 'dd' that would be causing the poor performance? (* BTW, I am using if=/dev/zero for the baseline, other if=/...'es may have lower performance as an input for compare*) Just my 2 cents. -sean Well, for starters, different os, different caching behaviour. You should read from and write to the raw device, sync and use the actual elapsed time. Even then your measurement might be influenced by caching. # sync # time (dd if=/dev/rsd0a of=/dev/null bs=1m sync) 133+1 records in 133+1 records out 139797504 bytes transferred in 1.558 secs (89709129 bytes/sec) 0m1.56s real 0m0.00s user 0m0.07s system # OTOH, we do use a smaller size of the physical writes than other OSes. So a lower performance for modern hardware is not really a big surprise. -Otto
Re: Read_Write buffers for dd WAS: little cp diff
On Mon, 08 Feb 2010 09:06 -0500, Sean Kennedy woodentu...@hotmail.com wrote: Moving this to m...@... Would part of this discussion usefully related to such issues like using 'dd' for diskwipes/copies/reformatting and slow data movement speeds? There are times when I am wiping (for reuse) hard disks using 'dd' and I set the BlockSize to 512 (like 1M or so sometimes) In my experience, a bs of 64k is about as big and fast as you'll get. Setting bs larger than that may make dd a tad faster, but not much. Also, when IO errors occur with a larger bs you'll drop more data than you would have using a 512 byte block. Some modified dd's, such as ddrescue, set larger blocksizes initially in an effort to increase speed, but revert to 512 bytes upon IO errors. Brad and the transfer speeds are quite a lot slower than for using 'dd' on some other Operating systems. (Linux or Windows) Mind you, for a lot of this, I am using oBSD RamDISK, so I am not anticipating a full-fledged OS support for the ATA or SCSI or USB2 platforms. But for those systems where I am using -stable or -current, the speeds are still comparably slow. I concur with Theo's point on portability and making a sysctl for kernel is hazardous, but what am I seeing in the above for 'dd' that would be causing the poor performance? (* BTW, I am using if=/dev/zero for the baseline, other if=/...'es may have lower performance as an input for compare*) Just my 2 cents. -sean Subject: Re: little cp diff 2010/2/8 Theo de Raadt dera...@cvs.openbsd.org: For those of you who asked why cp needs to be portable, come on. You've got it all wrong. If cp isn't written in a portable fashion, then what is the point of doing anything else in a portable fashion. This is good and reasonable answer. So I think we should stop discussion. antonvm
