P Witte wrote: <snip>
As far as I know, nothing my program does should be affected by the size of the buffer, apart from filling it in the first place. So my findings would seem to indicate that a buffer size of between 256 bytes! and 1k are optimal for this kind of thing. This is strange enough, considering that iob.fmul is called more frequently the smaller the buffer. What surprises me is why were not seeing the benefits of iof.load in this (or at least I dont). Anyone got a theory?
I was watching this thread with interest. Maybe I should have commented earlier... that the results you have obtained are exactly what I was expecting.
If you had run your test on a microdrive... you would have found that "scatter load" for reading in files in their entirety may have had some effect (on a 68008, where the cost of copying the data might have a noticeable effect).
I'm not sure that "scatter load" is implemented in any floppy driver, although it could be done. It could easily cut the time to load a file by a factor of three, on a standard sort of floppy. That's because of the interleave. (logical to physical sector mapping does every third sector... the idea being that you get two sectors worth of time between writing one sector to get you data sorted for writing the "next" sector. A consideration which is pretty much wasted when reading a file!)
You mention "doing unspeakable things" to the contents of the files. If that means doing an *enormous* amount of processing, then the buffer size and disk transfer times will always be irrelevant. Indeed, even if you only did a byte search through the data for a single fixed value byte, using Basic, that would mash the buffer copy by a factor of fifty, at least, at a guess.
Then there's caching... if you had enough memory, your test would produce some curious results... whichever buffer size you ran first would seem to be rather slow compared to the rest. As you ran your tests on your hard disk, the caching effect doesn't come into it, but I thought I'd mention it.
Finally, it all depends on how fast your processor and hard disk are. These days, processors tend to have a lot of cycles available per byte transferred from(to) disk. That's the reason why compressed hard disk partitions are actually *faster* to handle that uncompressed. The processor has lots of spare time on its hands to perform the (de)compression. Overall time to process a file becomes tied mainly to the raw disk access time to get all the data on/off the disk. A compressed file occupies about half the physical space, so can be handled in handled in half the overall time.
Back to my earlier mention of caching... hard drives and their controllers do caching as well. I'm not certain if they do read-ahead caching.
In summary, I believe your results are just giving the shear time it takes the drive to spin on its axis tracks * cylinders * interleave times, and have very little to do with any CPU processing.
Answer to your original question: "How big a buffer should I use?"... one byte?
--
Lau
http://www.bergbland.info
Get a domain from http://oneandone.co.uk/xml/init?k_id=5165217 and I'll get the commission!
