On 11/8/07, Julien Renggli <[EMAIL PROTECTED]> wrote: > As I said, the "nasty" trick works and we'll live with it (writing our > own FileSystem is not an option). We would just like to understand > better what's happening, to know why and when the "first run" is so much > slower. And should we read the whole file beforehand, only parts of it? > Does it depend on how much RAM is installed, ...? If you have any hints > they are welcome. I guess I should ask NTFS experts as well.
What you've discovered is pretty accurate: when the delay is due to disk I/O, pre-reading the file will load it into the OS's disk cache, so subsequent accesses are faster. It will depend on available RAM, not only what is physically installed but also memory pressure from other running applications, other disk I/O in progress, various OS settings, etc. There isn't any way to accurately predict it. Reading the entire file when the OS is unwilling to cache all of it will simply result in only part of the file being cached. It may also cause other data to be pushed into the pagefile, slowing down applications when they later access their own stale data. In the worst case, it could slow everything down for a short time. Reading the entire file from start to finish is pretty much the only effective way to pull it into cache. An ifstream is not the most efficient way to do that, since it does its own buffering, but that's not important as far as the disk caching effects are concerned. VACUUM removes internal fragmentation and writes rows in order, which helps make disk I/O more sequential. You may be able to one-up it by inserting data in the order you intend to access it (but create any indexes afterward). Beyond that, I'm not aware of anything that would help. ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
