On Mon, 11 Dec 2006 19:50:52 -0600, John Stanton wrote: > Use a Unix read or a Windows API ReadFile and read the whole file > in one call. > Find the size of it using a Unix lseek or > Win32 API GetFileSize call. You will certainly > pull the whole file into VM that way.
Would that be wise for a database larger than a few tens of MegaBytes? I guess it could cause some serious swapping, invalidate hardware disk caches again, and make the rest of the system slow... The large block (does Visual Basic support such a large 'blob' variable?) you'd read would be in Virtual Memory all right, but as there is no relation between a large variable in memory and the sqlite database, any database access would cause pages to be read from disk again, the memory would not be used, only hardware- and software diskcaches. Read/discard in 64 kByte chunks seems just a little better, that could still load the database in the OS disk cache, depending on the disk cache size, and VM strategy settings of the system. Defining a large sqlite database page cache and prefilling it with the most frequently used pages by executing a carefully crafted query would be a better answer, but that only helps when the database is kept open all the time. Frankly, I wouldn't bother; I'd just define a large sqlite cache and a suitable database page size and have the first user / process digest the initial delay. If the database isn't used for a while on an otherwise busy system, disk caches would quickly be reused by something else anyway. >Jay Sprenkle wrote: >> On 12/11/06, Dennis Cote <[EMAIL PROTECTED]> wrote: >> >>> >>> Simply read the entire file once using fread when you open the database. >>> That will load the entire file in to the OS cache so that when SQLite >>> starts reading the file, the required disk blocks are already in memory. >>> The result is the same fast lookups that you see in SQLite the second >>> time you run a query. The additional read is also quite fast since it >>> reads the file sequentially from start to finish with no seeking, >>> whereas SQLite will seek back and forth as it reads database pages into >>> memory. >> >> >> You probably can't fseek() to the end and get the same effect? >> >> You could do this in vb6 by writing a loop and using "input #1" >> and discarding what was read. In vb.net it's much simpler using >> a file stream reader. >> >> -- >> The JS Image Collector suite: >> http://groups-beta.google.com/group/js-image-collector?hl=en >> >> SqliteImporter and SqliteReplicator: Command line utilities for Sqlite >> http://www.reddawn.net/~jsprenkl/Sqlite >> >> Cthulhu Bucks! >> http://www.cthulhubucks.com -- ( Kees Nuyt ) c[_] ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
