Very interesting. 2 questions - (1) On the SQLite Download page - the Linux version just says "x86" while the Windows version is "Win-32 x86". Does this mean that the Linux version is a 64-bit version? In other words - if run on, say, RedHat 5.5 64-bit - it will be able to use >4GB of RAM?
(2) On Windows - a RamDisk "drive letter" can be used beneficially in 2 ways - (a) to contain the database itself (b) to serve as a destination for the TEMP and TMP - which SQLite (through Windows I suppose) uses often as a scratchpad. Is it possible - good/bad idea - to point the Windows TMP and TEMP to a "virtual" drive letter on top of a RamDisk? The RamDisk gets established as Windows comes up - but the question is whether Windows might need the TMP / TEMP prior to that point in time - when they are not yet going to be available - causing startup problems? On Sat, Mar 24, 2012 at 5:36 PM, Roger Binns <rog...@rogerbinns.com> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 24/03/12 13:48, Udi Karni wrote: > > I tried the SQLite in-memory DB feature - but when it exceeded about > > 2GB - I got the "out of memory" message. > > Technically it ran out of address space, not ram. > > > In my particular scenario - while the raw data being attached and read > > is hundreds of GB - the result sets are only a few GB. > > 2GB is the threshold to switch to 64 bits. Also note that SQLite does not > calculate the entire result set in advance. Instead when you call > sqlite3_step() it does whatever work is necessary to find the next result > row. It is only if you do something like require sorting that cannot be > satisfied by an index that the whole result set has to be calculated and > then sorted. > > > A 64-bit version of SQLite that could handle an in-memory DB of 5-10 > > GB would be quite useful and interesting to test. > > You can also use a ramdisk for the files with journalling turned off. > Microsoft used to have one, but now a search shows a number of companies. > I have no idea which ones are the most reputable. Under Linux you can use > tmpfs which makes things very easy. > > > Unfortunately I am not versed enough in compiling my own version - so > > hopefully this would be offered one of these days. > > Unfortunately making the free Microsoft dev tools generate 64 bit binaries > is a PITA. Another alternative is to use a 64 bit version Linux in which > case all this stuff just works. > > > Whether a RAM-only version would truly run subtantially faster than an > > SSD-based DB - remains to be seen - but the <2GB experiments seem to > > show that in-memory is quite promising. > > Any memory used to store databases is memory that cannot be used for I/O > caching. > > Roger > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > > iEYEARECAAYFAk9uaHgACgkQmOOfHg372QQhXQCg0c7MYT5s9WBSGcwyYWra7ZNx > 3qsAmwRSECp48Z2KkhfkMGvyvR7X1LsJ > =M4tq > -----END PGP SIGNATURE----- > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
