-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 24/03/12 13:48, Udi Karni wrote: > I tried the SQLite in-memory DB feature - but when it exceeded about > 2GB - I got the "out of memory" message.
Technically it ran out of address space, not ram. > In my particular scenario - while the raw data being attached and read > is hundreds of GB - the result sets are only a few GB. 2GB is the threshold to switch to 64 bits. Also note that SQLite does not calculate the entire result set in advance. Instead when you call sqlite3_step() it does whatever work is necessary to find the next result row. It is only if you do something like require sorting that cannot be satisfied by an index that the whole result set has to be calculated and then sorted. > A 64-bit version of SQLite that could handle an in-memory DB of 5-10 > GB would be quite useful and interesting to test. You can also use a ramdisk for the files with journalling turned off. Microsoft used to have one, but now a search shows a number of companies. I have no idea which ones are the most reputable. Under Linux you can use tmpfs which makes things very easy. > Unfortunately I am not versed enough in compiling my own version - so > hopefully this would be offered one of these days. Unfortunately making the free Microsoft dev tools generate 64 bit binaries is a PITA. Another alternative is to use a 64 bit version Linux in which case all this stuff just works. > Whether a RAM-only version would truly run subtantially faster than an > SSD-based DB - remains to be seen - but the <2GB experiments seem to > show that in-memory is quite promising. Any memory used to store databases is memory that cannot be used for I/O caching. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk9uaHgACgkQmOOfHg372QQhXQCg0c7MYT5s9WBSGcwyYWra7ZNx 3qsAmwRSECp48Z2KkhfkMGvyvR7X1LsJ =M4tq -----END PGP SIGNATURE----- _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
