On Mon, Dec 27, 2004 at 08:51:11PM -0800, Roger Binns wrote:Why do you need SQLite if you want everything in memory? All you gain is an SQL Parser to operate on the data structures instead of just doing it directly yourself.
All? You gain the relational model, which in some cases could be a huge win. I have considered using in-memory SQLite (haven't actually done it yet) rather than lots of ugly hash tables for exactly that reason.
That would be true for smaller amounts of data. The OP was concerned
about the 2GB address space limit. Bypassing SQLite in that case means your code can know very specifically what all data types are, can have perfect hash functions and can make memory vs cpu tradeoffs
as appropriate. It can also do interning if there are frequent common values. And it can do things like allocate all the memory at program startup rather than throughout execution in order to be
resilient to when the address space does fill up.
Other than some circumstances such as transient data, the in memory database doesn't buy you much. You can always tell SQLite how much memory to use to get desired performance, and having a disk based database
makes it easy to have checkpointing.
Roger
