On 07.02.2011 21:51, Stefan Sperling wrote: >> A lot of wc databases out there will be >> so small that the user will hardly notice the memory increase. > All we'd be doing is allowing sqlite to flush data to disk if needed. > Even with a temporary table backed by a file, most operations happen in > memory. Either in buffers managed by sqlite or the operating system's > buffer cache (until sqlite does an fsync). So for small databases it > shouldn't make a difference.
Indeed, some data points for the patched recursive proplist case, in-memory vs. file-backed temporary storage: * subversion trunk working copy (2k nodes): file-backed is 5% slower * subversion tags tree (200k nodes): file-backed is 10% slower, svn proces uses up to 80M of memory. These numbers are not really significant given the 10x to infinityx speedup compared to the current code. But I'd suggest that once /all/ the queries are optimized in this way, temporary storage should be made file-backed by default as that makes horrible memory consumption less likely. -- Brane