<[EMAIL PROTECTED]> wrote: > > The main reason seems to be that inserting in the btree is very slow (even > using transactions) because a lot of data has to be moved around in the file.
No. I think the reason is that the database file is too big to fit in your operating systems disk cache and hence reads and writes really do have to do actual disk I/O, which is slow. You just do not notice this so much on smaller databases because the I/O is really back and forth to the disk cache rather than to the disk itself. > > An other concern I have is row size: the average data in our data contains a > blob of about 3 kb. My understanding is that in that case, the first kilobyte > (primary key included) is stored in the b-tree node and the rest somewhere > else in the file. You might benefit from specifying a larger page size. Try: PRAGMA page_size=16384; Before creating any tables in a new database and see if that helps. Or recompile with -DSQLITE_DEFAULT_PAGE_SIZE=16384. -- D. Richard Hipp <[EMAIL PROTECTED]>