Simon, Dominique, and Arun - Thank you for the feedback. I'll leave the threading off for inserts since I've seen DB contention issues with other multithreaded/multiprocessed attempts. The indexing improvement is nice though.
I misspoke when I said it was 200-300gb for just the integers -- my brain was a little fuzzy. Right now, the integer table has 3 columns: rowid, integer, and foreign row id to a second table (so 8byte int, 8 byte int, variable byte int I believe, unless the rowid is also a variably-sized int). The rowid is left in to prevent using it as a primary key, or enforcing a unique constraint upfront which would result in an extremely slow insertion if I understand the basics correctly (and based on my own tests). This works out to about 180 gb of the table. There is a second table which stores some text information at 150 million rows, and then references my integer table. This makes up some of the difference. These rows are probably on the order of 200-300 bytes each so if I do the math, something in the 210-250 gb range seems reasonable. -AJ -- Sent from: http://sqlite.1065341.n5.nabble.com/ _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
