On 11/29/18, AJ Miles <ajm8...@gmail.com> wrote: > > I misspoke when I said it was 200-300gb for just the integers -- my brain > was a little fuzzy. Right now, the integer table has 3 columns: rowid, > integer, and foreign row id to a second table (so 8byte int, 8 byte int, > variable byte int I believe, unless the rowid is also a variably-sized int). > The rowid is left in to prevent using it as a primary key, or enforcing a > unique constraint upfront which would result in an extremely slow insertion > if I understand the basics correctly (and based on my own tests). This works > out to about 180 gb of the table. > > There is a second table which stores some text information at 150 million > rows, and then references my integer table. This makes up some of the > difference. These rows are probably on the order of 200-300 bytes each so if > I do the math, something in the 210-250 gb range seems reasonable. >
Just for grins, consider downloading the "bundle of command-line tools for managing SQLite database files" for your platform from https://sqlite.org/download.html and running the "sqlite3_analyzer" tool against your database. That tool will print out the average number of bytes used for each row of each table, along with additional information about how content is laid out in your database. Maybe post the results here. The utility will run at 1-5 GB/s so it might take a minute or three for it to complete. sqlite3_analyzer your-big-file.db >analysis.txt Then copy/paste analysis.txt into an email to this list. -- D. Richard Hipp d...@sqlite.org _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
