> I understand that the WAL log must take a lot of space. What I don't > understand is that it was 7x larger than the resulting DB size. (Actual > quotient is even larger because I compared to the DB size that contained > also other tables.)
Unlike a rollback journal a WAL file can have multiple copies of the same page. So from yor main loop, expanding the following code may help us understand. "insert all downloaded rows" If your inserted records is 5million separate insertions then each insertion could result in a table leaf page being written to the WAL file. Each insertion could potentially also result in one (or more) table interior pages being written to the WAL. Subsequent insertions could result in multiple copies of the same page. I don't know when any index would be updated - whether at each record update, or when the transaction is committed. But if the former then there could be a relatively large number of index leaf and interior pages. This could become signiifcant if an index is built on a large string field What is the schema for this table and what indexes are on it? The above is based on my knowledge of the file formats and what I have seen examining these files, not on the workings/code in SQLite so perhaps Dr Hipp could correct me if I'm wrong. Paul www.sandersonforensics.com skype: r3scue193 twitter: @sandersonforens Tel +44 (0)1326 572786 http://sandersonforensics.com/forum/content.php?195-SQLite-Forensic-Toolkit -Forensic Toolkit for SQLite http://sandersonforensics.com/forum/content.php?168-Reconnoitre - VSC processing made easy _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
