Re: [sqlite] Bug report: Out-Of-Memory error when doing DELETE from a table with 150 million records

2012-10-18 Thread Ivan P
Hi Richard, Shouldn't the delete statement be able to flush it's stored rowids to disk when it understands the memory is not enough for handling. Otherwise it doesn't seem scalable enough. To avoid this we decided to change a database structure. I would consider this thread as not solving my

Re: [sqlite] Bug report: Out-Of-Memory error when doing DELETE from a table with 150 million records

2012-10-18 Thread Ivan P
The operating system is Windows7 x64 Ultimate, 4 Gb RAM I have not specified any PRAGMAs when run sqlite3.exe. The program was launched normally (in non-elevated mode). The database file is located on the local disk (C:) with a 95 GB of free space Here's how I launch this:

Re: [sqlite] Bug report: Out-Of-Memory error when doing DELETE from a table with 150 million records

2012-10-17 Thread Richard Hipp
On Wed, Oct 17, 2012 at 11:58 AM, Ivan P wrote: > Hello! > > I've got Out-Of-Memory error when delete records from a table that has > about 150,000,000 records. > > The table is created as: > CREATE TABLE differential_parts_temp (plan_id TEXT, [version_id] INT NOT >

Re: [sqlite] Bug report: Out-Of-Memory error when doing DELETE from a table with 150 million records

2012-10-17 Thread Simon Slavin
On 17 Oct 2012, at 4:58pm, Ivan P wrote: > Why the DELETE statement can eat so much memory? Because it doesn't delete each one row singly, doing all the file updates that are needed to delete that row, then move on to the next row. If it did it would take an

[sqlite] Bug report: Out-Of-Memory error when doing DELETE from a table with 150 million records

2012-10-17 Thread Ivan P
Hello! I've got Out-Of-Memory error when delete records from a table that has about 150,000,000 records. The table is created as: CREATE TABLE differential_parts_temp (plan_id TEXT, [version_id] INT NOT NULL, [page_index] INT NOT NULL, [md5low] INT64 NOT NULL, [md5high] INT64 NOT NULL); CREATE