[sqlite] Anomalously slow performance on updates to early entries in a DB

2012-11-09 Thread O'Toole, Eamonn
Hello all, First of all, I'm a complete novice with respect to SQLite so apologies if there is an obvious answer to my question. I've also posted this question in IRC, and it was suggested that I post the question to this mailing list. We're running a test-bed of an object store (Openstack

Re: [sqlite] Anomalously slow performance on updates to early entries in a DB

2012-11-09 Thread Clemens Ladisch
O'Toole, Eamonn wrote: We are seeing a performance anomaly on updates to existing object records in the SQLite DB. If the container DB is sufficiently large (about 10 million objects, 3.3GB) then the time to update records at the beginning of the database by order of entry is anomalously

Re: [sqlite] Anomalously slow performance on updates to early entries in a DB

2012-11-09 Thread Black, Michael (IS)
Hmmm...is this a disk head seeking problem? You've got several TEXT entries which are either NULL or default to ''; I did a small experiment: on 3.7.13 CREATE TABLE container_stat ( account TEXT, container TEXT, created_at TEXT,

Re: [sqlite] Anomalously slow performance on updates to early entries in a DB

2012-11-09 Thread O'Toole, Eamonn
Thanks Michael. Comments below: -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- boun...@sqlite.org] On Behalf Of Black, Michael (IS) Sent: 09 November 2012 13:49 To: General Discussion of SQLite Database Subject: Re: [sqlite] Anomalously slow

Re: [sqlite] Anomalously slow performance on updates to early entries in a DB

2012-11-09 Thread O'Toole, Eamonn
Thanks Clemens, comments below: -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- boun...@sqlite.org] On Behalf Of Clemens Ladisch Sent: 09 November 2012 12:26 To: sqlite-users@sqlite.org Subject: Re: [sqlite] Anomalously slow performance on updates to

Re: [sqlite] Anomalously slow performance on updates to early entries in a DB

2012-11-09 Thread O'Toole, Eamonn
Sqlite_analyzer output: /** Disk-Space Utilization Report For container10M.db *** As of 2012-Nov-09 13:04:53 Page size in bytes 1024 Pages in the whole file (measured) 3412536 Pages in the whole file (calculated).. 3412535 Pages that store

Re: [sqlite] Anomalously slow performance on updates to early entries in a DB

2012-11-09 Thread Clemens Ladisch
O'Toole, Eamonn wrote: From: Clemens Ladisch Is it the DELETE or the INSERT that is slow, or both? Good question - I don't know Are you addressing the records by ROWID or by name? By name So when the DELETE searches for the record to be deleted, it has to scan the *entire* table ... ...