Re: [sqlite] Transaction behavior with large numbers of inserts

2009-01-28 Thread Ian Frosst
So thinking about this, what would be the impediment (apart from current implementation, of course,) to implementing a disk-based temp file page cache, that could grow arbitrarily large? Operations against the page cache in memory are already lost if your process dies before commission. Are

Re: [sqlite] Transaction behavior with large numbers of inserts

2009-01-28 Thread vlemaire
Hi, Does your reader threads access to the same table as your writer ? Maybe (I'm not 100% sure it works) you could try : on a single file database, with the PRAGMA read_uncommited=1 on the readers connections (this is not a good idea if the tables are the same between readers and writers, it

Re: [sqlite] Transaction behavior with large numbers of inserts

2009-01-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Griggs, Donald wrote: > The INSERT into main.table (with its associated SELECT) would still be > an implicit transaction and of course would require writing to the > database of the million records. > > During the rather long period of time for the

Re: [sqlite] Transaction behavior with large numbers of inserts

2009-01-28 Thread Griggs, Donald
-Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Roger Binns Sent: Wednesday, January 28, 2009 1:12 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] Transaction behavior with large numbers of inserts

Re: [sqlite] Transaction behavior with large numbers of inserts

2009-01-27 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ian Frosst wrote: > My question is; am I assessing the situation correctly, and if so, is there > anything I can do to avoid this rather nasty situation? I would rather not > take the periodic-commit approach, and increasing the page cache size would

Re: [sqlite] Transaction behavior with large numbers of inserts

2009-01-27 Thread Ken
; Subject: [sqlite] Transaction behavior with large numbers of inserts > To: sqlite-users@sqlite.org > Date: Tuesday, January 27, 2009, 6:49 PM > I have an application which on a single thread writes data > to a database. > This writing can result in millions of writes to the >

[sqlite] Transaction behavior with large numbers of inserts

2009-01-27 Thread Ian Frosst
I have an application which on a single thread writes data to a database. This writing can result in millions of writes to the database in a single transaction (started with BEGIN TRANSACTION.) I also have other threads and processes which read the same database, but never write to it, and never