Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread ohadp
what about the differences between the FULL and PARTIAL modes of pragma synchronous ? -- View this message in context: http://www.nabble.com/how-can-i-speed-up-inserts---tf3186848.html#a8860860 Sent from the SQLite mailing list archive at Nabble.com.

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread DragonK
On 2/8/07, Dennis Cote <[EMAIL PROTECTED]> wrote: DragonK wrote: > > > Well, using transactions would be a little difficult, because I have a > library which does something like logging to a database. Using > transaction > on a single insert would be useless in my opinion, and on the other >

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread Jay Sprenkle
On 2/7/07, ohadp <[EMAIL PROTECTED]> wrote: luckily this isn't mission critical failure, i'll take the risk that one out of ten thousand users experiences database corruption. i can only batch the inserts into maybe 4-5 a time, don't know how much of an improvement that will be... If

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread Gerry Snyder
ohadp wrote: when you say much less do you mean one in 1,000 or 1 in 1,000,000 ? Gerry Snyder-3 wrote: If you really can't get more than 4-5 in a batch, it's your call whether the time saving is worth the risk. BTW, the problem rate should be much less than one in ten thousand. Sorry. I

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread ohadp
when you say much less do you mean one in 1,000 or 1 in 1,000,000 ? Gerry Snyder-3 wrote: > > If you really can't get more than 4-5 in a batch, it's your call whether > the time saving is worth the risk. BTW, the problem rate should be much > less than one in ten thousand. > > Gerry > >

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread Dennis Cote
DragonK wrote: Well, using transactions would be a little difficult, because I have a library which does something like logging to a database. Using transaction on a single insert would be useless in my opinion, and on the other hand I can't use transactions on the all inserts, since

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread DragonK
On 2/7/07, John Stanton <[EMAIL PROTECTED]> wrote: DragonK wrote: > On 2/7/07, Ken <[EMAIL PROTECTED]> wrote: > >> >> Try without the pragma and wrap the inserts with a begin transaction >> and a >> commit... >> >> The performance will be almost as good as with the pragma, with the >> added

Re[4]: [sqlite] how can i speed up inserts ?

2007-02-07 Thread Teg
Hello DragonK, Essentially, when you have 1000's of uses banging on your software, many with poorly maintained/infected machines things that seldom happen on well maintained PC's will happen all the time. Some of my users have daily Windows crashes and assume that it's a normal experience for

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread John Stanton
DragonK wrote: On 2/7/07, Ken <[EMAIL PROTECTED]> wrote: Try without the pragma and wrap the inserts with a begin transaction and a commit... The performance will be almost as good as with the pragma, with the added benefit of consistent data and no corruption in the event of a crash

Re: Re[2]: [sqlite] how can i speed up inserts ?

2007-02-07 Thread Guillaume Fougnies
Wed, Feb 07, 2007 at 07:36:10PM +0200: DragonK wrote: > On 2/7/07, Teg <[EMAIL PROTECTED]> wrote: > > > >Hello ohadp, > > > >Well, my experience is, particularly when it's users using it and not > >a personal project, that corruption happens fairly frequently when you > >use this pragma. That's

Re: Re[2]: [sqlite] how can i speed up inserts ?

2007-02-07 Thread DragonK
On 2/7/07, Ken <[EMAIL PROTECTED]> wrote: Try without the pragma and wrap the inserts with a begin transaction and a commit... The performance will be almost as good as with the pragma, with the added benefit of consistent data and no corruption in the event of a crash or power failure. I

Re: Re[2]: [sqlite] how can i speed up inserts ?

2007-02-07 Thread Ken
Try without the pragma and wrap the inserts with a begin transaction and a commit... The performance will be almost as good as with the pragma, with the added benefit of consistent data and no corruption in the event of a crash or power failure. DragonK <[EMAIL PROTECTED]> wrote:

Re: Re[2]: [sqlite] how can i speed up inserts ?

2007-02-07 Thread DragonK
On 2/7/07, Teg <[EMAIL PROTECTED]> wrote: Hello ohadp, Well, my experience is, particularly when it's users using it and not a personal project, that corruption happens fairly frequently when you use this pragma. That's why I don't use it any more in my production code. Transactions are far

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread Michael Schlenker
Dennis Cote schrieb: ohadp wrote: holy smokes that pragma just made my several hundred inserts take half a second compared to 30 seconds earlier. thanks! Yes, but now your database can be corrupted by a power failure or OS crash. Wrapping several hundred inserts in a transaction will be

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread Gerry Snyder
ohadp wrote: luckily this isn't mission critical failure, i'll take the risk that one out of ten thousand users experiences database corruption. Only you can judge how important your data are. i can only batch the inserts into maybe 4-5 a time, don't know how much of an improvement that will

Re[2]: [sqlite] how can i speed up inserts ?

2007-02-07 Thread Teg
Hello ohadp, Well, my experience is, particularly when it's users using it and not a personal project, that corruption happens fairly frequently when you use this pragma. That's why I don't use it any more in my production code. Transactions are far safer and fast too. C Wednesday, February 7,

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread John Stanton
Try it. You might find that it executes in a similar time and it will still be ACID and safe. ohadp wrote: luckily this isn't mission critical failure, i'll take the risk that one out of ten thousand users experiences database corruption. i can only batch the inserts into maybe 4-5 a time,

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread ohadp
luckily this isn't mission critical failure, i'll take the risk that one out of ten thousand users experiences database corruption. i can only batch the inserts into maybe 4-5 a time, don't know how much of an improvement that will be... Dennis Cote wrote: > > ohadp wrote: >> holy smokes that

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread ohadp
holy smokes that pragma just made my several hundred inserts take half a second compared to 30 seconds earlier. thanks! DragonK wrote: > > On 2/7/07, Ohad Eder-Pressman <[EMAIL PROTECTED]> wrote: >> >> i've got sqlite compiled with visual-studio, with all the default >> options, >> didn't

Re: [sqlite] how can i speed up inserts ?

2007-02-07 Thread DragonK
On 2/7/07, Ohad Eder-Pressman <[EMAIL PROTECTED]> wrote: i've got sqlite compiled with visual-studio, with all the default options, didn't touch anything. inserts are just a bit too slow for me, is there a good way to speed this up using some flags ? any other flags that it would be smart to

[sqlite] how can i speed up inserts ?

2007-02-07 Thread Ohad Eder-Pressman
i've got sqlite compiled with visual-studio, with all the default options, didn't touch anything. inserts are just a bit too slow for me, is there a good way to speed this up using some flags ? any other flags that it would be smart to turn on for some better performance ? thanks