Re: [sqlite] EXTERNAL:Re: Sqlite Insert Speed Optimization

2010-07-15 Thread Paul Corke
On 14 July 2010 17:24, Werner Smit wrote: > I'd however like to test this with a local server - is there a faq > somewhere about compressed ssh tunneling on oracle ports? Is it open > source? Have you done it yourself? What was speed improvement? As mentioned by other posters, compressing on

Re: [sqlite] EXTERNAL:Re: Sqlite Insert Speed Optimization

2010-07-14 Thread Werner Smit
On 14 July 2010 17:00, Werner Smit wrote: > It DOES sound terrible since 90%? of the time is spend in retrieving > data from a remote oracle server over a slow line. I think you're trying to optimise the wrong thing :) Assuming you can't upgrade that slow line, how about running a compressed

Re: [sqlite] EXTERNAL:Re: Sqlite Insert Speed Optimization

2010-07-14 Thread Paul Corke
On 14 July 2010 17:00, Werner Smit wrote: > It DOES sound terrible since 90%? of the time is spend in retrieving > data from a remote oracle server over a slow line. I think you're trying to optimise the wrong thing :) Assuming you can't upgrade that slow line, how about running a compressed

Re: [sqlite] EXTERNAL:Re: Sqlite Insert Speed Optimization

2010-07-14 Thread Werner Smit
>According to my math your final database size should be on the order of 100Meg? > >That means at 200 minutes and 1,000,000 records: >83 inserts per second >8333 bytes per second >Both of these values are terrible. >#1 What kind of network connection do you have? 100BaseT? >#2 What kind of

Re: [sqlite] EXTERNAL:Re: Sqlite Insert Speed Optimization

2010-07-14 Thread Black, Michael (IS)
According to my math your final database size should be on the order of 100Meg? That means at 200 minutes and 1,000,000 records: 83 inserts per second 8333 bytes per second Both of these values are terrible. #1 What kind of network connection do you have? 100BaseT? #2 What kind of server

Re: [sqlite] EXTERNAL:Re: Sqlite Insert Speed Optimization

2010-07-14 Thread Black, Michael (IS)
You've got the source code. Modify sqlite3journalopen to put your journal in %TEMP% or or maybe getcwd(). I couldn't quite figure out where the journal filename is set -- there's no db-journal in the code so the name setting appears magjic to me. Since you've got complete control over the