Re: [sqlite] Curious performance issue with large number of inserts

2007-05-08 Thread Tim Bradshaw
On 5 May 2007, at 17:11, Joe Wilson wrote: What timings do you get when you run the perl script in my last email? Your script inevitably was fine. What seems to cause the trouble is something nasty in the pathnames. I've modified my perl script to sanitize them by removing any non-ASCII

Re: [sqlite] Curious performance issue with large number of inserts

2007-05-05 Thread Tim Bradshaw
On 5 May 2007, at 17:11, Joe Wilson wrote: What timings do you get when you run the perl script in my last email? I'll try it Tuesday, I'm not back at work till then. - To unsubscribe, send email to [EMAIL PROTECTED]

Re: [sqlite] Curious performance issue with large number of inserts

2007-05-05 Thread Joe Wilson
--- Tim Bradshaw <[EMAIL PROTECTED]> wrote: > > I can't reproduce your problem. I can insert 16M records into your > > table > > schema in 25 minutes on a 5 year old Windows machine. The sqlite3 > > process > > had peak RAM usage of less than 20M. > > Rats, I suspect it must be some

Re: [sqlite] Curious performance issue with large number of inserts

2007-05-05 Thread Tim Bradshaw
On 5 May 2007, at 15:59, Joe Wilson wrote: Ignore the idea above - an insert of NULL or an incrementing integer for the INTEGER PRIMARY KEY yields the same timings. I might try taking it out anyway - I don't really need it other than to be able to say "look at row n" to someone. I did

Re: [sqlite] Curious performance issue with large number of inserts

2007-05-05 Thread Joe Wilson
--- Joe Wilson <[EMAIL PROTECTED]> wrote: > > create table filemap(id integer primary key, > >uid integer, gid integer, mtime integer, > >vol integer, > >path varchar(1024)); > > > > It has no indices built yet. > > > > I'm adding quite a lot of records to it using a perl script

Re: [sqlite] Curious performance issue with large number of inserts

2007-05-05 Thread Joe Wilson
> create table filemap(id integer primary key, >uid integer, gid integer, mtime integer, >vol integer, >path varchar(1024)); > > It has no indices built yet. > > I'm adding quite a lot of records to it using a perl script which > generates SQL like this: > > begin; > insert into

[sqlite] Curious performance issue with large number of inserts

2007-05-04 Thread Tim Bradshaw
Apologies if I should have found an answer to this by searching or being less naive about SQL! I'm using 3.3.17 built by me with gcc (3.4.3? can't check just now as machine is at work - it's the one Sun ship anyway) on a SPARC Solaris 10u3 box. I don't have any significant experience of