Re: [sqlite] serious performance problems with indexes

2006-11-03 Thread Christian Smith
Peter De Rijk uttered: --On Friday 27 October 2006 15:31, [EMAIL PROTECTED] wrote: When a table is indexed, INSERT performance is logorithmic in the number of rows in the table and linear in the number of indices. This is because entries have to be inserted into the index in sorted

Re: [sqlite] serious performance problems with indexes

2006-10-31 Thread Scott Hess
On 10/30/06, Joe Wilson <[EMAIL PROTECTED]> wrote: If you have only one index, then pre-sorting your large datasets prior to inserting with the default sqlite cache will yield twice as good insert performance as not pre-sorting your data and using a huge cache. This stands to reason since you're

Re: [sqlite] serious performance problems with indexes

2006-10-30 Thread Joe Wilson
2006 1:00:25 PM Subject: Re: [sqlite] serious performance problems with indexes Presorting is unfortunately not always possible, e.g. when there are several indexes (with different order). But increasing the cache seems a good solution for this anyway (and is less cumbersome than presorting or s

Re: [sqlite] serious performance problems with indexes

2006-10-30 Thread Peter De Rijk
On Sunday 29 October 2006 18:47, Joe Wilson wrote: > The last test that simulated pre-sorting 2 million rows > in index order prior to insert may show optimal insert speed > since it's only appending pages to the table and the index, > but it may not be realistic, since you may not have sufficient

Re: [sqlite] serious performance problems with indexes

2006-10-29 Thread Joe Wilson
fd puts $o $num\t[join [runtest test.sql] \t] flush $o } close $o - Original Message From: Joe Wilson <[EMAIL PROTECTED]> To: sqlite-users@sqlite.org Sent: Saturday, October 28, 2006 1:08:20 PM Subject: Re: [sqlite] serious performance problems with indexes If you pre-sort the d

Re: [sqlite] serious performance problems with indexes

2006-10-28 Thread Joe Wilson
ct: Re: [sqlite] serious performance problems with indexes I have checked more fine grained (results and code at the end of the mail), and it seems indeed that the extreme growth of time needed for inserts only starts at around 16 rows (on my system). Before that, it also rises, but much muc

Re: [sqlite] serious performance problems with indexes

2006-10-28 Thread Peter De Rijk
--On Friday 27 October 2006 15:31, [EMAIL PROTECTED] wrote: > When a table is indexed, INSERT performance is logorithmic in the > number of rows in the table and linear in the number of indices. > This is because entries have to be inserted into the index in > sorted order (otherwise it

Re: [sqlite] serious performance problems with indexes

2006-10-27 Thread Joe Wilson
: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> To: sqlite-users@sqlite.org Sent: Friday, October 27, 2006 9:31:51 AM Subject: Re: [sqlite] serious performance problems with indexes Peter De Rijk <[EMAIL PROTECTED]> wrote: > I have run into a serious performance problem w

Re: [sqlite] serious performance problems with indexes

2006-10-27 Thread drh
Peter De Rijk <[EMAIL PROTECTED]> wrote: > I have run into a serious performance problem with tables with many rows. > The problem only occurs on tables with an index > The time needed for an insert into a table with an index is dependend on the > number of rows. I have not formally checked, but

Re: [sqlite] serious performance problems with indexes

2006-10-27 Thread Arjen Markus
Peter De Rijk wrote: I have run into a serious performance problem with tables with many rows. The problem only occurs on tables with an index The time needed for an insert into a table with an index is dependend on the number of rows. I have not formally checked, but from my tests it looks

[sqlite] serious performance problems with indexes

2006-10-27 Thread Peter De Rijk
I have run into a serious performance problem with tables with many rows. The problem only occurs on tables with an index The time needed for an insert into a table with an index is dependend on the number of rows. I have not formally checked, but from my tests it looks like an exponential