[PERFORM] Index creation time and distribution

2008-05-22 Thread Guillaume Smet
Hi -performance, I experienced this morning a performance problem when we imported a dump in a 8.1 database. The table is 5 millions rows large and when the dump creates an index on a specific text column called clazz it takes 27 minutes while on the other columns, it only takes a couple of secon

Re: [PERFORM] Index creation time and distribution

2008-05-22 Thread Tom Lane
"Guillaume Smet" <[EMAIL PROTECTED]> writes: > I experienced this morning a performance problem when we imported a > dump in a 8.1 database. > The table is 5 millions rows large and when the dump creates an index > on a specific text column called clazz it takes 27 minutes while on > the other colu

Re: [PERFORM] Index creation time and distribution

2008-05-22 Thread Guillaume Smet
On Thu, May 22, 2008 at 3:14 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > Do you have maintenance_work_mem set large enough that the index > creation sort is done in-memory? 8.1 depends on the platform's qsort > and a lot of them are kinda pessimal for input like this. FWIW, it's a 32 bits CentOS 4.

Re: [PERFORM] Index creation time and distribution

2008-05-22 Thread Matthew Wakeling
On Thu, 22 May 2008, Tom Lane wrote: Do you have maintenance_work_mem set large enough that the index creation sort is done in-memory? 8.1 depends on the platform's qsort and a lot of them are kinda pessimal for input like this. Looking at the fact that other indexes on the same table are crea

Re: [PERFORM] Index creation time and distribution

2008-05-22 Thread Scott Marlowe
On Thu, May 22, 2008 at 6:32 AM, Guillaume Smet <[EMAIL PROTECTED]> wrote: > Hi -performance, > > > LOG: duration: 1636301.317 ms statement: CREATE INDEX > index_journal_clazz ON journal USING btree (clazz); > LOG: duration: 20613.009 ms statement: CREATE INDEX > index_journal_date ON journal U

Re: [PERFORM] Index creation time and distribution

2008-05-22 Thread Guillaume Smet
On Thu, May 22, 2008 at 6:50 PM, Scott Marlowe <[EMAIL PROTECTED]> wrote: > Just curious, what happens if you create the date index first, then > the clazz one? It's not due to any cache effect if it's your question. It's mostly CPU time and changing the order doesn't change the behaviour. I'll m

Re: [PERFORM] Index creation time and distribution

2008-05-22 Thread Tom Lane
"Guillaume Smet" <[EMAIL PROTECTED]> writes: > On Thu, May 22, 2008 at 3:14 PM, Tom Lane <[EMAIL PROTECTED]> wrote: >> Do you have maintenance_work_mem set large enough that the index >> creation sort is done in-memory? 8.1 depends on the platform's qsort >> and a lot of them are kinda pessimal fo

Re: [PERFORM] Index creation time and distribution

2008-05-22 Thread Guillaume Smet
On Thu, May 22, 2008 at 9:18 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > Ermm .. this is in fact mostly broken in 8.3.0 and 8.3.1. If you don't > want to wait for 8.3.2, you need this patch: > http://archives.postgresql.org/pgsql-committers/2008-03/msg00566.php That's what I had in mind. We have to

Re: [PERFORM] "Big O" notation for postgres?

2008-05-22 Thread Gregory Stark
"Richard Huxton" <[EMAIL PROTECTED]> writes: > Jonah H. Harris wrote: >> On Wed, May 21, 2008 at 10:10 AM, H. Hall <[EMAIL PROTECTED]> wrote: >>> Does anyone know if there is a source that provides "Big O" notation for >>> postgres's aggregate functions and operations? For example is count(*) = >

Re: [PERFORM] I/O on select count(*)

2008-05-22 Thread Luke Lonergan
Hi Hannu, Interesting suggestion on the partial index! I'll find out if we can extract our code that did the work. It was simple but scattered in a few routines. In concept it worked like this: 1 - Ignore if hint bits are unset, use them if set. This affects heapam and vacuum I think. 2 - i