Re: [PERFORM] Message queue table - strange performance drop with changing limit size.

2010-01-03 Thread Greg Smith
Jesper Krogh wrote: So what I see is that "top 10" takes < 1ms, top 50 takes over 500 times more, and top 1000 only 1.5 times more than top 50. What can the reason be for the huge drop between limit 10 and limit 50 be? Normally this means you're hitting much higher performing cached behavio

Re: [PERFORM] Message queue table - strange performance drop with changing limit size.

2010-01-02 Thread Dimitri Fontaine
Jesper Krogh writes: > I have a "message queue" table, that contains in the order of 1-10m > "messages". It is implemented using TheSchwartz: > http://search.cpan.org/~bradfitz/TheSchwartz-1.07/lib/TheSchwartz.pm One way to approach queueing efficiently with PostgreSQL is to rely on PGQ. New upco

Re: [PERFORM] Message queue table - strange performance drop with changing limit size.

2010-01-01 Thread Jesper Krogh
Greg Williamson wrote: > Jesper -- > > I apologize for top-quoting -- a challenged reader. > > This doesn't directly address your question, but I can't help but > notice that the estimates for rows is _wildly_ off the actual number > in each and every query. How often / recently have you run ANAL

Re: [PERFORM] Message queue table - strange performance drop with changing limit size.

2010-01-01 Thread Greg Williamson
? Are the timing results consistent over several runs ? It is possible that caching effects are entering into the time results. Greg Williamson - Original Message From: Jesper Krogh To: pgsql-performance@postgresql.org Sent: Fri, January 1, 2010 3:48:43 AM Subject: [PERFORM] Message q

[PERFORM] Message queue table - strange performance drop with changing limit size.

2010-01-01 Thread Jesper Krogh
Hi. I have a "message queue" table, that contains in the order of 1-10m "messages". It is implemented using TheSchwartz: http://search.cpan.org/~bradfitz/TheSchwartz-1.07/lib/TheSchwartz.pm So when a "worker" picks the next job it goes into the "job" table an select the top X highest priority mes

Re: [PERFORM] Message queue table..

2008-04-18 Thread Chris Browne
[EMAIL PROTECTED] (Jesper Krogh) writes: > I have this "message queue" table.. currently with 8m+ > records. Picking the top priority messages seem to take quite > long.. it is just a matter of searching the index.. (just as explain > analyze tells me it does). > > Can anyone digest further optimiz

Re: [PERFORM] Message queue table..

2008-04-18 Thread Tom Lane
Jesper Krogh <[EMAIL PROTECTED]> writes: > I have this "message queue" table.. currently with 8m+ records. Picking > the top priority messages seem to take quite long.. it is just a matter > of searching the index.. (just as explain analyze tells me it does). > Limit (cost=0.00..0.09 rows=1 w

Re: [PERFORM] Message queue table..

2008-04-18 Thread Jesper Krogh
Craig Ringer wrote: Jesper Krogh wrote: Hi. I have this "message queue" table.. currently with 8m+ records. Picking the top priority messages seem to take quite long.. it is just a matter of searching the index.. (just as explain analyze tells me it does). Can anyone digest further optimi

Re: [PERFORM] Message queue table..

2008-04-18 Thread Craig Ringer
Jesper Krogh wrote: Hi. I have this "message queue" table.. currently with 8m+ records. Picking the top priority messages seem to take quite long.. it is just a matter of searching the index.. (just as explain analyze tells me it does). Can anyone digest further optimizations out of this ou

[PERFORM] Message queue table..

2008-04-18 Thread Jesper Krogh
Hi. I have this "message queue" table.. currently with 8m+ records. Picking the top priority messages seem to take quite long.. it is just a matter of searching the index.. (just as explain analyze tells me it does). Can anyone digest further optimizations out of this output? (All records h