Re: [PERFORM] slow plan for min/max

2003-09-09 Thread Matt Clark
> > Know what we (OK, I) need? An explicitly non-aggregate max() and min(), > > implemented differently, so they can be optimised. > > Not per se. The way I've been visualizing this is that we add to > pg_aggregate a column named, say, aggsortop, with the definition: ...snip of cunning potentiall

Re: [PERFORM] slow plan for min/max

2003-09-09 Thread Bruno Wolff III
On Tue, Sep 09, 2003 at 14:06:56 -0500, Thomas Swan <[EMAIL PROTECTED]> wrote: > > Would it be possible to rewrite min and max at the parser level into a > select/subselect (clause) condition ( repeat condition ) order by > (clause ) descending/ascending limit 1 and thereby avoiding the > penal

Re: [PERFORM] slow plan for min/max

2003-09-09 Thread Thomas Swan
Tom Lane wrote: >"scott.marlowe" <[EMAIL PROTECTED]> writes: > > >>On Mon, 8 Sep 2003, Neil Conway wrote: >> >> >>>As was pointed out in a thread a couple days ago, MIN/MAX() optimization >>>has absolutely nothing to do with MVCC. It does, however, make >>>optimizing COUNT() more difficult.

Re: [PERFORM] slow plan for min/max

2003-09-09 Thread Bruno Wolff III
On Tue, Sep 09, 2003 at 12:54:04 -0400, Greg Stark <[EMAIL PROTECTED]> wrote: > > So what would it take to implement this for "all" aggregates? Where I think > "all" really just means min(), max(), first(), last(). There can be other aggregates where indexes are helpful. The case of interest is

Re: [PERFORM] slow plan for min/max

2003-09-09 Thread Josh Berkus
Greg, > The only connection to MVCC is that the "obvious" solution doesn't work, > namely storing a cache of the aggregate in the table information. Well, that solution also doesn't work if you use a WHERE condition or JOIN, now does it? > So what would it take to implement this for "all" aggre

Re: [PERFORM] slow plan for min/max

2003-09-09 Thread Greg Stark
The only connection to MVCC is that the "obvious" solution doesn't work, namely storing a cache of the aggregate in the table information. So what would it take to implement this for "all" aggregates? Where I think "all" really just means min(), max(), first(), last(). I think it would mean hav

Re: [PERFORM] slow plan for min/max

2003-09-09 Thread Pailloncy Jean-Gérard
I did not expect so many answers about this question. Thanks. I find by myself the "order by trick" to speed min/max function. Jean-Gérard Pailloncy ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate s

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Tom Lane
"Matt Clark" <[EMAIL PROTECTED]> writes: > Know what we (OK, I) need? An explicitly non-aggregate max() and min(), > implemented differently, so they can be optimised. Not per se. The way I've been visualizing this is that we add to pg_aggregate a column named, say, aggsortop, with the definitio

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Matt Clark
> "Matt Clark" <[EMAIL PROTECTED]> writes: > > Actually, referring down to later parts of this thread, why can't this > > optimisation be performed internally for built-in types? I > understand the > > issue with aggregates over user-defined types, but surely > optimising max() > > for int4, text,

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Tom Lane
"Matt Clark" <[EMAIL PROTECTED]> writes: > Actually, referring down to later parts of this thread, why can't this > optimisation be performed internally for built-in types? I understand the > issue with aggregates over user-defined types, but surely optimising max() > for int4, text, etc is safe a

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Matt Clark
> Actually, referring down to later parts of this thread, why can't this > optimisation be performed internally for built-in types? I understand the > issue with aggregates over user-defined types, but surely optimising max() > for int4, text, etc is safe and easy? Sorry, missed the bit about use

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Matt Clark
>This is a Frequently asked question about something that isn't likely to >change any time soon. You're right, it is in the FAQ, but pretty well buried. It is entirely non-obvious to most people that min() and max() don't/can't use indices. Something so counterintuitive should be explicitly and

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Josh Berkus
Scott, > Not exactly. While max(id) is easily optimized by query replacement, > more complex aggregates will still have perfomance issues that would not > be present in a row locking database. i.e. max((field1/field2)*field3) is > still going to cost more to process, isn't it? Sorry, no.

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Christopher Browne
After takin a swig o' Arrakan spice grog, [EMAIL PROTECTED] ("scott.marlowe") belched out...: > On Mon, 8 Sep 2003, Neil Conway wrote: >> On Mon, 2003-09-08 at 11:56, scott.marlowe wrote: >> > Basically, Postgresql uses an MVCC locking system that makes massively >> > parallel operation possible,

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Greg Stark
"scott.marlowe" <[EMAIL PROTECTED]> writes: > Basically, Postgresql uses an MVCC locking system that makes massively As discussed, uh, a few days ago, this particular problem is not caused by MVCC but by postgres having a general purpose aggregate system and not having special code for handling

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Tom Lane
"scott.marlowe" <[EMAIL PROTECTED]> writes: > On Mon, 8 Sep 2003, Neil Conway wrote: >> As was pointed out in a thread a couple days ago, MIN/MAX() optimization >> has absolutely nothing to do with MVCC. It does, however, make >> optimizing COUNT() more difficult. > Not exactly. While max(id) is

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread scott.marlowe
On Mon, 8 Sep 2003, Neil Conway wrote: > On Mon, 2003-09-08 at 11:56, scott.marlowe wrote: > > Basically, Postgresql uses an MVCC locking system that makes massively > > parallel operation possible, but costs in certain areas, and one of those > > areas is aggregate performance over large sets.

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Christopher Browne
[EMAIL PROTECTED] ("scott.marlowe") writes: > On Sun, 7 Sep 2003, Pailloncy Jean-Gérard wrote: > > Asking a question about why max(id) is so much slower than select id order > by id desc limit 1, Pailloncy said: > >> I ask for the same thing. That's better ! > > This is a Frequently asked questio

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread Neil Conway
On Mon, 2003-09-08 at 11:56, scott.marlowe wrote: > Basically, Postgresql uses an MVCC locking system that makes massively > parallel operation possible, but costs in certain areas, and one of those > areas is aggregate performance over large sets. MVCC makes it very hard > to optimize all but

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread scott.marlowe
On Sun, 7 Sep 2003, Pailloncy Jean-Gérard wrote: Asking a question about why max(id) is so much slower than select id order by id desc limit 1, Pailloncy said: > I ask for the same thing. > That's better ! This is a Frequently asked question about something that isn't likely to change any time

[PERFORM] slow plan for min/max

2003-09-07 Thread Pailloncy Jean-Gérard
I have: psql (PostgreSQL) 7.3.2 I do a modification of 'access/index/indexam.c' where I comment: #ifdef NOT_USED if (scan->keys_are_unique && scan->got_tuple) { if (ScanDirectionIsForward(direction)) { if (scan->unique_tuple_po