Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-09 Thread Jim Nasby
On May 9, 2007, at 11:01 AM, Magnus Hagander wrote: On Wed, May 09, 2007 at 10:55:12AM -0500, Jim Nasby wrote: On May 8, 2007, at 2:24 PM, Magnus Hagander wrote: Speaking of which, it might be interesting to actually show these values in the stats collector. I was thinking three cols for each d

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-09 Thread Heikki Linnakangas
Magnus Hagander wrote: Gregory Stark wrote: "Magnus Hagander" <[EMAIL PROTECTED]> writes: What 3 columns? In-memory sorts, on-disk sorts, and on-disk size? (Sum of how much spilled to disk). I was thinking in-mem sorts, on-disk sorts, limited-by-LIMIT sorts (that would be the new feature..)

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-09 Thread Magnus Hagander
Gregory Stark wrote: > "Magnus Hagander" <[EMAIL PROTECTED]> writes: > >>> What 3 columns? In-memory sorts, on-disk sorts, and on-disk size? >>> (Sum of how much spilled to disk). >> I was thinking in-mem sorts, on-disk sorts, limited-by-LIMIT sorts (that >> would be the new feature..) > > Tom'

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-09 Thread Gregory Stark
"Magnus Hagander" <[EMAIL PROTECTED]> writes: >> What 3 columns? In-memory sorts, on-disk sorts, and on-disk size? >> (Sum of how much spilled to disk). > > I was thinking in-mem sorts, on-disk sorts, limited-by-LIMIT sorts (that > would be the new feature..) Tom's code distinguished in-memory

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-09 Thread Magnus Hagander
On Wed, May 09, 2007 at 10:55:12AM -0500, Jim Nasby wrote: > On May 8, 2007, at 2:24 PM, Magnus Hagander wrote: > >Speaking of which, it might be interesting to actually show these > >values > >in the stats collector. I was thinking three cols for each database > >(probably the best level?) that

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-09 Thread Jim Nasby
On May 8, 2007, at 2:24 PM, Magnus Hagander wrote: Speaking of which, it might be interesting to actually show these values in the stats collector. I was thinking three cols for each database (probably the best level?) that counts each of those three counters. If you have a lot of sorts (perc

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-08 Thread Magnus Hagander
Magnus Hagander wrote: > Alvaro Herrera wrote: >> Jim Nasby wrote: >> >>> If we really want to make the logfile the approved method for >>> monitoring performance, then why do we have the stats infrastructure >>> at all? It could all be replaced with logging output and pgfouine. >> First we'd h

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-07 Thread Magnus Hagander
Alvaro Herrera wrote: > Jim Nasby wrote: > >> If we really want to make the logfile the approved method for >> monitoring performance, then why do we have the stats infrastructure >> at all? It could all be replaced with logging output and pgfouine. > > First we'd have to fix the usability pr

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-06 Thread Alvaro Herrera
Jim Nasby wrote: > If we really want to make the logfile the approved method for > monitoring performance, then why do we have the stats infrastructure > at all? It could all be replaced with logging output and pgfouine. First we'd have to fix the usability problem of our redirect_stderr stuf

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-06 Thread Tom Lane
Jim Nasby <[EMAIL PROTECTED]> writes: > There's several problems with that. First, trace_sort isn't > documented (or at least it's not in postgresql.conf), so most folks > don't know it exists. Second, in order to see it's output you have to > drop log_min_messages to debug. That results in a

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-06 Thread Jim Nasby
On May 6, 2007, at 9:32 AM, Tom Lane wrote: Jim Nasby <[EMAIL PROTECTED]> writes: There's several problems with that. First, trace_sort isn't documented (or at least it's not in postgresql.conf), so most folks don't know it exists. Second, in order to see it's output you have to drop log_min_mes

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-05 Thread Jim Nasby
On May 4, 2007, at 7:49 PM, Tom Lane wrote: Jim Nasby <[EMAIL PROTECTED]> writes: On a related note, it would also be *really* nice if we kept stats on how many sorts or hashes had spilled to disk, perhaps along with how much had spilled. Right now the only way to monitor that in a production sy

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Tom Lane
Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes: > Guillaume Smet wrote: >> On 5/4/07, Tom Lane <[EMAIL PROTECTED]> wrote: >>> Sort Method: disk Memory: 1000KB Disk: 18482KB >> >> +1 for this one. > I like that one too ... OK, in the event it looks like one of these four messages: "S

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Stefan Kaltenbrunner
Guillaume Smet wrote: > On 5/4/07, Tom Lane <[EMAIL PROTECTED]> wrote: >> Sort Method: disk Memory: 1000KB Disk: 18482KB > > +1 for this one. I like that one too ... Stefan ---(end of broadcast)--- TIP 1: if posting/reading through Use

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Tom Lane
"Guillaume Smet" <[EMAIL PROTECTED]> writes: > Is it possible to have something like Sort (disk|top-N|memory) instead > of Sort? That would be sane if the decision were fixed at plan time, but it isn't. What do you think of the add-a-line approach? regards, tom lane -

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Guillaume Smet
On 5/4/07, Tom Lane <[EMAIL PROTECTED]> wrote: Sort Method: disk Memory: 1000KB Disk: 18482KB +1 for this one. -- Guillaume ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Guillaume Smet
On 5/4/07, Tom Lane <[EMAIL PROTECTED]> wrote: No, you can turn on trace_sort and track it from watching the log. If pgfouine hasn't got something for that already, I'd be surprised. Well, it hasn't. I never used trace_sort so i didn't think of implementing something to use it. I'll take a look

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Guillaume Smet
On 5/4/07, Tom Lane <[EMAIL PROTECTED]> wrote: -> Sort (cost=840.19..865.19 rows=1 width=244) (actual time=140.492..140.880 rows=100 loops=1 method=top-N) Sort K

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Tom Lane
Jim Nasby <[EMAIL PROTECTED]> writes: > If the method is disk it would be nice to know how much spilled to > disk. That would tell you if it would be worth increasing work_mem, > and by how much. Well, a more radical proposal is to add a whole 'nother line to the output, which would give us ro

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Josh Berkus
> > What do you think the output should look like? The first thought that > > comes to mind is to add "method=memory" (or disk or top-N) to the > > "actual" annotation: Having the "disk" and "memory" would be really useful too. -- Josh Berkus PostgreSQL @ Sun San Francisco -

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Tom Lane
Jim Nasby <[EMAIL PROTECTED]> writes: > On a related note, it would also be *really* nice if we kept stats on > how many sorts or hashes had spilled to disk, perhaps along with how > much had spilled. Right now the only way to monitor that in a > production system is to setup a cron job to wa

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Jim Nasby
On May 4, 2007, at 7:08 PM, Tom Lane wrote: What do you think the output should look like? The first thought that comes to mind is to add "method=memory" (or disk or top-N) to the "actual" annotation: regression=# explain analyze select * from tenk1 order by fivethous limit 100;

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Magnus Hagander
> >> It's not exactly easy to do, because (a) none of this information > >> is exposed outside tuplesort.c, and (b) the tuplesortstate object > >> is probably gone by the time EXPLAIN ANALYZE runs, anyway. > > > Hmm. Ok. Don't know enough about those parts of the code to comment on > > that, but I

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about "top N" sorting, in which only the first

2007-05-04 Thread Tom Lane
Magnus Hagander <[EMAIL PROTECTED]> writes: > On Fri, May 04, 2007 at 12:38:18PM -0400, Tom Lane wrote: >> Magnus Hagander <[EMAIL PROTECTED]> writes: >>> Could we show it in EXPLAIN ANALYZE somehow? I'm thinking it would be good >>> to see at runtime (for example as a hint that if you put in a bit