Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-09 Thread Jim Nasby
On May 8, 2007, at 2:24 PM, Magnus Hagander wrote: Speaking of which, it might be interesting to actually show these values in the stats collector. I was thinking three cols for each database (probably the best level?) that counts each of those three counters. If you have a lot of sorts

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-09 Thread Magnus Hagander
On Wed, May 09, 2007 at 10:55:12AM -0500, Jim Nasby wrote: On May 8, 2007, at 2:24 PM, Magnus Hagander wrote: Speaking of which, it might be interesting to actually show these values in the stats collector. I was thinking three cols for each database (probably the best level?) that counts

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-09 Thread Gregory Stark
Magnus Hagander [EMAIL PROTECTED] writes: What 3 columns? In-memory sorts, on-disk sorts, and on-disk size? (Sum of how much spilled to disk). I was thinking in-mem sorts, on-disk sorts, limited-by-LIMIT sorts (that would be the new feature..) Tom's code distinguished in-memory, top-N,

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-09 Thread Magnus Hagander
Gregory Stark wrote: Magnus Hagander [EMAIL PROTECTED] writes: What 3 columns? In-memory sorts, on-disk sorts, and on-disk size? (Sum of how much spilled to disk). I was thinking in-mem sorts, on-disk sorts, limited-by-LIMIT sorts (that would be the new feature..) Tom's code

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-09 Thread Heikki Linnakangas
Magnus Hagander wrote: Gregory Stark wrote: Magnus Hagander [EMAIL PROTECTED] writes: What 3 columns? In-memory sorts, on-disk sorts, and on-disk size? (Sum of how much spilled to disk). I was thinking in-mem sorts, on-disk sorts, limited-by-LIMIT sorts (that would be the new feature..)

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-09 Thread Jim Nasby
On May 9, 2007, at 11:01 AM, Magnus Hagander wrote: On Wed, May 09, 2007 at 10:55:12AM -0500, Jim Nasby wrote: On May 8, 2007, at 2:24 PM, Magnus Hagander wrote: Speaking of which, it might be interesting to actually show these values in the stats collector. I was thinking three cols for each

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-08 Thread Magnus Hagander
Magnus Hagander wrote: Alvaro Herrera wrote: Jim Nasby wrote: If we really want to make the logfile the approved method for monitoring performance, then why do we have the stats infrastructure at all? It could all be replaced with logging output and pgfouine. First we'd have to fix the

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-07 Thread Magnus Hagander
Alvaro Herrera wrote: Jim Nasby wrote: If we really want to make the logfile the approved method for monitoring performance, then why do we have the stats infrastructure at all? It could all be replaced with logging output and pgfouine. First we'd have to fix the usability problem of

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-06 Thread Jim Nasby
On May 6, 2007, at 9:32 AM, Tom Lane wrote: Jim Nasby [EMAIL PROTECTED] writes: There's several problems with that. First, trace_sort isn't documented (or at least it's not in postgresql.conf), so most folks don't know it exists. Second, in order to see it's output you have to drop

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-06 Thread Tom Lane
Jim Nasby [EMAIL PROTECTED] writes: There's several problems with that. First, trace_sort isn't documented (or at least it's not in postgresql.conf), so most folks don't know it exists. Second, in order to see it's output you have to drop log_min_messages to debug. That results in a huge

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-06 Thread Alvaro Herrera
Jim Nasby wrote: If we really want to make the logfile the approved method for monitoring performance, then why do we have the stats infrastructure at all? It could all be replaced with logging output and pgfouine. First we'd have to fix the usability problem of our redirect_stderr stuff

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-05 Thread Jim Nasby
On May 4, 2007, at 7:49 PM, Tom Lane wrote: Jim Nasby [EMAIL PROTECTED] writes: On a related note, it would also be *really* nice if we kept stats on how many sorts or hashes had spilled to disk, perhaps along with how much had spilled. Right now the only way to monitor that in a production

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: On Fri, May 04, 2007 at 12:38:18PM -0400, Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Could we show it in EXPLAIN ANALYZE somehow? I'm thinking it would be good to see at runtime (for example as a hint that if you put in a bit more

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Magnus Hagander
It's not exactly easy to do, because (a) none of this information is exposed outside tuplesort.c, and (b) the tuplesortstate object is probably gone by the time EXPLAIN ANALYZE runs, anyway. Hmm. Ok. Don't know enough about those parts of the code to comment on that, but I'll certainly

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Jim Nasby
On May 4, 2007, at 7:08 PM, Tom Lane wrote: What do you think the output should look like? The first thought that comes to mind is to add method=memory (or disk or top-N) to the actual annotation: regression=# explain analyze select * from tenk1 order by fivethous limit 100;

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Tom Lane
Jim Nasby [EMAIL PROTECTED] writes: On a related note, it would also be *really* nice if we kept stats on how many sorts or hashes had spilled to disk, perhaps along with how much had spilled. Right now the only way to monitor that in a production system is to setup a cron job to watch

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Josh Berkus
What do you think the output should look like? The first thought that comes to mind is to add method=memory (or disk or top-N) to the actual annotation: Having the disk and memory would be really useful too. -- Josh Berkus PostgreSQL @ Sun San Francisco

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Tom Lane
Jim Nasby [EMAIL PROTECTED] writes: If the method is disk it would be nice to know how much spilled to disk. That would tell you if it would be worth increasing work_mem, and by how much. Well, a more radical proposal is to add a whole 'nother line to the output, which would give us room

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Guillaume Smet
On 5/4/07, Tom Lane [EMAIL PROTECTED] wrote: - Sort (cost=840.19..865.19 rows=1 width=244) (actual time=140.492..140.880 rows=100 loops=1 method=top-N) Sort

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Guillaume Smet
On 5/4/07, Tom Lane [EMAIL PROTECTED] wrote: No, you can turn on trace_sort and track it from watching the log. If pgfouine hasn't got something for that already, I'd be surprised. Well, it hasn't. I never used trace_sort so i didn't think of implementing something to use it. I'll take a look

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Guillaume Smet
On 5/4/07, Tom Lane [EMAIL PROTECTED] wrote: Sort Method: disk Memory: 1000KB Disk: 18482KB +1 for this one. -- Guillaume ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Tom Lane
Guillaume Smet [EMAIL PROTECTED] writes: Is it possible to have something like Sort (disk|top-N|memory) instead of Sort? That would be sane if the decision were fixed at plan time, but it isn't. What do you think of the add-a-line approach? regards, tom lane

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Stefan Kaltenbrunner
Guillaume Smet wrote: On 5/4/07, Tom Lane [EMAIL PROTECTED] wrote: Sort Method: disk Memory: 1000KB Disk: 18482KB +1 for this one. I like that one too ... Stefan ---(end of broadcast)--- TIP 1: if posting/reading through Usenet,

Re: [HACKERS] [COMMITTERS] pgsql: Teach tuplesort.c about top N sorting, in which only the first

2007-05-04 Thread Tom Lane
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Guillaume Smet wrote: On 5/4/07, Tom Lane [EMAIL PROTECTED] wrote: Sort Method: disk Memory: 1000KB Disk: 18482KB +1 for this one. I like that one too ... OK, in the event it looks like one of these four messages: Sort Method: