Re: [PATCHES] Remove all trace of EXPLAIN EXECUTE

2005-08-10 Thread Simon Riggs
On Tue, 2005-08-09 at 18:50 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > There once was a rumour of a EXPLAIN EXECUTE command. > > This minor patch removes all trace of that, but without disturbing other > > valid occurrences of the EXECUTE command, which still lives on. >

[PATCHES] 5 new entries for FAQ

2005-08-10 Thread Martijn van Oosterhout
[Sorry for the duplicate post, sent to wrong list first time] Hi, After going through pgsql-general a bit I figured there were a few important questions missing from the FAQ, so I wrote some. Comments welcome. I can write more, if people can suggest things to write about. I was thinking somethin

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Simon Riggs
On Tue, 2005-08-09 at 21:48 -0700, Luke Lonergan wrote: > The key thing that is missing is the lack of micro-parallelism in the > character processing in this version. By "inverting the loop", or putting > the characters into a buffer on the outside, then doing fast character > scanning inside wi

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Luke Lonergan
Simon, > That part of the code was specifically written to take advantage of > processing pipelines in the hardware, not because the actual theoretical > algorithm for that approach was itself faster. Yup, good point. > Nobody's said what compiler/hardware they have been using, so since both >

Re: [PATCHES] 5 new entries for FAQ

2005-08-10 Thread Matt Miller
> If you're really desparate, you can increase the pagesize a bit (to Should be "desperate," not "desparate." ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PATCHES] 5 new entries for FAQ

2005-08-10 Thread Robert Treat
On Wed, 2005-08-10 at 04:04, Martijn van Oosterhout wrote: > [Sorry for the duplicate post, sent to wrong list first time] > > Hi, > > After going through pgsql-general a bit I figured there were a few > important questions missing from the FAQ, so I wrote some. > > Comments welcome. I can write

Re: [PATCHES] 5 new entries for FAQ

2005-08-10 Thread Alvaro Herrera
On Wed, Aug 10, 2005 at 10:04:23AM +0200, Martijn van Oosterhout wrote: > Comments welcome. I can write more, if people can suggest things to > write about. I was thinking something about collation and locales but > I'm sure sure I understand them myself. I'd really love to see a Q&A for encoding

Re: [PATCHES] Fix oversight in pts_error_callback()

2005-08-10 Thread Tom Lane
"Qingqing Zhou" <[EMAIL PROTECTED]> writes: > "Tom Lane" <[EMAIL PROTECTED]> writes >> Please exhibit a case in which you feel this is needed. > Suppose I want to print a debug info in parseTypeString() like this: > + elog(DEBUG1, "parse type %s", buf.data); > raw_parsetree_list = raw_par

Re: [PATCHES] 5 new entries for FAQ

2005-08-10 Thread Martijn van Oosterhout
On Wed, Aug 10, 2005 at 11:17:40AM -0400, Alvaro Herrera wrote: > On Wed, Aug 10, 2005 at 10:04:23AM +0200, Martijn van Oosterhout wrote: > > > Comments welcome. I can write more, if people can suggest things to > > write about. I was thinking something about collation and locales but > > I'm sure

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Nobody's said what compiler/hardware they have been using, so since both > Alon and Tom say their character finding logic is faster, it is likely > to be down to that? Name your platforms gentlemen, please. I tested on HPPA with gcc 2.95.3 and on a Pentium

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Tom Lane
"Luke Lonergan" <[EMAIL PROTECTED]> writes: > Yes, I think one thing we've learned is that there are important parts > of the code, those that are in the data path (COPY, sort, spill to > disk, etc) that are in dire need of optimization. For instance, the > fgetc() pattern should be banned everywh

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Alvaro Herrera
On Wed, Aug 10, 2005 at 09:16:08AM -0700, Luke Lonergan wrote: > On 8/10/05 8:37 AM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > > > Luke, I dislike whacking people upside the head, but this discussion > > seems to presume that raw speed on Intel platforms is the only thing > > that matters. We have

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Bruce Momjian
Luke Lonergan wrote: > Tom, > > On 8/10/05 8:37 AM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > > > Luke, I dislike whacking people upside the head, but this discussion > > seems to presume that raw speed on Intel platforms is the only thing > > that matters. We have a few other concerns. Portabili

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Joshua D. Drake
Also, as we proved the last time the correctness argument was thrown in, we can fix the bugs and still make it a lot faster - and I would stick to that whether it's a PA-RISC, DEC Alpha, Intel or AMD or event Ultra Sparc. Luke this comment doesn't work. Do you have a test case that shows that o

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Bruce Momjian
Alvaro Herrera wrote: > Another question that comes to mind is: have you tried another compiler? > I see you are all using GCC at most 3.4; maybe the new optimizing > infrastructure in GCC 4.1 means you can have most of the speedup without > uglifying the code. What about Intel's compiler? Enterp

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Luke Lonergan
Alvaro, On 8/10/05 9:46 AM, "Alvaro Herrera" <[EMAIL PROTECTED]> wrote: > AFAIR he never claimed otherwise ... his point was that to gain that > additional speedup, the code has to be made considerable "worse" (in > maintenability terms.) Have you (or Alon) tried to port the rest of the > speed

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Alvaro Herrera
On Wed, Aug 10, 2005 at 12:57:18PM -0400, Bruce Momjian wrote: > Alvaro Herrera wrote: > > Another question that comes to mind is: have you tried another compiler? > > I see you are all using GCC at most 3.4; maybe the new optimizing > > infrastructure in GCC 4.1 means you can have most of the spee

Re: [PATCHES] COPY FROM performance improvements

2005-08-10 Thread Luke Lonergan
Tom, On 8/10/05 8:37 AM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > Luke, I dislike whacking people upside the head, but this discussion > seems to presume that raw speed on Intel platforms is the only thing > that matters. We have a few other concerns. Portability, readability, > maintainability,

Re: [PATCHES] [HACKERS] Autovacuum loose ends

2005-08-10 Thread Alvaro Herrera
Updated this patch again: > > - vacuum_cost_delay and vacuum_cost_limit can be set per table, as well > > as globally with autovacuum_vacuum_cost_{limit,delay} > > > > - pgstat is reset if recovery is required > > > > - pgstat reset at postmaster start is disabled by default > > - Xid-wraparo

Re: [PATCHES] (was COPY FROM) performance improvements

2005-08-10 Thread Simon Riggs
On Wed, 2005-08-10 at 11:37 -0400, Tom Lane wrote: > "Luke Lonergan" <[EMAIL PROTECTED]> writes: > > Yes, I think one thing we've learned is that there are important parts > > of the code, those that are in the data path (COPY, sort, spill to > > disk, etc) that are in dire need of optimization. F

Re: [PATCHES] (was COPY FROM) performance improvements

2005-08-10 Thread Bruce Momjian
Simon Riggs wrote: > As you know, I have raised the need for specific hardware tuning in > certain critical areas on a number of occasions. I very much respect the > need for all of the other aspects of code quality mentioned. > > Pipeline parallelism is a feature of all modern CPUs since the Pent

[PATCHES] Proposed patch for preventing OID collisions

2005-08-10 Thread Tom Lane
Attached is a reasonably complete working-out of my earlier proposal to prevent OID collisions by probing catalogs' OID indexes and advancing to the next OID whenever there is a duplicate. The code is driven by the existence of a unique index, and so it is possible for users to take advantage of t

Re: [PATCHES] Fix oversight in pts_error_callback()

2005-08-10 Thread Qingqing Zhou
"Tom Lane" <[EMAIL PROTECTED]> writes: > > That's a contrived example (and if I believed it, I would think that the > right answer is to emit no errcontext if the elevel is less than ERROR). > Yes, I've thought about ignore errcontext by considering elevel. But I scratched the source code for oth

[PATCHES] Bug in canonicalize_path()

2005-08-10 Thread Bruce Momjian
I found that in port/path.c::canonicalize_path, that if the path was supplied as "/usr/local/bin/../.." we would return /usr/local/bin. The problem is then when we saw a trailing ".." we stripped it off and the previous directory, but we never checked if the previous directory was itself "..". Pa

Re: [PATCHES] [HACKERS] For review: Server instrumentation patch

2005-08-10 Thread Bruce Momjian
Dave Page wrote: > > The list isn't complete. pgadmin uses these three functions > > for logfile > > tracking: > > > > - pg_logdir_ls to list logfiles > > - pg_file_length to check for changes of the current logfile > > - pg_file_read to retrieve a logfile > > Yes you're right, I didn't check t

[PATCHES] remove BufferBlockPointers for speed and space

2005-08-10 Thread Qingqing Zhou
It is said that the BufferBlockPointers is used to speedup the BufferGetBlock() macro. I compared three ways of getting block pointers. I.e., existing method (arrary method), calculating block pointer by adding base addr and offset*blockid method (mul method) and optimizing mul method by using bit

Re: [PATCHES] remove BufferBlockPointers for speed and space

2005-08-10 Thread Gavin Sherry
On Thu, 11 Aug 2005, Qingqing Zhou wrote: > It is said that the BufferBlockPointers is used to speedup the > BufferGetBlock() macro. I compared three ways of getting block pointers. > I.e., existing method (arrary method), calculating block pointer by adding > base addr and offset*blockid method (

Re: [PATCHES] remove BufferBlockPointers for speed and space

2005-08-10 Thread Tom Lane
Gavin Sherry <[EMAIL PROTECTED]> writes: > On Thu, 11 Aug 2005, Qingqing Zhou wrote: >> It is said that the BufferBlockPointers is used to speedup the >> BufferGetBlock() macro. I compared three ways of getting block pointers. > Do you have results for more recent gcc releases? Or more than one h

Re: [PATCHES] remove BufferBlockPointers for speed and space

2005-08-10 Thread Gavin Sherry
On Thu, 11 Aug 2005, Tom Lane wrote: > Gavin Sherry <[EMAIL PROTECTED]> writes: > > On Thu, 11 Aug 2005, Qingqing Zhou wrote: > >> It is said that the BufferBlockPointers is used to speedup the > >> BufferGetBlock() macro. I compared three ways of getting block pointers. > > > Do you have results

Re: [PATCHES] remove BufferBlockPointers for speed and space

2005-08-10 Thread Qingqing Zhou
"Tom Lane" <[EMAIL PROTECTED]> writes: > > Also, I would like to see the actual test code. I wonder whether what > you measured is the ability of the compiler to optimize references to > successive elements of an array inside a loop; that has little or > nothing to do with the typical usage of Bu

Re: [PATCHES] remove BufferBlockPointers for speed and space

2005-08-10 Thread Mark Kirkwood
FreeBSD 5.4 RELEASE gcc 3.4.2 on Intel dual PIII 1Ghz [postgres:~/develop/c/testbuf]$ gcc -Wall -o testbuf testbuf.c [postgres:~/develop/c/testbuf]$ ./testbuf duration round 1 of array method: 1.737 ms duration round 2 of array method: 1.676 ms duration round 3 of array method: 1.527 ms duration