Re: [HACKERS] Automatically setting work_mem

2006-03-16 Thread Qingqing Zhou
"Simon Riggs" <[EMAIL PROTECTED]> wrote > > We would only look at dynamically changing work_mem in those few > restricted cases where we track that against the work_mem limit. If we > hit that limit, we would make a request to the central pool: "Can I be > allotted another 2MB please?" (etc). The

Re: [HACKERS] qsort, once again

2006-03-16 Thread Dann Corbit
Well, my point was that it is a snap to implement and test. It will be better, worse, or the same. I agree that Bentley is a bloody genius. > -Original Message- > From: Tom Lane [mailto:[EMAIL PROTECTED] > Sent: Thursday, March 16, 2006 9:27 PM > To: Dann Corbit > Cc: Jonah H. Harris; pg

Re: [HACKERS] qsort, once again

2006-03-16 Thread Tom Lane
"Dann Corbit" <[EMAIL PROTECTED]> writes: >> So my feeling is we should just remove the swap_cnt code and return >> to the original B&M algorithm. > Even if "hunks" of the input are sorted, the test is a very good idea. Yah know, guys, Bentley and McIlroy are each smarter than any five of us, and

Re: [HACKERS] Bug report form: locale/encoding

2006-03-16 Thread William ZHANG
- Original Message - From: "Andrew Dunstan" <[EMAIL PROTECTED]> To: "William ZHANG" <[EMAIL PROTECTED]> Cc: Sent: Thursday, March 16, 2006 11:04 PM Subject: Re: [HACKERS] Bug report form: locale/encoding > William ZHANG wrote: > >>>Has anyone considered something like "reportbug" in De

Re: [HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Qingqing Zhou
"Simon Riggs" <[EMAIL PROTECTED]> wrote > > I think Tom's right... the OS blocksize is smaller than BLCKSZ, so > reducing the size might help with a very high transaction load when > commits are required very frequently. At checkpoint it sounds like we > might benefit from a large WAL blocksize be

Re: [HACKERS] qsort, once again

2006-03-16 Thread Dann Corbit
> -Original Message- > From: [EMAIL PROTECTED] [mailto:pgsql-hackers- > [EMAIL PROTECTED] On Behalf Of Dann Corbit > Sent: Thursday, March 16, 2006 4:42 PM > To: Tom Lane > Cc: Jonah H. Harris; pgsql-hackers@postgresql.org; Jerry Sievers > Subject: Re: [HACKERS] qsort, once again > > > So

Re: [HACKERS] qsort, once again

2006-03-16 Thread Dann Corbit
> So my feeling is we should just remove the swap_cnt code and return to > the original B&M algorithm. Being much faster than expected for > presorted input doesn't justify being far slower than expected for > other inputs, IMHO. In the context of Postgres I doubt that perfectly > sorted input sh

Re: [HACKERS] qsort, once again

2006-03-16 Thread Tom Lane
Darcy Buskermolen <[EMAIL PROTECTED]> writes: > On Thursday 16 March 2006 12:09, Tom Lane wrote: >> So we still have a problem of software archaeology: who added the >> insertion sort switch to the NetBSD version, and on what grounds? > This is when that particular code was pushed in, as to why ex

Re: [HACKERS] qsort, once again

2006-03-16 Thread Darcy Buskermolen
On Thursday 16 March 2006 12:09, Tom Lane wrote: > "Dann Corbit" <[EMAIL PROTECTED]> writes: > > I sent him a copy > > Thanks. This is really interesting: the switch to insertion sort on > perfect pivot is simply not there in Bentley & McIlroy's paper. So > it was added later, and evidently not

Re: [HACKERS] qsort, once again

2006-03-16 Thread Tom Lane
>> So at least on randomized data, the swap_cnt thing is a serious loser. >> Need to run some tests on special-case inputs though. Anyone have a >> test suite they like? > Here is a distribution maker that will create some torture tests for > sorting programs. I fleshed out the sort tester that

Re: [HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Mark Wong
On Thu, 16 Mar 2006 20:51:54 + Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2006-03-16 at 12:22 -0800, Mark Wong wrote: > > > I was hoping that in the case where 2 or more data blocks are written to > > the log that they could written once within a single larger log block. > > The log bl

Re: [HACKERS] qsort, once again

2006-03-16 Thread Dann Corbit
After you run the program with CREATE_DISTRIBS defined, you will have the following integer distributions of 4 million entries: 03/16/2006 01:34 PM16,000,000 trig.int 03/16/2006 01:34 PM16,000,000 camel.int 03/16/2006 01:34 PM16,000,000 reverse.int 03/16/2006 01:34 PM

Re: [HACKERS] qsort, once again

2006-03-16 Thread Dann Corbit
Actually, if you compile with CREATE_DISTRIBS defined, it does define a main() function and create sample distributions. > -Original Message- > From: [EMAIL PROTECTED] [mailto:pgsql-hackers- > [EMAIL PROTECTED] On Behalf Of Dann Corbit > Sent: Thursday, March 16, 2006 1:28 PM > To: Tom Lan

Re: [HACKERS] qsort, once again

2006-03-16 Thread Dann Corbit
[snip] > So at least on randomized data, the swap_cnt thing is a serious loser. > Need to run some tests on special-case inputs though. Anyone have a > test suite they like? > > regards, tom lane Here is a distribution maker that will create some torture tests for sorting p

Re: [HACKERS] qsort, once again

2006-03-16 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > On 3/16/06, Tom Lane <[EMAIL PROTECTED]> wrote: >> So we still have a problem of software archaeology: who added the >> insertion sort switch to the NetBSD version, and on what grounds? > AFAICS, the insertion sort was added in BSD 4.4-lite and was i

Re: [HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Simon Riggs
On Thu, 2006-03-16 at 12:22 -0800, Mark Wong wrote: > I was hoping that in the case where 2 or more data blocks are written to > the log that they could written once within a single larger log block. > The log block size must be larger than the data block size, of course. I think Tom's right...

Re: [HACKERS] qsort, once again

2006-03-16 Thread Jonah H. Harris
On 3/16/06, Tom Lane <[EMAIL PROTECTED]> wrote: So we still have a problem of software archaeology: who added theinsertion sort switch to the NetBSD version, and on what grounds? AFAICS, the insertion sort was added in BSD 4.4-lite and was inherited by NetBSD in CVS version 1.1.1.2. The previous

Re: [HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Overall, the two things are fairly separate, apart from the fact that we > do currently log whole data blocks straight to the log. Usually just > one, but possibly 2 or three. So I have a feeling that things would > become less efficient if you did this, no

Re: [HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Mark Wong
On Thu, 16 Mar 2006 19:37:07 + Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2006-03-16 at 08:21 -0800, Mark Wong wrote: > > > I've been wondering if there might be anything to gain by having a > > separate block size for logging and data. I thought I might try > > defining DATA_BLCKSZ an

Re: [HACKERS] qsort, once again

2006-03-16 Thread Dann Corbit
I have seen a lot of dumb "fixes" to sort routines. In a commercial sort function I saw some time ago, the check for a condition that causes qsort to go quadratic was removed. There was a comment there that said the engineer "didn't see any improvement in performance". Of course, the check was n

Re: [HACKERS] qsort, once again

2006-03-16 Thread Tom Lane
"Dann Corbit" <[EMAIL PROTECTED]> writes: > I sent him a copy Thanks. This is really interesting: the switch to insertion sort on perfect pivot is simply not there in Bentley & McIlroy's paper. So it was added later, and evidently not tested as carefully as it should have been. At this point I

Re: [HACKERS] qsort, once again

2006-03-16 Thread Jonah H. Harris
On 3/16/06, Tom Lane <[EMAIL PROTECTED]> wrote: "Jonah H. Harris" <[EMAIL PROTECTED]> writes:> doh! Sorry for the duplicate (again) Tom...  I wish Gmail would auto-reply to all.  grr.  Anyway, this is copied for others and the archives: There's a program I had used in the past to test qsort imple

Re: [HACKERS] qsort, once again

2006-03-16 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > doh! > On 3/16/06, Dann Corbit <[EMAIL PROTECTED]> wrote: >> I sent him a copy Got it, thanks guys ... regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our

Re: [HACKERS] qsort, once again

2006-03-16 Thread Jonah H. Harris
doh!On 3/16/06, Dann Corbit <[EMAIL PROTECTED]> wrote: I sent him  a copy   From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Jonah H. Harris Sent: Thursday, March 16, 2006 11:43 AM To: Tom Lane Cc: pgsql-hackers@postgresql.org; Jerry Sievers Subject: Re

Re: [HACKERS] qsort, once again

2006-03-16 Thread Dann Corbit
I sent him  a copy   From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Jonah H. Harris Sent: Thursday, March 16, 2006 11:43 AM To: Tom Lane Cc: pgsql-hackers@postgresql.org; Jerry Sievers Subject: Re: [HACKERS] qsort, once again   On 3/16/06, Tom Lane <[EMAIL P

Re: [HACKERS] qsort, once again

2006-03-16 Thread Jonah H. Harris
On 3/16/06, Tom Lane <[EMAIL PROTECTED]> wrote: I'm wondering what the authors were expecting the insertion sort tohandle exactly.  Does anyone have a copy of the paper that's referencedin the code comment?/* * Qsort routine from Bentley & McIlroy's "Engineering a Sort Function". */ Yes, I have it

[HACKERS] qsort, once again

2006-03-16 Thread Tom Lane
I was just looking at the behavior of src/port/qsort.c on the test case that Jerry Sievers was complaining about in pgsql-admin this morning. I found out what the real weak spot is: it's got nothing directly to do with good or bad pivots, it's this code right here: if (swap_cnt == 0) {

Re: [HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Simon Riggs
On Thu, 2006-03-16 at 08:21 -0800, Mark Wong wrote: > I've been wondering if there might be anything to gain by having a > separate block size for logging and data. I thought I might try > defining DATA_BLCKSZ and LOG_BLCKSZ and see what kind of trouble I get > myself into. > > I wasn't able to

Re: [HACKERS] problems compiling CVS HEAD - LDAP auth and Kerberos

2006-03-16 Thread Stefan Kaltenbrunner
Tom Lane wrote: > "Albe Laurenz" <[EMAIL PROTECTED]> writes: > >>Compiling src/interfaces/libpq/fe-auth.c on Linux I get > > >>gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline >>-Wdeclaration-after-statement -fno-strict-aliasing >>-I../../../src/include -D_GNU_SOURCE -I/usr/kerberos/

Re: [HACKERS] problems compiling CVS HEAD - LDAP auth and Kerberos

2006-03-16 Thread Magnus Hagander
> > Compiling src/interfaces/libpq/fe-auth.c on Linux I get > > > gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline > > -Wdeclaration-after-statement -fno-strict-aliasing > > -I../../../src/include -D_GNU_SOURCE > -I/usr/kerberos/include -c -o > > auth.o auth.c > > auth.c:793:1: di

Re: [HACKERS] problems compiling CVS HEAD - LDAP auth and Kerberos

2006-03-16 Thread Tom Lane
"Albe Laurenz" <[EMAIL PROTECTED]> writes: > Compiling src/interfaces/libpq/fe-auth.c on Linux I get > gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline > -Wdeclaration-after-statement -fno-strict-aliasing > -I../../../src/include -D_GNU_SOURCE -I/usr/kerberos/include -c -o > auth.o au

Re: [HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Jonah H. Harris
On 3/16/06, Mark Wong <[EMAIL PROTECTED]> wrote: I've been wondering if there might be anything to gain by having aseparate block size for logging and data.  I thought I might trydefining DATA_BLCKSZ and LOG_BLCKSZ and see what kind of trouble I getmyself into. If you're going to try it out, here'

[HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Mark Wong
Hi all, I've been wondering if there might be anything to gain by having a separate block size for logging and data. I thought I might try defining DATA_BLCKSZ and LOG_BLCKSZ and see what kind of trouble I get myself into. I wasn't able to find any previous discussion but pehaps 'separate BLKSZ'

Re: [HACKERS] Bug report form: locale/encoding

2006-03-16 Thread Tom Lane
"Magnus Hagander" <[EMAIL PROTECTED]> writes: > Output from pg_config: > Output from pg_controldata: > should do a lot. With that, we could even remove the version field if we > wanted to - but do we? Might be good to keep it in case ppl have a > question that doesn't need/want pg_config/pg_contro

Re: [HACKERS] Bug report form: locale/encoding

2006-03-16 Thread Magnus Hagander
> > > Has anyone considered something like "reportbug" in Debian which > > > collects a certain amount of information about the system > and then > > > creates an email with the information included. Of perhaps a > > > pg_buginfo like this: > > > > pg_config and pg_controldata together cover m

Re: [HACKERS] Bug report form: locale/encoding

2006-03-16 Thread Martijn van Oosterhout
On Thu, Mar 16, 2006 at 10:07:52AM -0500, Tom Lane wrote: > Martijn van Oosterhout writes: > > Has anyone considered something like "reportbug" in Debian which > > collects a certain amount of information about the system and then > > creates an email with the information included. Of perhaps a pg

[HACKERS] Automatically setting work_mem

2006-03-16 Thread Simon Riggs
One of the key points influencing performance of certain operations is the amount of memory that is available to them. Sorts, Materialize, Hash Joins and Aggs and hashed subquery plans all want lots of memory. Static allocation of memory is good in some situations, but not in others. In many cases

Re: [HACKERS] problems compiling CVS HEAD - LDAP auth and Kerberos

2006-03-16 Thread Albe Laurenz
> Compiling src/interfaces/libpq/fe-auth.c on Linux I get [compiler error] Sorry to follow myself up, but I was talking about src/backend/libpq/auth.c Yours, Laurenz Albe ---(end of broadcast)--- TIP 4: Have you searched our list archives?

Re: [HACKERS] Pregunta sobr

2006-03-16 Thread Alvaro Herrera
Alvaro Herrera wrote: > Paulina Quezada wrote: > > Hola, necesito saber con que sentencias sql puedo controlar desde un trigger > > el número de registros a actualizar o a borrar, esto para que desde la > > consola no se hagan deletes o updates masivos. > > No hay manera alguna. Sorry, I didn't n

[HACKERS] problems compiling CVS HEAD - LDAP auth and Kerberos

2006-03-16 Thread Albe Laurenz
I have tried the --with-ldap option since I also intend to submit a patch that uses LDAP functionality. Compiling src/interfaces/libpq/fe-auth.c on Linux I get gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline -Wdeclaration-after-statement -fno-strict-aliasing -I../../../src/include -D_

Re: [HACKERS] Bug report form: locale/encoding

2006-03-16 Thread Tom Lane
Martijn van Oosterhout writes: > Has anyone considered something like "reportbug" in Debian which > collects a certain amount of information about the system and then > creates an email with the information included. Of perhaps a pg_buginfo > like this: pg_config and pg_controldata together cover

Re: [HACKERS] Pregunta sobr

2006-03-16 Thread Alvaro Herrera
Paulina Quezada wrote: > Hola, necesito saber con que sentencias sql puedo controlar desde un trigger > el número de registros a actualizar o a borrar, esto para que desde la > consola no se hagan deletes o updates masivos. No hay manera alguna. Si quieres experimentar, podrias tratar con un trig

Re: [HACKERS] Bug report form: locale/encoding

2006-03-16 Thread Andrew Dunstan
William ZHANG wrote: Has anyone considered something like "reportbug" in Debian which collects a certain amount of information about the system and then creates an email with the information included. Of perhaps a pg_buginfo like this: It sounds good. Should we setup a bug tracker system

Re: [HACKERS] Bug report form: locale/encoding

2006-03-16 Thread William ZHANG
>Has anyone considered something like "reportbug" in Debian which >collects a certain amount of information about the system and then >creates an email with the information included. Of perhaps a pg_buginfo >like this: It sounds good. Should we setup a bug tracker system for pgsql? Regards, Willi

Re: [HACKERS] Pregunta sobre limitar el

2006-03-16 Thread Simon Riggs
On Thu, 2006-03-16 at 11:26 +0700, Jeroen T. Vermeulen wrote: > On Tue, March 14, 2006 22:14, Paulina Quezada wrote: > > Hola, necesito saber con que sentencias sql puedo controlar desde un > > trigger > > el número de registros a actualizar o a borrar, esto para que desde la > > consola no se haga

Re: [HACKERS] Bug report form: locale/encoding

2006-03-16 Thread Martijn van Oosterhout
On Thu, Mar 16, 2006 at 10:11:23AM +0100, Magnus Hagander wrote: > Considering the amount of bug repotrs that get their first question as > "what's the locale" and "what's the encoding", should we perhaps add > those fields to the bug reporting form? > > And if we should, should we add them as opt

[HACKERS] Bug report form: locale/encoding

2006-03-16 Thread Magnus Hagander
Considering the amount of bug repotrs that get their first question as "what's the locale" and "what's the encoding", should we perhaps add those fields to the bug reporting form? And if we should, should we add them as optional or mandatory? //Magnus ---(end of broadca

Re: [HACKERS] Static build of psql with readline support

2006-03-16 Thread Martijn van Oosterhout
On Thu, Mar 16, 2006 at 04:09:53PM +1300, Mark Kirkwood wrote: > Christopher Kings-Lynne wrote: > >Hi guys, > > > >I've been trying to build the cvs checkout of 8.1.3 on my freebsd 4.9 > >box with a STATIC psql utility. I keep getting failures trying to hook > >in libreadline I think: > Add a