Re: [HACKERS] GiST rtree logic is not right

2005-06-22 Thread John Hansen
> I'll look at problem after GiST concurrency. Fixing > rtree_gist is bug a fix, not a new feature, so I'm not > limited by 1 July. Wont fixing rtree(_gist) require initdb, since the behaviour of the operators will change? ... John ---(end of broadcast)

Re: [HACKERS] GiST rtree logic is not right

2005-06-22 Thread Teodor Sigaev
I'll look at problem after GiST concurrency. Fixing rtree_gist is bug a fix, not a new feature, so I'm not limited by 1 July. This is doubtless not as high priority as the concurrency stuff you are working on, but it'd be good to fix anyway. I was thinking of proposing that we move rtree_gist

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Curt Sampson
On Thu, 23 Jun 2005, Tom Lane wrote: The bottom line here seems to be the same as always: you can't run an industrial strength database on piece-of-junk consumer grade hardware. Sure you can, though it may take several bits of piece-of-junk consumer-grade hardware. It's far more about how you

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Gavin Sherry
On Thu, 23 Jun 2005, Tom Lane wrote: > Gavin Sherry <[EMAIL PROTECTED]> writes: > >> Curt Sampson <[EMAIL PROTECTED]> writes: > >>> But is it really a problem? I somewhere got the impression that some > >>> drives, on power failure, will be able to keep going for long enough to > >>> write out the

Re: [HACKERS] HaveNFreeProcs ?

2005-06-22 Thread Tom Lane
I wrote: > ... because it's written to not loop more than > superuser_reserved_connections times, and it's hard to imagine anyone > would set that to more than half a dozen or so. We could help keep people on the correct path if guc.c enforced a sane upper limit on superuser_reserved_connections.

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Tom Lane
Gavin Sherry <[EMAIL PROTECTED]> writes: >> Curt Sampson <[EMAIL PROTECTED]> writes: >>> But is it really a problem? I somewhere got the impression that some >>> drives, on power failure, will be able to keep going for long enough to >>> write out the cache and park the heads anyway. If so, the dri

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Gregory Maxwell
On 6/23/05, Gavin Sherry <[EMAIL PROTECTED]> wrote: > > inertia) but seeking to a lot of new tracks to write randomly-positioned > > dirty sectors would require significant energy that just ain't there > > once the power drops. I seem to recall reading that the seek actuators > > eat the largest

Re: [HACKERS] HaveNFreeProcs ?

2005-06-22 Thread Tom Lane
I wrote: > Also, that routine will disappear entirely if we agree to remove > commit_siblings (see nearby thread), so right at the moment I'm not very > concerned about improving it. If it is still there forty-eight hours > from now, let's talk about it then. Oh, never mind that, I was momentaril

Re: [HACKERS] HaveNFreeProcs ?

2005-06-22 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > I just noticed the HaveNFreeProcs routine is coded as a loop around the > ProcGlobal struct members. I wonder if it's possible to use a simple > check in procArray->numBackends against procArray->maxBackends instead? It used to look like that, but now

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Gavin Sherry
On Thu, 23 Jun 2005, Tom Lane wrote: > [ on the other point... ] > > Curt Sampson <[EMAIL PROTECTED]> writes: > > But is it really a problem? I somewhere got the impression that some > > drives, on power failure, will be able to keep going for long enough to > > write out the cache and park the he

[HACKERS] HaveNFreeProcs ?

2005-06-22 Thread Alvaro Herrera
Hackers, I just noticed the HaveNFreeProcs routine is coded as a loop around the ProcGlobal struct members. I wonder if it's possible to use a simple check in procArray->numBackends against procArray->maxBackends instead? -- Alvaro Herrera () Jason Tesser: You might not have understood me or I

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Tom Lane
[ on the other point... ] Curt Sampson <[EMAIL PROTECTED]> writes: > But is it really a problem? I somewhere got the impression that some > drives, on power failure, will be able to keep going for long enough to > write out the cache and park the heads anyway. If so, the drive is still > guarantee

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Curt Sampson
On Wed, 22 Jun 2005, Tom Lane wrote: [ shudder ] I can see the complaints now: "Merely starting up Postgres cut my overall system performance by a factor of 10! Yeah, quite the scenario. This can *not* be default behavior, and unfortunately that limits its value quite a lot. Indeed. Maybe

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Tom Lane
Curt Sampson <[EMAIL PROTECTED]> writes: > But regardless, perhaps we can add some stuff to the various OSes' > startup scripts that could help with this. For example, in NetBSD you > can "dkctl setcache r" for most any disk device (certainly all > SCSI and ATA) to enable the read cache and disabl

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Curt Sampson
On Thu, 22 Jun 2005, Greg Stark wrote: Tom Lane <[EMAIL PROTECTED]> writes: Unfortunately, I cannot believe these numbers --- the near equality of fsync off and fsync on means there is something very wrong with the measurements. What I suspect is that your ATA drives are doing write caching a

Re: [HACKERS] commit_delay, siblings

2005-06-22 Thread Tom Lane
"Qingqing Zhou" <[EMAIL PROTECTED]> writes: > Would commit_delay/commit_siblings helps or we need a > background xlog writer and notify us the completion of xlogflush is better > (so we don't compete for this lock)? The existing bgwriter already does a certain amount of xlog flushing (since it mus

Re: [HACKERS] commit_delay, siblings

2005-06-22 Thread Qingqing Zhou
"Josh Berkus" writes > Hackers: > > I've been trying to get a test result for 8.1 that shows that we can eliminate > commit_delay and commit_siblings, as I believe that these settings no longer > have any real effect on performance. However, the checkpointing performance > issues have so far pre

[HACKERS] GiST rtree logic is not right

2005-06-22 Thread Tom Lane
It occurred to me to wonder whether contrib/rtree_gist fixes the rtree bug documented here: http://archives.postgresql.org/pgsql-general/2004-03/msg01143.php The answer is unfortunately "no". In the regression database, install rtree_gist and do: regression=# create table gist_emp4000 as select

[HACKERS] COPY FROM performance improvements

2005-06-22 Thread Alon Goldshuv
This is a second iteration of a previous thread that didn't resolve few weeks ago. I made some more modifications to the code to make it compatible with the current COPY FROM code and it should be more agreeable this time. The main premise of the new code is that it improves the text data parsing

[HACKERS] [PATCH] pgcrypto: pgp_encrypt

2005-06-22 Thread Marko Kreen
Finally, here is pgp_encrypt()/pgp_decrypt() - implementation of password-based encryption from RFC2440 (OpenPGP). The goal of this code is to be more featureful encryption solution than current encrypt(), which only functionality is running cipher over data. Compared to encrypt(), pgp_encrypt()

Re: [HACKERS] pg_terminate_backend idea

2005-06-22 Thread Tom Lane
"Magnus Hagander" <[EMAIL PROTECTED]> writes: > Assuming we don't get such a case, and a chance to fix it, before 8.1 > (while still hoping we will get it fixed properly, we can't be sure, can > we? If we were, it'd be fixed already). In this case, will you consider > such a kludgy solution as a te

Re: [HACKERS] Problem with dblink regression test

2005-06-22 Thread Chris Campbell
On Jun 22, 2005, at 12:52, Tom Lane wrote: "Jim C. Nasby" <[EMAIL PROTECTED]> writes: Is there a way to confirm which libpq.so psql and/or dblink.so has linked to? Are there any other tests I could run to shed some light on this? On Linux you use "ldd" to find out what the linker will do

Re: [HACKERS] pg_terminate_backend idea

2005-06-22 Thread Magnus Hagander
> >> In any case the correct way to solve the problem is to find out > >> what's being left corrupt by SIGTERM, rather than install more > >> messiness in order to avoid facing the real issue ... > > > That is unfortunatly way over my head. And it doesn't seem like > > anybody who actually has

Re: [HACKERS] [PATCHES] Removing Kerberos 4

2005-06-22 Thread Tom Lane
"Magnus Hagander" <[EMAIL PROTECTED]> writes: > Yeah. But it has been declared dead by the Kerberos folks > (http://www.faqs.org/faqs/kerberos-faq/general/section-7.html. And this > document is from 2000, an dit was declared already then)... Right. The real question here is who's going to be usin

Re: [HACKERS] pl/pgsql: END verbosity

2005-06-22 Thread Merlin Moncure
> > Neil Conway said: > > > In PL/PgSQL, "END LOOP" is used to terminate loop blocks, and "END IF" > > > is used to terminate IF blocks. This is needlessly verbose: we could > > > simply accept "END" in both cases without syntactic ambiguity. I'd > like > > > to make this change, so that END can b

Re: [HACKERS] [PATCHES] Removing Kerberos 4

2005-06-22 Thread Magnus Hagander
> > Last chance for any Kerberos 4 users to speak up --- otherwise I'll > > apply this soon. > > If you just want someone to test it I can do that. I don't > actually use it normally though. I don't think "just testing" is enough - somebody needs to actually maintain it... > As far as securi

Re: [HACKERS] commit_delay, siblings

2005-06-22 Thread Greg Stark
Hans-Jürgen Schönig <[EMAIL PROTECTED]> writes: > > The theory is good, but useful values for commit_delay would probably be > > under a millisecond, and there isn't any portable way to sleep for such > > short periods. Just because there's no "portable" way to be sure it'll work doesn't mean th

Re: [HACKERS] Why is checkpoint so costly?

2005-06-22 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > The question should be why is there any time when a checkpoint *isn't* > happening? For maximum performance the combination of bgwriter (basically > preemptive checkpoint i/o) and the actual checkpoint i/o should be executing > at a more or less even pace th

Re: [HACKERS] [PATCHES] Removing Kerberos 4

2005-06-22 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Last chance for any Kerberos 4 users to speak up --- otherwise I'll > apply this soon. If you just want someone to test it I can do that. I don't actually use it normally though. As far as security issues the only issues I'm aware of is a) it uses plain DES

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> writes: >> Unfortunately, I cannot believe these numbers --- the near equality of >> fsync off and fsync on means there is something very wrong with the >> measurements. What I suspect is that your ATA drives are doing write >>

Re: [HACKERS] pl/pgsql: END verbosity

2005-06-22 Thread Pavel Stehule
On Tue, 21 Jun 2005, Andrew Dunstan wrote: > Neil Conway said: > > In PL/PgSQL, "END LOOP" is used to terminate loop blocks, and "END IF" > > is used to terminate IF blocks. This is needlessly verbose: we could > > simply accept "END" in both cases without syntactic ambiguity. I'd like > > to mak

Re: [HACKERS] Why is checkpoint so costly?

2005-06-22 Thread Greg Stark
Josh Berkus writes: > Folks, > > Going over some performance test results at OSDL, our single greatest > performance issue seems to be checkpointing.Not matter how I fiddle > with it, checkpoints seem to cost us 1/2 of our throughput while they're > taking place. Overally, checkpointing

Re: [HACKERS] pg_terminate_backend idea

2005-06-22 Thread Andrew - Supernews
On 2005-06-22, Tom Lane <[EMAIL PROTECTED]> wrote: > Andrew - Supernews <[EMAIL PROTECTED]> writes: >> On 2005-06-22, Tom Lane <[EMAIL PROTECTED]> wrote: >>> Andreas Pflug <[EMAIL PROTECTED]> writes: I've seen cancel *not* working. >>> >>> Even a moment's perusal of the code will prove that t

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-06-22 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Unfortunately, I cannot believe these numbers --- the near equality of > fsync off and fsync on means there is something very wrong with the > measurements. What I suspect is that your ATA drives are doing write > caching and thus the "fsyncs" are not really

Re: [HACKERS] pg_terminate_backend idea

2005-06-22 Thread Tom Lane
Andrew - Supernews <[EMAIL PROTECTED]> writes: > On 2005-06-22, Tom Lane <[EMAIL PROTECTED]> wrote: >> Andreas Pflug <[EMAIL PROTECTED]> writes: >>> I've seen cancel *not* working. >> >> Even a moment's perusal of the code will prove that there is no >> situation in which a backend will respond to

Re: [HACKERS] pg_terminate_backend idea

2005-06-22 Thread Andrew - Supernews
On 2005-06-22, Tom Lane <[EMAIL PROTECTED]> wrote: > Andreas Pflug <[EMAIL PROTECTED]> writes: >>> I thought we agreed that using the cancel functionality, which we know >>> works and is tested, > >> I've seen cancel *not* working. In 80 % this was the reason to use >> terminate. > > Even a moment

Re: [HACKERS] User Quota Implementation

2005-06-22 Thread Jonah H. Harris
Tom, You're right, this is going to take more work to make sure all is perfect. Let me work up a formal definition and send it to the group. Thanks for bringing me back to my senses. -Jonah Tom Lane wrote: "Jonah H. Harris" <[EMAIL PROTECTED]> writes: If I recall correctly, I never go

Re: [HACKERS] commit_delay, siblings

2005-06-22 Thread Josh Berkus
Hans, Tom, > We have done extensive testing some time ago. > We could not see any difference on any platform we have tested (AIX, > Linux, Solaris). I don't think that there is one at all - at least not > on common systems. Keen then. Any objections to removing the GUC? We desperately need mea

Re: [HACKERS] commit_delay, siblings

2005-06-22 Thread Hans-Jürgen Schönig
Tom Lane wrote: Josh Berkus writes: I've been trying to get a test result for 8.1 that shows that we can eliminate commit_delay and commit_siblings, as I believe that these settings no longer have any real effect on performance. I don't think they ever did :-(. The theory is good, but use

Re: [HACKERS] pl/pgsql: END verbosity

2005-06-22 Thread Alvaro Herrera
On Wed, Jun 22, 2005 at 09:23:17AM -0700, Steve Atkins wrote: > On Thu, Jun 23, 2005 at 01:41:49AM +1000, Neil Conway wrote: > > Andrew Dunstan wrote: > > >But this doesn't make it easier to use - users don't just include those who > > >write it. The antecedent language of these, Ada, from which th

Re: [HACKERS] commit_delay, siblings

2005-06-22 Thread Tom Lane
Josh Berkus writes: > I've been trying to get a test result for 8.1 that shows that we can > eliminate > commit_delay and commit_siblings, as I believe that these settings no longer > have any real effect on performance. I don't think they ever did :-(. The theory is good, but useful values f

Re: [HACKERS] Problem with dblink regression test

2005-06-22 Thread Jim C. Nasby
On Tue, Jun 21, 2005 at 08:49:12PM -0700, Joe Conway wrote: > I think most people would expect that if they don't specify a port, they > would be talking to the same postmaster that they are running under on > whatever port it is using, not some compiled in default. So your > proposal makes perf

Re: [HACKERS] Problem with dblink regression test

2005-06-22 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Is there a way to confirm which libpq.so psql and/or dblink.so has > linked to? Are there any other tests I could run to shed some light on > this? On Linux you use "ldd" to find out what the linker will do with dependencies of an executable or shared l

[HACKERS] commit_delay, siblings

2005-06-22 Thread Josh Berkus
Hackers: I've been trying to get a test result for 8.1 that shows that we can eliminate commit_delay and commit_siblings, as I believe that these settings no longer have any real effect on performance. However, the checkpointing performance issues have so far prevented me from getting a good t

Re: [HACKERS] Problem with dblink regression test

2005-06-22 Thread Jim C. Nasby
On Wed, Jun 22, 2005 at 11:45:09AM -0400, Tom Lane wrote: > "Andrew Dunstan" <[EMAIL PROTECTED]> writes: > > Tom Lane said: > >> There are several buildfarm machines failing like this. I think a > >> possible solution is for the postmaster to do putenv("PGPORT=nnn") so > >> that libpq instances ru

Re: [HACKERS] pl/pgsql: END verbosity

2005-06-22 Thread Steve Atkins
On Thu, Jun 23, 2005 at 01:41:49AM +1000, Neil Conway wrote: > Andrew Dunstan wrote: > >But this doesn't make it easier to use - users don't just include those who > >write it. The antecedent language of these, Ada, from which this syntax > >comes, was explicitly designed to be reader-friendly as o

Re: [HACKERS] User Quota Implementation

2005-06-22 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > If I recall correctly, I never got a response. I can still get it done > quickly and probably before the July 1st feature freeze (if that's still > the date). Tom, Bruce, Josh, et al what are your thoughts if I submit a > patch in the next few da

Re: [HACKERS] Problem with dblink regression test

2005-06-22 Thread Tom Lane
"Andrew Dunstan" <[EMAIL PROTECTED]> writes: > Tom Lane said: >> There are several buildfarm machines failing like this. I think a >> possible solution is for the postmaster to do putenv("PGPORT=nnn") so >> that libpq instances running in postmaster children will default to the >> local installati

Re: [HACKERS] pl/pgsql: END verbosity

2005-06-22 Thread Neil Conway
Andrew Dunstan wrote: But this doesn't make it easier to use - users don't just include those who write it. The antecedent language of these, Ada, from which this syntax comes, was explicitly designed to be reader-friendly as opposed to writer-friendly, and this is a part of that. IMHO it is ju

Re: [HACKERS] User Quota Implementation

2005-06-22 Thread Jonah H. Harris
If I recall correctly, I never got a response. I can still get it done quickly and probably before the July 1st feature freeze (if that's still the date). Tom, Bruce, Josh, et al what are your thoughts if I submit a patch in the next few days? Is everyone already too busy reviewing the curre

Re: [HACKERS] pg_terminate_backend idea

2005-06-22 Thread Tom Lane
Andreas Pflug <[EMAIL PROTECTED]> writes: >> I thought we agreed that using the cancel functionality, which we know >> works and is tested, > I've seen cancel *not* working. In 80 % this was the reason to use > terminate. Even a moment's perusal of the code will prove that there is no situation

Re: [HACKERS] pg_terminate_backend idea

2005-06-22 Thread Tom Lane
"Magnus Hagander" <[EMAIL PROTECTED]> writes: >> In any case the correct way to solve the problem is to find >> out what's being left corrupt by SIGTERM, rather than install >> more messiness in order to avoid facing the real issue ... > That is unfortunatly way over my head. And it doesn't seem

Re: [HACKERS] pl/pgsql: END verbosity

2005-06-22 Thread Andrew Dunstan
Neil Conway said: > Andrew Dunstan wrote: >> I'm unkeen. I see no technical advantage - it's just a matter of >> taste. > > There is no "technical advantage" to case insensitive keywords, or > dollar quoting, or a variety of other programming language features > that don't change functionality but

Re: [HACKERS] PROPOSAL FE/BE extension to handle IN/OUT parameters

2005-06-22 Thread Dave Cramer
Tom, This will work just great, please go ahead, and I'll adjust the driver accordingly Dave On 21-Jun-05, at 5:49 PM, Tom Lane wrote: Dave Cramer <[EMAIL PROTECTED]> writes: Yeah, I think that might work if I understand it correctly. Assuming I would be able to prepare, and bind all the

Re: [HACKERS] Server instrumentation patch

2005-06-22 Thread Dave Page
> -Original Message- > From: Bruce Momjian [mailto:[EMAIL PROTECTED] > Sent: 22 June 2005 04:08 > To: Andreas Pflug > Cc: Dave Page; PostgreSQL-development > Subject: Re: Server instrumentation patch > > > > The move of dbsize into the backend is similar. He moves > the parts of > >

Re: [HACKERS] pg_terminate_backend idea

2005-06-22 Thread Andreas Pflug
Bruce Momjian wrote: Tom Lane wrote: "Magnus Hagander" <[EMAIL PROTECTED]> writes: But it still requires me to send some data (such as a dummy query) to the backend before it exits. This is because server side libpq blocks when reading and ignores signals at this time. I believe the fix for t

Re: [HACKERS] User Quota Implementation

2005-06-22 Thread Yann Michel
Hi again, On Mon, Jun 13, 2005 at 04:47:20PM -0600, Jonah H. Harris wrote: > Well... a maximum tablespace size would be much easier to implement and > would still accomplish this level of quota for larger organizations and > database systems. > > I vote for implmenting the maximum tablespace si

Re: [HACKERS] Server instrumentation patch

2005-06-22 Thread Dave Page
> -Original Message- > From: Bruce Momjian [mailto:[EMAIL PROTECTED] > Sent: 21 June 2005 18:06 > To: Dave Page > Cc: PostgreSQL-development; Andreas Pflug > Subject: Server instrumentation patch > > > OK, let me address this, but you might not like what I have > to say. ;-) > > Ba

Re: [HACKERS] pg_terminate_backend idea

2005-06-22 Thread Magnus Hagander
> > But it still requires me to send some data (such as a dummy > query) to > > the backend before it exits. This is because server side > libpq blocks > > when reading and ignores signals at this time. I believe > the fix for > > this would be to pass a flag down to the libpq routines > tha

Re: [HACKERS] Schedule for 8.1 feature freeze

2005-06-22 Thread Fabien COELHO
Dear Stephen, I'd really like to see role support added into 8.1. I'm also pretty interested in this, and was planing loosely to think about implementing roles someday. It is even better if it is done by someone else;-) I've sent Alvaro and Tom versions of the patch in the past and I was

Re: [HACKERS] CONTINUE error, even though inside a loop

2005-06-22 Thread Neil Conway
Michael Fuhr wrote: I'm getting "CONTINUE cannot be used outside a loop" errors even though it's inside a loop. The error appears to be happening when CONTINUE passes control to the beginning of the loop but there's no more iterating to be done. Woops, sorry for missing this. This should be fi