Re: [HACKERS] Simplifying replication

2010-10-27 Thread Simon Riggs
On Tue, 2010-10-26 at 22:03 -0400, Robert Haas wrote: On Tue, Oct 26, 2010 at 9:59 PM, Josh Berkus j...@agliodbs.com wrote: If you set wal_keep_segments=0, archive_mode=on, and archive_command=something, you might run out of disk space. If you set wal_keep_segments=-1, you might run out

Re: [HACKERS] add label to enum syntax

2010-10-27 Thread Dean Rasheed
On 27 October 2010 02:16, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Dean Rasheed's message of mar oct 26 15:46:56 -0300 2010: Well ELEMENT is a reserved keyword in SQL:2008, to support multisets, so if we ever supported that feature...

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Simon Riggs
On Tue, 2010-10-19 at 15:32 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Oct 19, 2010 at 12:18 PM, Josh Berkus j...@agliodbs.com wrote: On 10/19/2010 09:06 AM, Greg Smith wrote: I think Magnus's idea to bump the default to 5 triages the worst of the annoyance

Re: [HACKERS] add label to enum syntax

2010-10-27 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Dean Rasheed's message: Well ELEMENT is a reserved keyword in SQL:2008, to support multisets, so if we ever supported that feature... Hah! Well, here's a patch for LABEL in any case. If we're going to have to reserve

Re: [HACKERS] add label to enum syntax

2010-10-27 Thread Andrew Dunstan
On 10/26/2010 09:16 PM, Tom Lane wrote: But ... having said all that, I have to agree that ELEMENT seems preferable to LABEL if we ignore micro-considerations of parser efficiency --- I still think LABEL is a pretty poor choice of word here. Personally I'd still take VALUE as being my first

Re: [HACKERS] Re: ECPG dynamic cursor fix for UPDATE/DELETE ... WHERE CURRENT OF :curname

2010-10-27 Thread Boszormenyi Zoltan
Hi, Michael Meskes írta: 1. The statement UPDATE table SET fld1 = :input1 WHERE CURRENT OF :curname RETURNING id + :input2; is transformed into UPDATE table SET fld1 = $1 WHERE CURRENT OF $0 RETURNING id + $2; and the $0 is past $1. The current code

[HACKERS] Tracking latest timeline in standby mode

2010-10-27 Thread Heikki Linnakangas
At the moment, when you specify recovery_target_timeline='latest', we scan for the latest timeline at the beginning of recovery, and pick that as the target. If new timelines appear during recovery, we stick to the target chosen in the beginning, the new timelines are ignored. That's

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-27 Thread Markus Wanner
On 10/26/2010 05:52 PM, Alvaro Herrera wrote: And horrible for performance, I imagine. Those locks are highly trafficked. Note, however, that offloading this to the file-system just moves congestion there. So we are effectively saying that we expect filesystems to do a better job (in that

Re: [HACKERS] add label to enum syntax

2010-10-27 Thread Andrew Dunstan
On 10/27/2010 10:00 AM, Kevin Grittner wrote: Alvaro Herreraalvhe...@commandprompt.com wrote: Excerpts from Dean Rasheed's message: Well ELEMENT is a reserved keyword in SQL:2008, to support multisets, so if we ever supported that feature... Hah! Well, here's a patch for LABEL in any

Re: [HACKERS] add label to enum syntax

2010-10-27 Thread Alvaro Herrera
Excerpts from Andrew Dunstan's message of mié oct 27 11:18:44 -0300 2010: The reason I chose LABEL was that it's consistent with what we have used elsewhere, both in the docs and the catalog. But I'm not strongly wedded to it. If it's a choice between ELEMENT and VALUE, I too prefer VALUE at

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-27 Thread Alvaro Herrera
Excerpts from Markus Wanner's message of mié oct 27 11:44:20 -0300 2010: On 10/26/2010 05:52 PM, Alvaro Herrera wrote: And horrible for performance, I imagine. Those locks are highly trafficked. Note, however, that offloading this to the file-system just moves congestion there. So we are

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Josh Berkus
It is the same to the user either way. In either case you have to change some settings and restart the master. Except that changing wal_keep_segments doesn't require restarting the master. Our docs say that it does: This parameter can only be set in the postgresql.conf file or on the

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Kevin Grittner
Josh Berkus j...@agliodbs.com wrote: Except that changing wal_keep_segments doesn't require restarting the master. Our docs say that it does: This parameter can only be set in the postgresql.conf file or on the server command line. That sounds as though a reload would do it; I don't see

[HACKERS] Bikeshedding on enum vocabulary

2010-10-27 Thread Aidan Van Dyk
On Wed, Oct 27, 2010 at 10:57 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Wow, this must be the most difficult smallest thing I have ever seen discussed in pg-hackers.  It doesn't seem like there are enough votes in any particular direction.  Now *this* is proper bikeshedding.

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-10-27 Thread Heikki Linnakangas
On 26.10.2010 18:34, Boszormenyi Zoltan wrote: thank you very much for pointing me to dynahash, here is the next version that finally seems to work. Two patches are attached, the first is the absolute minimum for making it work, this still has the Tree type for canon_pathkeys and eq_classes got

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Simon Riggs
On Wed, 2010-10-27 at 10:05 -0700, Josh Berkus wrote: Josh has completely failed to make a case that that should be the default. Agreed. In what way have a failed to make a case? I just removed a huge hurdle on the journey to simplification. That doesn't mean I think you have come up

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: Josh has completely failed to make a case that that should be the default. Agreed. In what way have a failed to make a case? You're assuming that we should set up the default behavior to support replication and penalize those who aren't using it.

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Josh Berkus j...@agliodbs.com wrote: Except that changing wal_keep_segments doesn't require restarting the master. Our docs say that it does: This parameter can only be set in the postgresql.conf file or on the server command line. That

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Joshua D. Drake
On Wed, 2010-10-27 at 15:33 -0400, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Josh has completely failed to make a case that that should be the default. Agreed. In what way have a failed to make a case? You're assuming that we should set up the default behavior to

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-27 Thread Fujii Masao
On Wed, Oct 27, 2010 at 3:03 AM, fazool mein fazoolm...@gmail.com wrote: Might I suggest adopting the same technique walsender does, ie just read the data back from disk?  There's a reason why we gave up trying to have walsender read directly from the buffers. That is exactly what I do not

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Josh Berkus
You're assuming that we should set up the default behavior to support replication and penalize those who aren't using it. What's the penalty? Simon just said that there isn't one. And there's a difference between saying that I failed to make a case vs. the cost is too great. Saying the

Re: [HACKERS] foreign keys for array/period contains relationships

2010-10-27 Thread Josh Berkus
On 10/26/10 11:53 AM, Jeff Davis wrote: Intuitively, I would expect all 1's to be replaced by 6's in all arrays. But I can now see why you would be hesitant to try to support that. If people want cascading update to work, they shouldn't be denormalizing. --

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Josh Berkus
You have to put the WAL files *somewhere* while you do the base backup. PostgreSQL can't itself work out where that is, nor can it work out ahead of time how big it will need to be, since it is up to you how you do your base backup. Setting a parameter to -1 doesn't make the problem go away,

Re: [HACKERS] crash in plancache with subtransactions

2010-10-27 Thread Tom Lane
I wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: One simple idea is to keep a flag along with the executor state to indicate that the executor state is currently in use. Set it just before calling ExecEvalExpr, and reset afterwards. If the flag is already set in the

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Greg Stark
On Wed, Oct 27, 2010 at 12:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: You're assuming that we should set up the default behavior to support replication and penalize those who aren't using it.  Considering that we haven't even *had* replication until now, it seems a pretty safe bet that the

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: You're assuming that we should set up the default behavior to support replication and penalize those who aren't using it. What's the penalty? Simon just said that there isn't one. I don't know what Simon is thinking, but I think he's nuts. There is is

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Josh Berkus
That's not even considering the extra WAL that is generated when you move up from wal_level = minimal. That's probably the bigger performance issue in practice. Yeah, I think we've established that we can't change that. I said, and meant, that you didn't make the case at all; you just

[HACKERS] An unfortunate logging behavior when (mis)configuring recovery.conf

2010-10-27 Thread Daniel Farina
Hello list, I just encountered an interesting undesirable behavior in Postgres 9.0's error reporting dealing with (trivially) malformed recovery.conf, as might be the case when setting up hot standby. In this case, there were some missing fields, and they were checked as they are supposed to be

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Joshua D. Drake
On Wed, 2010-10-27 at 16:13 -0700, Josh Berkus wrote: That's not even considering the extra WAL that is generated when you move up from wal_level = minimal. That's probably the bigger performance issue in practice. Yeah, I think we've established that we can't change that. I said,

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Josh Berkus
I would also agree that the minority of our users will want replication. The majority of CMD customers, PGX customers, EDB Customers will want replication but that is by far NOT the majority of our (.Org) users. That just means that *you don't know* how many .org users plan to implement

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Robert Haas
On Wed, Oct 27, 2010 at 7:45 PM, Josh Berkus j...@agliodbs.com wrote: I would also agree that the minority of our users will want replication. The majority of CMD customers, PGX customers, EDB Customers will want replication but that is by far NOT the majority of our (.Org) users. That just

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Joshua D. Drake
On Wed, 2010-10-27 at 19:52 -0400, Robert Haas wrote: On Wed, Oct 27, 2010 at 7:45 PM, Josh Berkus j...@agliodbs.com wrote: I would also agree that the minority of our users will want replication. The majority of CMD customers, PGX customers, EDB Customers will want replication but that is

Re: [HACKERS] max_wal_senders must die

2010-10-27 Thread Josh Berkus
None of us know. What I do know is that I don't want PostgreSQL to be slower out of the box. Understandable. So it seems like the answer is getting replication down to one configuration variable for the common case. That eliminates the cycle of oops, need to set X and restart/reload without

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Robert Haas
On Wed, Oct 27, 2010 at 3:53 AM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, 2010-10-26 at 22:03 -0400, Robert Haas wrote: On Tue, Oct 26, 2010 at 9:59 PM, Josh Berkus j...@agliodbs.com wrote: If you set wal_keep_segments=0, archive_mode=on, and archive_command=something, you might

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Robert Haas
On Wed, Oct 27, 2010 at 5:01 PM, Josh Berkus j...@agliodbs.com wrote: You have to put the WAL files *somewhere* while you do the base backup. PostgreSQL can't itself work out where that is, nor can it work out ahead of time how big it will need to be, since it is up to you how you do your

Re: [HACKERS] An unfortunate logging behavior when (mis)configuring recovery.conf

2010-10-27 Thread Jaime Casanova
On Wed, Oct 27, 2010 at 6:18 PM, Daniel Farina drfar...@acm.org wrote: As a result, aborting startup due to startup process failure is seen in the log, but not the messages seen in xlog.c:readRecoveryCommandFile that triggered the failure. can you explain what steps you did to reproduce

Re: [HACKERS] Composite Types and Function Parameters

2010-10-27 Thread Andrew Dunstan
On 10/25/2010 09:32 PM, Andrew Dunstan wrote: On 10/25/2010 07:12 PM, Tom Lane wrote: However, that objection doesn't hold for plperl or pltcl (and likely not plpython, though I don't know that language enough to be sure). So it would be a reasonable feature request to teach those PLs to

Re: [HACKERS] Composite Types and Function Parameters

2010-10-27 Thread Andrew Dunstan
On 10/25/2010 09:32 PM, Andrew Dunstan wrote: On 10/25/2010 07:12 PM, Tom Lane wrote: However, that objection doesn't hold for plperl or pltcl (and likely not plpython, though I don't know that language enough to be sure). So it would be a reasonable feature request to teach those PLs to

Re: [HACKERS] An unfortunate logging behavior when (mis)configuring recovery.conf

2010-10-27 Thread Daniel Farina
On Wed, Oct 27, 2010 at 6:44 PM, Jaime Casanova ja...@2ndquadrant.com wrote: On Wed, Oct 27, 2010 at 6:18 PM, Daniel Farina drfar...@acm.org wrote: As a result, aborting startup due to startup process failure is seen in the log, but not the messages seen in xlog.c:readRecoveryCommandFile that

Re: [HACKERS] Composite Types and Function Parameters

2010-10-27 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: But I think we can do better than this. We should really pass an hashref with the record's column names as keys rather than just calling record_out. I'll work on that. Definitely. If you aren't providing that info then it's hard to write a generic

Re: [HACKERS] Composite Types and Function Parameters

2010-10-27 Thread Andrew Dunstan
On 10/27/2010 11:38 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: But I think we can do better than this. We should really pass an hashref with the record's column names as keys rather than just calling record_out. I'll work on that. Definitely. If you aren't providing that

Re: [HACKERS] Simplifying replication

2010-10-27 Thread Josh Berkus
I sort of agree with you that the current checkpoint_segments parameter is a bit hard to tune, at least if your goal is to control the amount of disk space that will be used by WAL files. But I'm not sure your proposal is better. Instead of having a complicated formula for predicting how much