Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Simon Riggs
On Wed, 2009-09-23 at 17:45 +0300, Heikki Linnakangas wrote: Heikki Linnakangas wrote: Simon Riggs wrote: On Wed, 2009-09-23 at 11:13 +0300, Heikki Linnakangas wrote: I note that we don't emit RunningXacts after a shutdown checkpoint. So if recovery starts at a shutdown checkpoint, we

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2009-09-23 at 19:07 +0300, Heikki Linnakangas wrote: Rather than keep the numHeldLocks counters per-proc in proc array, I think it would be simpler to have a single (or one per lock partition) counter in shared memory in lock.c. It's just an optimization to make it

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Simon Riggs
On Fri, 2009-09-25 at 11:49 +0300, Heikki Linnakangas wrote: Looking at the startup sequence now. I see that you modified ExtendSUBTRANS so that it doesn't wipe out previously set values if it's called with out-of-order xids. I guess that works, although I think it can leave pages unzeroed

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Simon Riggs
On Fri, 2009-09-25 at 13:23 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: On Wed, 2009-09-23 at 17:45 +0300, Heikki Linnakangas wrote: XactClearRecoveryTransactions() when we see a shutdown checkpoint, which clears all recovery locks. But doesn't that prematurely clear all locks

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Heikki Linnakangas
Simon Riggs wrote: Definitely need to cope with them for Hot Standby. My point was general one to say that behaviour is very non-useful for users with prepared transactions. It just causes manual effort by a DBA each time the system is shutdown. The transaction manager is supposed to

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2009-09-23 at 17:45 +0300, Heikki Linnakangas wrote: XactClearRecoveryTransactions() when we see a shutdown checkpoint, which clears all recovery locks. But doesn't that prematurely clear all locks belonging to prepared transactions as well? Much better to read

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Simon Riggs
On Fri, 2009-09-25 at 13:14 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: On Wed, 2009-09-23 at 19:07 +0300, Heikki Linnakangas wrote: Rather than keep the numHeldLocks counters per-proc in proc array, I think it would be simpler to have a single (or one per lock partition)

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Heikki Linnakangas
In XidInMVCCSnapshot: if (snapshot-takenDuringRecovery) { /* * If the snapshot contains full subxact data, the fastest way to check * things is just to compare the given XID against both subxact XIDs and * top-level

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Simon Riggs
On Fri, 2009-09-25 at 15:50 +0300, Heikki Linnakangas wrote: Hang on, isn't this 180 degrees backwards? A witty riposte escapes me; yes, the if test is !correct. What I find amazing is that it passed the test where I put it doing make installcheck in an infinite loop for a long time. I guess

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Alvaro Herrera
Simon Riggs wrote: What I find amazing is that it passed the test where I put it doing make installcheck in an infinite loop for a long time. I guess that means the regression tests hardly touch the concurrency code at all, which now I think about it makes sense but I still find that very

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Simon Riggs
On Fri, 2009-09-25 at 16:30 -0400, Alvaro Herrera wrote: Simon Riggs wrote: What I find amazing is that it passed the test where I put it doing make installcheck in an infinite loop for a long time. I guess that means the regression tests hardly touch the concurrency code at all, which

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Andrew Dunstan
Alvaro Herrera wrote: Simon Riggs wrote: What I find amazing is that it passed the test where I put it doing make installcheck in an infinite loop for a long time. I guess that means the regression tests hardly touch the concurrency code at all, which now I think about it makes sense but

Re: [HACKERS] Hot Standby 0.2.1

2009-09-25 Thread Alvaro Herrera
Andrew Dunstan wrote: Alvaro Herrera wrote: Try installcheck-parallel, which should be a bit better. It's probably not yet good enough though because it always runs the same tests concurrently. It is also quite easy to set up your own schedule. ... except you have to be careful with

Re: [HACKERS] Hot Standby 0.2.1

2009-09-24 Thread Heikki Linnakangas
Heikki Linnakangas wrote: The problem becomes a lot easier if we accept that it's OK to have a lock included in the running-xacts snapshot and also appear in a XLOG_RELATION_LOCK record later. The standby should handle that gracefully already. If we just remove RecoveryInfoLock, that can

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Heikki Linnakangas
Looking at the way cache invalidations are handled in two-phase transactions, it would be simpler if we store the shared cache invalidation messages in the twophase state file header, like we store deleted relations and subxids. This allows them to be copied to the COMMIT_PREPARED WAL record, so

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Heikki Linnakangas
The logic in the lock manager to track the number of held AccessExclusiveLocks (with ProcArrayIncrementNumHeldLocks and ProcArrayDecrementNumHeldLocks) seems to be broken. I added an Assertion into ProcArrayDecrementNumHeldLocks: --- a/src/backend/storage/ipc/procarray.c +++

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Simon Riggs
On Wed, 2009-09-23 at 11:13 +0300, Heikki Linnakangas wrote: Looking at the way cache invalidations are handled in two-phase transactions, it would be simpler if we store the shared cache invalidation messages in the twophase state file header, like we store deleted relations and subxids.

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Simon Riggs
On Wed, 2009-09-23 at 12:07 +0300, Heikki Linnakangas wrote: it highlights that we need be careful to avoid putting any extra work into the normal recovery path. Otherwise bugs in hot standby related code can cause crash recovery to fail. Excellent point. I will put in additional protective

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Simon Riggs
On Wed, 2009-09-23 at 12:07 +0300, Heikki Linnakangas wrote: seems to be broken Agreed. Patch withdrawn for correction and rework. Nothing serious, but not much point doing further testing to all current issues resolved. Tracking of issues raised and later solved via Wiki. -- Simon Riggs

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2009-09-23 at 11:13 +0300, Heikki Linnakangas wrote: I note that we don't emit RunningXacts after a shutdown checkpoint. So if recovery starts at a shutdown checkpoint, we don't let read-only backends in until the first online checkpoint. Could we treat a shutdown

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Simon Riggs wrote: On Wed, 2009-09-23 at 11:13 +0300, Heikki Linnakangas wrote: I note that we don't emit RunningXacts after a shutdown checkpoint. So if recovery starts at a shutdown checkpoint, we don't let read-only backends in until the first online checkpoint.

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Jeff Janes
On Tue, Sep 22, 2009 at 10:02 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Heikki Linnakangas escribió: Simon Riggs wrote: On Mon, 2009-09-21 at 19:42 -0700, Jeff Janes wrote: jjanes=# begin; BEGIN jjanes=# lock table pgbench_history in access exclusive mode; LOCK TABLE

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: Unfortunately, isolation level serializable is not truly serializable. Usually it is good enough, but when it isn't good enough and you need an explicit table lock (a very rare but not nonexistent situation), I think it should either lock the table in

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2009-09-23 at 12:07 +0300, Heikki Linnakangas wrote: seems to be broken Agreed. Looking at the relation lock stuff a bit more... When someone tries to acquire an AccessExclusiveLock, but can't get it immediately, we sleep while holding RecoveryInfoLock. That

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Josh Berkus
Simon, Patch withdrawn for correction and rework. Nothing serious, but not much point doing further testing to all current issues resolved. :-( Good thing we went for 4 CFs. Is there a GIT branch of Simon's current working version up somewhere? -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Heikki Linnakangas
Josh Berkus wrote: Patch withdrawn for correction and rework. Nothing serious, but not much point doing further testing to all current issues resolved. :-( Good thing we went for 4 CFs. I think we should try to hammer this in in this commitfest. None of the issues found this far are too

Re: [HACKERS] Hot Standby 0.2.1

2009-09-23 Thread Josh Berkus
Heikki, I think we should try to hammer this in in this commitfest. None of the issues found this far are too serious, nothing that requires major rewrites. It would certainly be valuable to get users testing it starting with Alpha2 instead of waiting 2 months. -- Josh Berkus PostgreSQL

Re: [HACKERS] Hot Standby 0.2.1

2009-09-22 Thread Heikki Linnakangas
Simon Riggs wrote: On Mon, 2009-09-21 at 14:01 +0100, Simon Riggs wrote: On Mon, 2009-09-21 at 13:50 +0300, Heikki Linnakangas wrote: is this that we seem to be missing conflict resolution for GiST index tuples deleted by the kill_prior_tuples mechanism. Unless I'm missing something, we

Re: [HACKERS] Hot Standby 0.2.1

2009-09-22 Thread Simon Riggs
On Mon, 2009-09-21 at 19:42 -0700, Jeff Janes wrote: On Tue, Sep 15, 2009 at 2:41 PM, Simon Riggs si...@2ndquadrant.com wrote: OK, here is the latest version of the Hot Standby patchset. This is about version 30+ by now, but we should regard this as 0.2.1 Patch against CVS HEAD (now):

Re: [HACKERS] Hot Standby 0.2.1

2009-09-22 Thread Heikki Linnakangas
Simon Riggs wrote: On Mon, 2009-09-21 at 19:42 -0700, Jeff Janes wrote: jjanes=# begin; BEGIN jjanes=# lock table pgbench_history in access exclusive mode; LOCK TABLE jjanes=# select count(*) from pgbench_history; count 519104 (1 row) jjanes=# select count(*) from

Re: [HACKERS] Hot Standby 0.2.1

2009-09-22 Thread Simon Riggs
On Tue, 2009-09-22 at 11:04 +0300, Heikki Linnakangas wrote: By me, yes. WAL replay does not require a table lock to progress. Any changes are protected with block-level locks. It does acquire a table lock and cancel conflicting queries when it is about to replay something that would

Re: [HACKERS] Hot Standby 0.2.1

2009-09-22 Thread Heikki Linnakangas
In testing, it looks like there's still something wrong with the subtransaction handling. I created a test function to create a large number of subtransactions: CREATE LANGUAGE plpgsql; CREATE TABLE bar (id int4); CREATE OR REPLACE FUNCTION subxids (n integer) RETURNS void LANGUAGE plpgsql AS $$

Re: [HACKERS] Hot Standby 0.2.1

2009-09-22 Thread Simon Riggs
On Tue, 2009-09-22 at 12:53 +0300, Heikki Linnakangas wrote: In testing, it looks like there's still something wrong with the subtransaction handling. I created a test function to create a large number of subtransactions: OK, looking at this now. Thanks for the report. -- Simon Riggs

Re: [HACKERS] Hot Standby 0.2.1

2009-09-22 Thread Simon Riggs
On Tue, 2009-09-22 at 12:53 +0300, Heikki Linnakangas wrote: It looks like the standby tries to remove XID 4323 from the known-assigned hash table, but it's not there because it was removed and set in pg_subtrans by an XLOG_XACT_ASSIGNMENT record earlier. I guess we should just not throw an

Re: [HACKERS] Hot Standby 0.2.1

2009-09-22 Thread Simon Riggs
On Tue, 2009-09-22 at 12:53 +0300, Heikki Linnakangas wrote: In testing, it looks like there's still something wrong with the subtransaction handling. I created a test function to create a large number of subtransactions: CREATE LANGUAGE plpgsql; CREATE TABLE bar (id int4); CREATE OR

Re: [HACKERS] Hot Standby 0.2.1

2009-09-22 Thread Alvaro Herrera
Heikki Linnakangas escribió: Simon Riggs wrote: On Mon, 2009-09-21 at 19:42 -0700, Jeff Janes wrote: jjanes=# begin; BEGIN jjanes=# lock table pgbench_history in access exclusive mode; LOCK TABLE jjanes=# select count(*) from pgbench_history; count 519104 (1 row)

Re: [HACKERS] Hot Standby 0.2.1

2009-09-22 Thread Bruce Momjian
Simon Riggs wrote: OK, here is the latest version of the Hot Standby patchset. This is about version 30+ by now, but we should regard this as 0.2.1 Patch against CVS HEAD (now): clean apply, compile, no known bugs. OVERVIEW Anyone who is interested in how the hot standby behaves should

Re: [HACKERS] Hot Standby 0.2.1

2009-09-21 Thread Heikki Linnakangas
Simon Riggs wrote: OK, here is the latest version of the Hot Standby patchset. This is about version 30+ by now, but we should regard this as 0.2.1 Patch against CVS HEAD (now): clean apply, compile, no known bugs. Thanks! Attached is some minor comment and fixes, and some dead code removal.

Re: [HACKERS] Hot Standby 0.2.1

2009-09-21 Thread Simon Riggs
On Mon, 2009-09-21 at 13:50 +0300, Heikki Linnakangas wrote: The only bug I've found ! is this that we seem to be missing conflict resolution for GiST index tuples deleted by the kill_prior_tuples mechanism. Unless I'm missing something, we need similar handling there that we have in

Re: [HACKERS] Hot Standby 0.2.1

2009-09-21 Thread Simon Riggs
On Mon, 2009-09-21 at 13:50 +0300, Heikki Linnakangas wrote: The documentation talks about setting and checking default_transaction_read_only, but I think it doesn't say anything about transaction_read_only, which I find odd. This in particular: Users will be able to tell whether their

Re: [HACKERS] Hot Standby 0.2.1

2009-09-21 Thread Simon Riggs
On Mon, 2009-09-21 at 14:01 +0100, Simon Riggs wrote: On Mon, 2009-09-21 at 13:50 +0300, Heikki Linnakangas wrote: is this that we seem to be missing conflict resolution for GiST index tuples deleted by the kill_prior_tuples mechanism. Unless I'm missing something, we need similar

Re: [HACKERS] Hot Standby 0.2.1

2009-09-21 Thread Robert Haas
On Mon, Sep 21, 2009 at 9:01 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-09-21 at 13:50 +0300, Heikki Linnakangas wrote: The only bug I've found ! Yeah, wow. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Hot Standby 0.2.1

2009-09-21 Thread Bruce Momjian
Simon Riggs wrote: OK, here is the latest version of the Hot Standby patchset. This is about version 30+ by now, but we should regard this as 0.2.1 Patch against CVS HEAD (now): clean apply, compile, no known bugs. Wow, great! Simon has allowed us to pass a great milestone in Postgres

Re: [HACKERS] Hot Standby 0.2.1

2009-09-21 Thread Jeff Janes
On Tue, Sep 15, 2009 at 2:41 PM, Simon Riggs si...@2ndquadrant.com wrote: OK, here is the latest version of the Hot Standby patchset. This is about version 30+ by now, but we should regard this as 0.2.1 Patch against CVS HEAD (now): clean apply, compile, no known bugs. OVERVIEW You can

Re: [HACKERS] Hot Standby 0.2.1

2009-09-18 Thread Simon Riggs
On Thu, 2009-09-17 at 19:01 -0400, Robert Haas wrote: I'm going to put the index-only scans aside for now to focus on hot standby and streaming replication. Both are big patches, so there's plenty of work in those two alone, and not only for me. What is the best way to attack this?

Re: [HACKERS] Hot Standby 0.2.1

2009-09-18 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Also, stepping back from me personally, should we try to assign some additional reviewers to these patches? Is there some way we can divide up review tasks among multiple people so that we're not repeating each others work? Thoughts appreciated,

Re: [HACKERS] Hot Standby 0.2.1

2009-09-18 Thread Jeff Janes
On Tue, Sep 15, 2009 at 2:41 PM, Simon Riggs si...@2ndquadrant.com wrote: OK, here is the latest version of the Hot Standby patchset. This is about version 30+ by now, but we should regard this as 0.2.1 Patch against CVS HEAD (now): clean apply, compile, no known bugs. Hi Simon, Is there

Re: [HACKERS] Hot Standby 0.2.1

2009-09-18 Thread Simon Riggs
On Fri, 2009-09-18 at 07:23 -0700, Jeff Janes wrote: On Tue, Sep 15, 2009 at 2:41 PM, Simon Riggs si...@2ndquadrant.com wrote: OK, here is the latest version of the Hot Standby patchset. This is about version 30+ by now, but we should regard this as 0.2.1

Re: [HACKERS] Hot Standby 0.2.1

2009-09-18 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Fri, 2009-09-18 at 07:23 -0700, Jeff Janes wrote: Is there a reason that you remove the WAL_DEBUG shown below? WAL_DEBUG is not removed by the patch, though that section of code is removed, as you observe. I recall an earlier bug report by

Re: [HACKERS] Hot Standby 0.2.1

2009-09-18 Thread Simon Riggs
On Fri, 2009-09-18 at 11:14 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Fri, 2009-09-18 at 07:23 -0700, Jeff Janes wrote: Is there a reason that you remove the WAL_DEBUG shown below? WAL_DEBUG is not removed by the patch, though that section of code is removed,

Re: [HACKERS] Hot Standby 0.2.1

2009-09-18 Thread Marcos Luis Ortiz Valmaseda
Enviados: Jueves, 17 de Septiembre 2009 20:53:24 GMT -10:00 Hawai Asunto: Re: [HACKERS] Hot Standby 0.2.1 On Thu, 2009-09-17 at 19:01 -0400, Robert Haas wrote: I'm going to put the index-only scans aside for now to focus on hot standby and streaming replication. Both are big patches, so there's

Re: [HACKERS] Hot Standby 0.2.1

2009-09-17 Thread Heikki Linnakangas
Robert Haas wrote: On Wed, Sep 16, 2009 at 6:05 PM, Josh Berkus j...@agliodbs.com wrote: Now that Simon has submitted this, can some of the heavy-hitters here review it? Heikki? Nobody's name is next to it. I don't think anyone is planning to ignore this patch, but it wasn't included in

Re: [HACKERS] Hot Standby 0.2.1

2009-09-17 Thread Simon Riggs
On Thu, 2009-09-17 at 09:54 +0300, Heikki Linnakangas wrote: This is a pretty small CommitFest, so there shouldn't be any shortage of reviewers, though Heikki's time may be stretched a little thin, since Streaming Replication is also in the queue, and he is working on index-only scans.

Re: [HACKERS] Hot Standby 0.2.1

2009-09-17 Thread Robert Haas
On Thu, Sep 17, 2009 at 2:54 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Robert Haas wrote: On Wed, Sep 16, 2009 at 6:05 PM, Josh Berkus j...@agliodbs.com wrote: Now that Simon has submitted this, can some of the heavy-hitters here review it?  Heikki? Nobody's name is

Re: [HACKERS] Hot Standby 0.2.1

2009-09-16 Thread Bernd Helmle
--On 15. September 2009 22:41:59 +0100 Simon Riggs si...@2ndquadrant.com wrote: http://wiki.postgresql.org/images/0/01/Hot_Standby_Recovery_Functions.pdf This doesn't work for me, it seems the correct link is http://wiki.postgresql.org/images/1/10/Hot_Standby_Recovery_Functions.pdf ?

Re: [HACKERS] Hot Standby 0.2.1

2009-09-16 Thread Josh Berkus
All, Now that Simon has submitted this, can some of the heavy-hitters here review it? Heikki? Nobody's name is next to it. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Hot Standby 0.2.1

2009-09-16 Thread Robert Haas
On Wed, Sep 16, 2009 at 6:05 PM, Josh Berkus j...@agliodbs.com wrote: Now that Simon has submitted this, can some of the heavy-hitters here review it?  Heikki? Nobody's name is next to it. I don't think anyone is planning to ignore this patch, but it wasn't included in the first round of

Re: [HACKERS] Hot Standby 0.2.1

2009-09-15 Thread David Fetter
On Tue, Sep 15, 2009 at 10:41:59PM +0100, Simon Riggs wrote: OK, here is the latest version of the Hot Standby patchset. This is about version 30+ by now, but we should regard this as 0.2.1 Patch against CVS HEAD (now): clean apply, compile, no known bugs. Kudos Cheers, David. --

Re: [HACKERS] Hot Standby, max_connections and max_prepared_transactions

2009-09-04 Thread Simon Riggs
On Fri, 2009-09-04 at 01:25 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Thu, 2009-09-03 at 22:22 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: I propose we just accept that both max_connections and max_prepared_transactions need to be set correctly for

Re: [HACKERS] Hot Standby, max_connections and max_prepared_transactions

2009-09-04 Thread Robert Haas
On Thu, Sep 3, 2009 at 5:50 PM, Simon Riggssi...@2ndquadrant.com wrote: On Thu, 2009-09-03 at 22:22 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: I propose we just accept that both max_connections and max_prepared_transactions need to be set correctly for recovery to work. This will

Re: [HACKERS] Hot Standby, max_connections and max_prepared_transactions

2009-09-04 Thread Simon Riggs
On Fri, 2009-09-04 at 09:26 -0400, Robert Haas wrote: On Thu, Sep 3, 2009 at 5:50 PM, Simon Riggssi...@2ndquadrant.com wrote: On Thu, 2009-09-03 at 22:22 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: I propose we just accept that both max_connections and

Re: [HACKERS] Hot Standby, max_connections and max_prepared_transactions

2009-09-04 Thread Simon Riggs
On Fri, 2009-09-04 at 09:26 -0400, Robert Haas wrote: Hopefully you're planning to keep that part, as it would be a shame if I had done it for nothing. Not promising anything in that regard, but I appreciate your efforts to assist. I doubt it was wasted effort in any sense. It will certainly

Re: [HACKERS] Hot Standby, max_connections and max_prepared_transactions

2009-09-03 Thread Heikki Linnakangas
Simon Riggs wrote: I propose we just accept that both max_connections and max_prepared_transactions need to be set correctly for recovery to work. This will make the state transitions more robust and it will avoid spurious and hard to test error messages. Any objections to me removing this

Re: [HACKERS] Hot Standby, max_connections and max_prepared_transactions

2009-09-03 Thread Simon Riggs
On Thu, 2009-09-03 at 22:22 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: I propose we just accept that both max_connections and max_prepared_transactions need to be set correctly for recovery to work. This will make the state transitions more robust and it will avoid spurious and

Re: [HACKERS] Hot Standby, max_connections and max_prepared_transactions

2009-09-03 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Thu, 2009-09-03 at 22:22 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: I propose we just accept that both max_connections and max_prepared_transactions need to be set correctly for recovery to work. This will make the state transitions more

Re: [HACKERS] hot standby - further cleanup of recovery procs stuff

2009-08-31 Thread Heikki Linnakangas
Robert Haas wrote: I've made a few further cleanups to the hot standby patch: Thanks! I am not sure why we have a single GUC to size both the number of PGPROC structures we allow and the size of UnobservedXids. A read-only slave might only need to allow a few connections for reporting

Re: [HACKERS] Hot Standby, conflict cache

2009-08-31 Thread Simon Riggs
On Mon, 2009-08-31 at 15:50 +0300, Heikki Linnakangas wrote: The conflict cache code is broken I've already removed that code from the version I'm working on as mentioned on another thread, for the same reasons you just mentioned. I think that the conflict options require more discussion and

Re: [HACKERS] hot standby - merged up to CVS HEAD

2009-08-27 Thread Simon Riggs
On Mon, 2009-08-17 at 11:19 +0300, Heikki Linnakangas wrote: I think there's a race condition in the way LogCurrentRunningXacts() is called at the end of checkpoint. This can happen in the master: 1. Checkpoint starts 2. Transaction 123 begins, and does some updates 3. Checkpoint ends.

Re: [HACKERS] hot standby - merged up to CVS HEAD

2009-08-27 Thread David Fetter
On Thu, Aug 27, 2009 at 07:08:28PM +0100, Simon Riggs wrote: On Mon, 2009-08-17 at 11:19 +0300, Heikki Linnakangas wrote: I think there's a race condition in the way LogCurrentRunningXacts() is called at the end of checkpoint. This can happen in the master: 1. Checkpoint starts 2.

Re: [HACKERS] hot standby - merged up to CVS HEAD

2009-08-20 Thread Robert Haas
On Thu, Aug 20, 2009 at 1:55 AM, Heikki Linnakangasheikki.linnakan...@enterprisedb.com wrote: When that is replayed, ProcArrayUpdateTransactions() will zap the unobserved xids array with the list that includes XID 123, even though we already saw a commit record for it. I looked at this a

Re: [HACKERS] hot standby - merged up to CVS HEAD

2009-08-19 Thread Heikki Linnakangas
Robert Haas wrote: On Mon, Aug 17, 2009 at 6:50 AM, Robert Haasrobertmh...@gmail.com wrote: I think there's a race condition in the way LogCurrentRunningXacts() is called at the end of checkpoint. This can happen in the master: 1. Checkpoint starts 2. Transaction 123 begins, and does some

Re: [HACKERS] hot standby - merged up to CVS HEAD

2009-08-17 Thread Heikki Linnakangas
Robert Haas wrote: I had some review comments I was hoping to get responses to, in the section beginning with A few other comments based on a preliminary reading of this patch: http://archives.postgresql.org/pgsql-hackers/2009-07/msg00854.php Having read the patch now, here's a one issue in

Re: [HACKERS] hot standby - merged up to CVS HEAD

2009-08-17 Thread Robert Haas
On Mon, Aug 17, 2009 at 4:19 AM, Heikki Linnakangasheikki.linnakan...@enterprisedb.com wrote: Robert Haas wrote: I had some review comments I was hoping to get responses to, in the section beginning with A few other comments based on a preliminary reading of this patch:

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Greg Smith
On Tue, 11 Aug 2009, Dimitri Fontaine wrote: We should somehow provide a default archive and restore command integrated into the main product, so that it's as easy as turning it 'on' in the configuration for users to have something trustworthy: PostgreSQL will keep past logs into a

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: *scratches head* I don't really know how you COULD pick a safe default location. Presumably any location that's in the default postgresql.conf file would be under $PGDATA, which kind of defeats the purpose of the whole thing. In other words,

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Josh Berkus
All, The other reason is what I think Greg Smith was mentioning -- simplifying the process of grabbing a usable PITR backup for novice users. That seems like it has merit. While we're at this, can we add xlog_location as a file-location GUC? It seems inconsistent that we're still requiring

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: While we're at this, can we add xlog_location as a file-location GUC? That was proposed and rejected quite a long time ago. We don't *want* people to be able to just change a GUC and have their xlog go somewhere else, because of the foot-gun potential. You

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Josh Berkus
Tom, That was proposed and rejected quite a long time ago. We don't *want* people to be able to just change a GUC and have their xlog go somewhere else, because of the foot-gun potential. You need to be sure that the existing WAL files get moved over when you do something like that, and

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: That was proposed and rejected quite a long time ago. We don't *want* people to be able to just change a GUC and have their xlog go somewhere else, because of the foot-gun potential. You need to be sure that the existing WAL files get moved over when you

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Josh Berkus
Now admittedly it's not hard to screw yourself with a careless manual move of xlog, either. But at least the database didn't hand you a knob that invites clueless frobbing. So really rather than a GUC we should have a utility for moving the xlog. -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: Now admittedly it's not hard to screw yourself with a careless manual move of xlog, either. But at least the database didn't hand you a knob that invites clueless frobbing. So really rather than a GUC we should have a utility for moving the xlog. Yeah,

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Josh Berkus
Yeah, that would work. Although it would probably take as much verbiage to document the utility as it does to document how to do it manually. Yes, but it would *feel* less hackish to sysadmins and DBAs, and make them more confident about moving the xlogs. Getting it to work on windows will

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Andrew Dunstan
Josh Berkus wrote: Yeah, that would work. Although it would probably take as much verbiage to document the utility as it does to document how to do it manually. Yes, but it would *feel* less hackish to sysadmins and DBAs, and make them more confident about moving the xlogs. Getting it

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-13 Thread Jaime Casanova
On Thu, Aug 13, 2009 at 1:49 PM, Josh Berkusj...@agliodbs.com wrote: Yeah, that would work.  Although it would probably take as much verbiage to document the utility as it does to document how to do it manually. Yes, but it would *feel* less hackish to sysadmins and DBAs, and make them more

Re: [HACKERS] Hot standby?

2009-08-12 Thread Greg Stark
On Wed, Aug 12, 2009 at 4:19 AM, Robert Haasrobertmh...@gmail.com wrote: On Tue, Aug 11, 2009 at 9:44 PM, Greg Starkgsst...@mit.edu wrote: No! This is *not* what hot standby means, at least not in the Oracle world. I'm perplexed by this.  For example: http://en.wikipedia.org/wiki/Hot_standby

Re: [HACKERS] Hot standby?

2009-08-12 Thread Joshua Tolley
On Tue, Aug 11, 2009 at 11:19:18PM -0400, Robert Haas wrote: On Tue, Aug 11, 2009 at 9:44 PM, Greg Starkgsst...@mit.edu wrote: On Tue, Aug 11, 2009 at 10:13 PM, Robert Haasrobertmh...@gmail.com wrote: On Tue, Aug 11, 2009 at 4:07 PM, Josh Berkusj...@agliodbs.com wrote: So really, the

Re: [HACKERS] Hot standby?

2009-08-12 Thread Kevin Grittner
Greg Stark gsst...@mit.edu wrote: I'm not actually certain we can handle streaming synchronous mode to a standby slave. Does the slave need to have connections enabled to handle feeding wal sync status to the master? I thought there were major concerns about the interaction of those

Re: [HACKERS] Hot standby?

2009-08-12 Thread Bruce Momjian
Mark Mielke wrote: To further confuse things, the temperature might apply to only a particular aspect of the solution. For example, hot swappable disk drives are drives that probably do sit on a shelf until they are needed, and the hot aspect only implies that the server does not need to

Re: [HACKERS] Hot standby?

2009-08-12 Thread Markus Wanner
Robert Haas wrote: I have also long argued that Synchronous Replication should really be called Streaming Replication. +1 Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-11 Thread Magnus Hagander
On Tuesday, August 11, 2009, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: 2009/8/11 Robert Haas robertmh...@gmail.com We should probably have a separate discussion about what the least committable unit would be for this patch.  I wonder if it might be sufficient to provide

Re: [HACKERS] Hot standby?

2009-08-11 Thread Robert Haas
On Tue, Aug 11, 2009 at 5:30 AM, Peter Eisentrautpete...@gmx.net wrote: What is hot and standby about the proposed hot standby feature? Absolutely nothing. It's horribly misnamed. I have also long argued that Synchronous Replication should really be called Streaming Replication. Perhaps it

Re: [HACKERS] Hot standby?

2009-08-11 Thread Matheus Ricardo Espanhol
Am I off? What other definition of terms justifies the description of hot standby? I think that Hot Standby is associated with the high WAL recovery capacity. In my opinion, is a good term to symbolizes the superiority compared with Warm Standby. -- Matheus Ricardo Espanhol

Re: [HACKERS] Hot standby?

2009-08-11 Thread Bruce Momjian
Robert Haas wrote: On Tue, Aug 11, 2009 at 5:30 AM, Peter Eisentrautpete...@gmx.net wrote: What is hot and standby about the proposed hot standby feature? OK, so it is warm slave. Absolutely nothing. It's horribly misnamed. I have also long argued that Synchronous Replication should

Re: [HACKERS] Hot standby?

2009-08-11 Thread Kevin Grittner
Bruce Momjian br...@momjian.us wrote: OK, so it is warm slave. That is technically accurate, given the preceding definitions, but it has disturbing connotations. Enough so, in my view, to merit getting a little more creative in the naming. How about warm replica? Other ideas? I agree

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-11 Thread Fujii Masao
Hi, On Tue, Aug 11, 2009 at 3:33 PM, Magnus Hagandermag...@hagander.net wrote: We should probably have a separate discussion about what the least committable unit would be for this patch.  I wonder if it might be sufficient to provide a facility for streaming WAL, plus a standalone tool for

Re: [HACKERS] Hot standby?

2009-08-11 Thread David E. Wheeler
On Aug 11, 2009, at 5:32 AM, Bruce Momjian wrote: OK, so it is warm slave. I suggest steaming servant. Or tepid assistant. David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hot standby and synchronous replication status

2009-08-11 Thread Fujii Masao
Hi, On Tue, Aug 11, 2009 at 1:25 PM, Robert Haasrobertmh...@gmail.com wrote: But just to kick off the discussion, here is Heikki's review of Synch Rep on 7/15: http://archives.postgresql.org/pgsql-hackers/2009-07/msg00913.php I think the key phrases in this review are I believe we have

Re: [HACKERS] Hot standby?

2009-08-11 Thread Gianni Ciolli
On Tue, Aug 11, 2009 at 12:30:58PM +0300, Peter Eisentraut wrote: warm - If the first node dies, the replacement node needs to do some work to get ready, but it's a lot quicker than cold. hot - If the first node dies, the replacement node can take over immediately. For example, I'd say

Re: [HACKERS] Hot standby?

2009-08-11 Thread Joshua D. Drake
On Tue, 2009-08-11 at 08:12 -0700, David E. Wheeler wrote: On Aug 11, 2009, at 5:32 AM, Bruce Momjian wrote: OK, so it is warm slave. I suggest steaming servant. Or tepid assistant. We can't use those, I think they are on the list for Ubuntu. Joshua D. Drake David -- PostgreSQL -

Re: [HACKERS] Hot standby?

2009-08-11 Thread David Fetter
On Tue, Aug 11, 2009 at 08:56:38AM -0500, Kevin Grittner wrote: Bruce Momjian br...@momjian.us wrote: OK, so it is warm slave. That is technically accurate, given the preceding definitions, but it has disturbing connotations. Enough so, in my view, to merit getting a little more

<    1   2   3   4   5   6   7   8   9   10   >