Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Peter Eisentraut
On mån, 2009-12-14 at 08:54 +, Simon Riggs wrote: Wednesday because that seemed a good delay to allow review. Josh and I had discussed the value of getting patch into Alpha3, so that was my wish and aim. I'm not aware of any particular date for end of commitfest, though possibly you are

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Heikki Linnakangas
Simon Riggs wrote: Enclose latest version of Hot Standby. This is the basic patch, with no must-fix issues and no known bugs. Further additions will follow after commit, primarily around usability, which will include additional control functions for use in testing. Various thoughts discussed

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Magnus Hagander
On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: * Please remove any spurious whitespace.  git diff --color makes them stand out like a sore thumb, in red. (pgindent will fix them but always better to fix them before committing, IMO). +1 in general,

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
Thanks for the further review, much appreciated. On Mon, 2009-12-14 at 11:54 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: Enclose latest version of Hot Standby. This is the basic patch, with no must-fix issues and no known bugs. Further additions will follow after commit, primarily

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: * Please remove any spurious whitespace. git diff --color makes them stand out like a sore thumb, in red. (pgindent will fix them but always

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Fujii Masao
On Mon, Dec 14, 2009 at 8:07 PM, Simon Riggs si...@2ndquadrant.com wrote: * Please remove any spurious whitespace.  git diff --color makes them stand out like a sore thumb, in red. (pgindent will fix them but always better to fix them before committing, IMO). What is your definition of

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 6:11 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: * Please remove any spurious whitespace.  git diff --color makes

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 06:21 -0500, Robert Haas wrote: On Mon, Dec 14, 2009 at 6:11 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: * Please

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 6:35 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14 at 06:21 -0500, Robert Haas wrote: On Mon, Dec 14, 2009 at 6:11 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: On Mon, Dec 14, 2009 at 10:54,

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Magnus Hagander
On Mon, Dec 14, 2009 at 12:51, Robert Haas robertmh...@gmail.com wrote: On Mon, Dec 14, 2009 at 6:35 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14 at 06:21 -0500, Robert Haas wrote: On Mon, Dec 14, 2009 at 6:11 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 11:07 +, Simon Riggs wrote: Thanks for the further review, much appreciated. On Mon, 2009-12-14 at 11:54 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: Enclose latest version of Hot Standby. * LockAcquireExtended needs a function comment. Or at least

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 7:22 AM, Simon Riggs si...@2ndquadrant.com wrote: I am now unable to push these changes to the shared repo. What is happening? Perhaps you need to run cvs update on your local copy? (I find this flavor the most useful: cvs -q update -d YMMV.) If that's not it, error

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 06:51 -0500, Robert Haas wrote: On Mon, Dec 14, 2009 at 6:35 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14 at 06:21 -0500, Robert Haas wrote: On Mon, Dec 14, 2009 at 6:11 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14 at 11:09 +0100,

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 07:33 -0500, Robert Haas wrote: On Mon, Dec 14, 2009 at 7:22 AM, Simon Riggs si...@2ndquadrant.com wrote: I am now unable to push these changes to the shared repo. What is happening? Perhaps you need to run cvs update on your local copy? (I find this flavor the

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Magnus Hagander
On Mon, Dec 14, 2009 at 13:47, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14 at 07:33 -0500, Robert Haas wrote: On Mon, Dec 14, 2009 at 7:22 AM, Simon Riggs si...@2ndquadrant.com wrote: I am now unable to push these changes to the shared repo. What is happening? Perhaps you

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 7:42 AM, Simon Riggs si...@2ndquadrant.com wrote: There are no changes in this patch that are purely whitespace changes. Those were removed prior to patch submission. This is all about code I am adding or changing. My additions may disrupt their patches, but not because

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 09:32 -0500, Robert Haas wrote: If every patch perfectly matched the pgident style, then the pgindent run would change nothing and we would all be VERY happy. I've made all whitespace changes to the patch. I understand the reason for acting as you suggest, but we either

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: If we are going to run pgindent anyway, what is the point? Perhaps it would take less time to run this than to argue the point?: sed -e 's/[ \t][ \t]*$//' -e 's/ *\t/\t/g' * -Kevin -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 13:56 +0100, Magnus Hagander wrote: Same issue can be it in git - did you do a git pull before? You may need merging with what's on there, and for that to work you must pull before you can push. Found some merge conflicts and resolved them. I did fetch and merge at

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Dec 14, 2009 at 6:35 AM, Simon Riggs si...@2ndquadrant.com wrote: Why is (1) important, and if it is important, why is it being mentioned only now? Are we saying that all previous reviewers of my work (and others') removed these without ever

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 10:08 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14 at 09:32 -0500, Robert Haas wrote: If every patch perfectly matched the pgident style, then the pgindent run would change nothing and we would all be VERY happy. I've made all whitespace changes to

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Alvaro Herrera
Tom Lane escribió: The whole thing would be a lot easier if someone would put together an easily-installable version of pgindent. Bruce has posted the patches he uses but I don't know what version of indent they're against... And we're still unclear on the typedef list that's going to be

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Heikki Linnakangas
Simon Riggs wrote: On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: * Please remove any spurious whitespace. git diff --color makes them stand out like a sore thumb, in red. (pgindent will

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 09:14 -0600, Kevin Grittner wrote: Simon Riggs si...@2ndquadrant.com wrote: If we are going to run pgindent anyway, what is the point? Perhaps it would take less time to run this than to argue the point?: sed -e 's/[ \t][ \t]*$//' -e 's/ *\t/\t/g' * Not

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread David Fetter
On Mon, Dec 14, 2009 at 11:09:49AM +0100, Magnus Hagander wrote: On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: * Please remove any spurious whitespace.  git diff --color makes them stand out like a sore thumb, in red. (pgindent will fix them

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 10:49 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2009-12-14 at 09:14 -0600, Kevin Grittner wrote: Simon Riggs si...@2ndquadrant.com wrote: If we are going to run pgindent anyway, what is the point? Perhaps it would take less time to run this than to argue

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Bruce Momjian
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Dec 14, 2009 at 6:35 AM, Simon Riggs si...@2ndquadrant.com wrote: Why is (1) important, and if it is important, why is it being mentioned only now? Are we saying that all previous reviewers of my work (and others') removed

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Greg Smith
Heikki Linnakangas wrote: I note that if it was easy to run pgindent yourself on a patch before committing/submitting, we wouldn't need to have this discussion. I don't know hard it is to get it working right, but I recall I tried once and gave up. What sort of problems did you run into? --

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Greg Stark
On Mon, Dec 14, 2009 at 11:07 AM, Simon Riggs si...@2ndquadrant.com wrote: * vacuum_defer_cleanup_age is very hard to tune. You'll need an estimate on your transaction rate to begin with. Do we really want this setting in its current form? Does it make sense as PGC_USERSET, as if one backend

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Heikki Linnakangas
Greg Smith wrote: Heikki Linnakangas wrote: I note that if it was easy to run pgindent yourself on a patch before committing/submitting, we wouldn't need to have this discussion. I don't know hard it is to get it working right, but I recall I tried once and gave up. What sort of problems

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Heikki Linnakangas
Simon Riggs wrote: On Mon, 2009-12-14 at 11:54 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: * Are you planning to remove the recovery_connections setting before release? The documentation makes it sound like it's a temporary hack that we're not really sure is needed at all. That's not

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 18:02 +, Greg Stark wrote: It doesn't seem any more wrong than using hash indexes right after recovery, crash recovery or otherwise. It's certainly broken, but I don't see much value in a partial fix. The bottom line is that hash indexes should be WAL-logged.

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Sun, 2009-12-13 at 22:25 +, Simon Riggs wrote: * Which exact tables are we talking about: just pg_class and the shared catalogs? Everything else is in pg_class, so if we can find it we're OK? formrdesc() tells me the list of nailed relations is: pg_database, pg_class, pg_attribute,

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: * Disallow clustering system relations. This will definitely NOT work * for shared relations (we have no way to update pg_class rows in other * databases), nor for nailed-in-cache relations (the relfilenode values * for those are hardwired, see

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 20:32 +0200, Heikki Linnakangas wrote: * You removed this comment from KnownAssignedXidsInit: - /* -* XXX: We should check that we don't exceed maxKnownAssignedXids. -* Even though the hash table might hold a few more entries than that, -* we use

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 13:48 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: * Disallow clustering system relations. This will definitely NOT work * for shared relations (we have no way to update pg_class rows in other * databases), nor for nailed-in-cache relations (the

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Mon, 2009-12-14 at 13:48 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: * Disallow clustering system relations. This will definitely NOT work * for shared relations (we have no way to update pg_class rows in other * databases),

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 14:14 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Mon, 2009-12-14 at 13:48 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: * Disallow clustering system relations. This will definitely NOT work * for shared relations (we have

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Mon, 2009-12-14 at 20:32 +0200, Heikki Linnakangas wrote: I have ensured that they are always the same size, by definition, so no need to check. How did you ensure that? The hash table has no hard size limit. The hash table is in shared memory

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 15:24 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Mon, 2009-12-14 at 20:32 +0200, Heikki Linnakangas wrote: I have ensured that they are always the same size, by definition, so no need to check. How did you ensure that? The hash table has

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: What is the best way of restricting the hash table to a maximum size? There is nothing in dynahash that will enforce a maximum size against calling code that's not cooperating; and I'll resist any attempt to add such a thing, because it would create a

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 16:39 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: What is the best way of restricting the hash table to a maximum size? There is nothing in dynahash that will enforce a maximum size against calling code that's not cooperating; and I'll resist any

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 11:44 +0200, Peter Eisentraut wrote: On mån, 2009-12-14 at 08:54 +, Simon Riggs wrote: Wednesday because that seemed a good delay to allow review. Josh and I had discussed the value of getting patch into Alpha3, so that was my wish and aim. I'm not aware of

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Greg Smith
Simon Riggs wrote: On Mon, 2009-12-14 at 11:44 +0200, Peter Eisentraut wrote: On mån, 2009-12-14 at 08:54 +, Simon Riggs wrote: Wednesday because that seemed a good delay to allow review. Josh and I had discussed the value of getting patch into Alpha3, so that was my wish and aim.

Re: [HACKERS] Hot Standby, release candidate?

2009-12-13 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: * NonTransactionalInvalidation logging has been removed following review, but AFAICS that means VACUUM FULL doesn't work correctly on catalog tables, which regrettably will be the only ones still standing even after we apply VFI patch. Did I

Re: [HACKERS] Hot Standby, release candidate?

2009-12-13 Thread Simon Riggs
On Sun, 2009-12-13 at 15:45 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: * NonTransactionalInvalidation logging has been removed following review, but AFAICS that means VACUUM FULL doesn't work correctly on catalog tables, which regrettably will be the only ones still

Re: [HACKERS] Hot standby, recent changes

2009-12-07 Thread Heikki Linnakangas
Simon Riggs wrote: On Sun, 2009-12-06 at 17:26 -0500, Robert Haas wrote: For what it's worth, this doesn't seem particularly unlikely or unusual to me. I don't know many people who shutdown both nodes of a highly available application at the same time. If they did, I wouldn't expect them

Re: [HACKERS] Hot standby, recent changes

2009-12-07 Thread Simon Riggs
On Mon, 2009-12-07 at 10:02 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Sun, 2009-12-06 at 17:26 -0500, Robert Haas wrote: For what it's worth, this doesn't seem particularly unlikely or unusual to me. I don't know many people who shutdown both nodes of a highly available

Re: [HACKERS] Hot standby, misc issues

2009-12-06 Thread Simon Riggs
On Sat, 2009-12-05 at 22:56 +0200, Heikki Linnakangas wrote: So that RecordKnownAssignedTransactionIds() call needs to be put back. OK BTW, if you want to resurrect the check in KnownAssignedXidsRemove(), you also need to not complain before you reach the running-xacts record and open up

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: 2. Allow RULEs ON INSERT, ON UPDATE and ON DELETE iff they generate only SELECT statements that act INSTEAD OF the actual event. This affects any read-only transaction, not just hot standby, so I think this should be discussed and

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: 1. The XLogFlush() call you added to dbase_redo doesn't help where it is. You need to call XLogFlush() after the *commit* record of the DROP DATABASE. The idea is minimize the window where the files have already been deleted but the

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: 3. The Out of lock mem killer in StandbyAcquireAccessExclusiveLock is quite harsh. It aborts all read-only transactions. It should be enough to kill just one random one, or maybe the one that's holding most locks. Also, if there

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 11:20 +, Simon Riggs wrote: On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: 3. The Out of lock mem killer in StandbyAcquireAccessExclusiveLock is quite harsh. It aborts all read-only transactions. It should be enough to kill just one random one, or

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 10:51 +, Simon Riggs wrote: 5. You removed this comment from KnownAssignedXidsAdd: - /* -* XXX: We should check that we don't exceed maxKnownAssignedXids. -* Even though the hash table might hold a few more entries than that, -* we use

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Heikki Linnakangas
Simon Riggs wrote: On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: 4. Need to handle the case where master is started up with wal_standby_info=true, shut down, and restarted with wal_standby_info=false, while the standby server runs continuously. And the code in StartupXLog() to

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 20:32 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: 4. Need to handle the case where master is started up with wal_standby_info=true, shut down, and restarted with wal_standby_info=false, while the

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Robert Haas
On Sun, Dec 6, 2009 at 3:06 PM, Simon Riggs si...@2ndquadrant.com wrote: On Sun, 2009-12-06 at 20:32 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: 4. Need to handle the case where master is started up with

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Dimitri Fontaine
Le 6 déc. 2009 à 23:26, Robert Haas a écrit : Consider this scenario: 0. You have a master and a standby configured properly, and up and running. 1. You shut down master for some reason. 2. You restart standby. For some reason. Maybe by accident, or you want to upgrade minor version or

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 17:26 -0500, Robert Haas wrote: For what it's worth, this doesn't seem particularly unlikely or unusual to me. I don't know many people who shutdown both nodes of a highly available application at the same time. If they did, I wouldn't expect them to complain they

Re: [HACKERS] Hot standby, misc issues

2009-12-05 Thread Simon Riggs
On Fri, 2009-12-04 at 10:23 +0200, Heikki Linnakangas wrote: @Heikki: Why is error checking in KnownAssignedXidsRemove() #ifdef'd out?? It's explained in the comment: /* XXX: This can still happen: If a transaction with a subtransaction * that haven't been reported yet aborts, and no

Re: [HACKERS] Hot standby, misc issues

2009-12-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Fri, 2009-12-04 at 10:23 +0200, Heikki Linnakangas wrote: @Heikki: Why is error checking in KnownAssignedXidsRemove() #ifdef'd out?? It's explained in the comment: /* XXX: This can still happen: If a transaction with a subtransaction * that haven't been reported yet

Re: [HACKERS] Hot Standby remaining issues

2009-12-04 Thread Heikki Linnakangas
Regarding this item from the wiki page: The standby delay is measured as current timestamp - timestamp of last replayed commit record. If there's little activity in the master, that can lead to surprising results. For example, imagine that max_standby_delay is set to 8 hours. The standby is

Re: [HACKERS] Hot Standby remaining issues

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 10:37 +0200, Heikki Linnakangas wrote: Regarding this item from the wiki page: The standby delay is measured as current timestamp - timestamp of last replayed commit record. If there's little activity in the master, that can lead to surprising results. For example,

Re: [HACKERS] Hot Standby remaining issues

2009-12-04 Thread Heikki Linnakangas
Simon Riggs wrote: On Fri, 2009-12-04 at 10:37 +0200, Heikki Linnakangas wrote: Regarding this item from the wiki page: The standby delay is measured as current timestamp - timestamp of last replayed commit record. If there's little activity in the master, that can lead to surprising

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Sat, 2009-11-21 at 20:20 +0200, Heikki Linnakangas wrote: VACUUM FULL does a peculiar hack: once it's done moving tuples, and before it truncates the relation, it calls RecordTransactionCommit to mark the transaction as committed in clog and WAL, but the transaction is still kept open in

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Heikki Linnakangas
Simon Riggs wrote: I've just reviewed the VACUUM FULL patch to see if it does all we need it to do, i.e. does the removal of HS code match the new VF patch. Oh, good! My answer is it doesn't, we will still have the problem noted above for catalog tables. So we still have a must-fix issue for

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 11:22 +0200, Heikki Linnakangas wrote: Could you just mark the transaction as committed when you see the 1st commit record, but leave the XID in the known-assigned list and not release locks? That would emulate pretty closely what happens in the master. OK, works for me.

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 11:22 +0200, Heikki Linnakangas wrote: My answer is it doesn't, we will still have the problem noted above for catalog tables. So we still have a must-fix issue for HS, though that is no barrier to the new VF patch. I think the VACUUM FULL patch will have to address

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Heikki Linnakangas
Simon Riggs wrote: ISTM premature to remove all traces of VF from code. We may yet need it for some reason, especially if doing so creates complex dependencies on important features. Well, it's still in the repository. So modified proposal looks like this 1. (In normal running) Provide

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 13:31 +0200, Heikki Linnakangas wrote: b) seems much simpler to me. OK. Least ugly wins, but she ain't pretty. 2. (In HS recovery) When we see first commit record for the VF xid we commit the transaction in clog, yet maintain locks and KnownAssigned xids 3. (In

Re: [HACKERS] Hot Standby remaining issues

2009-12-04 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: If the system is completely idle, and no WAL is written, we skip xlog switches too, even if archive_timeout is set . It would be pointless to create a stream of WAL files with no content except for the XLOG_SWITCH records. It's

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-03 Thread Simon Riggs
On Wed, 2009-11-25 at 03:12 +, Greg Stark wrote: On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: As long as there's not anything the master actually does differently then I can't see where there'd be any performance testing to do. What's bothering me about this is

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Simon Riggs wrote: @@ -654,10 +656,13 @@ LockAcquire(const LOCKTAG *locktag, elog(PANIC, lock table corrupted); } LWLockRelease(partitionLock); -ereport(ERROR, -

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Simon Riggs
On Wed, 2009-12-02 at 12:49 +0200, Heikki Linnakangas wrote: If a read-only transaction holds a lot of locks, consuming so much lock space that there's none left for the startup process to hold the lock it wants, it will abort and bring down postmaster. The patch attempts to kill any

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2009-12-02 at 12:49 +0200, Heikki Linnakangas wrote: If a read-only transaction holds a lot of locks, consuming so much lock space that there's none left for the startup process to hold the lock it wants, it will abort and bring down postmaster. The patch attempts

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Simon Riggs
On Wed, 2009-12-02 at 16:41 +, Simon Riggs wrote: On Tue, 2009-12-01 at 20:26 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: commit 02c3eadb766201db084b668daa271db4a900adc9 Author: Simon Riggs sri...@ebony.(none) Date: Sat Nov 28 06:23:33 2009 + Added

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Heikki Linnakangas
Simon Riggs wrote: Hmm, what happens if someone enables wal_standby_info in postgresql.conf while the server is shutdown. It would still be a valid starting point in that case. Yeah, true. I'll just make a note, I think. Yeah, a manual (or automatic, if you just wait) checkpoint will

Re: [HACKERS] Hot Standby remaining issues

2009-12-01 Thread Heikki Linnakangas
Simon Riggs wrote: commit 02c3eadb766201db084b668daa271db4a900adc9 Author: Simon Riggs sri...@ebony.(none) Date: Sat Nov 28 06:23:33 2009 + Added wal_standby_info GUC to turn RM_STANDBY_ID messages on/off. Various comments added also. This patch makes it unsafe to start hot

Re: [HACKERS] Hot Standby remaining issues

2009-11-30 Thread Heikki Linnakangas
Simon Riggs wrote: @@ -654,10 +656,13 @@ LockAcquire(const LOCKTAG *locktag, elog(PANIC, lock table corrupted); } LWLockRelease(partitionLock); - ereport(ERROR, -

Re: [HACKERS] Hot Standby and cancelling idle queries

2009-11-27 Thread Simon Riggs
On Thu, 2009-11-26 at 08:30 +0200, Heikki Linnakangas wrote: I suspect you missed the context of this change. It's about the code in tablespc.c, to kill all backends that might have a temporary file in a tablespace that's being dropped. It's not about tuple visibility but temporary files. Got

Re: [HACKERS] Hot Standby remaining issues

2009-11-27 Thread Simon Riggs
On Wed, 2009-11-25 at 13:00 +0200, Heikki Linnakangas wrote: I've put up a wiki page with the issues I see with the patch as it stands. They're roughly categorized by seriousness. http://wiki.postgresql.org/wiki/Hot_Standby_TODO New issues can and probably will still pop up, let's add them

Re: [HACKERS] Hot Standby remaining issues

2009-11-25 Thread Simon Riggs
On Wed, 2009-11-25 at 13:00 +0200, Heikki Linnakangas wrote: I've put up a wiki page with the issues I see with the patch as it stands. They're roughly categorized by seriousness. http://wiki.postgresql.org/wiki/Hot_Standby_TODO New issues can and probably will still pop up, let's add them

Re: [HACKERS] Hot Standby and cancelling idle queries

2009-11-25 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: An idle-in-transaction transaction can also hold a temporary file. Think of an open cursor, for example. Therefore, remove the distinction between CONFLICT_MODE_ERROR and CONFLICT_MODE_ERROR_IF_NOT_IDLE, idle-in-transaction backends need to be killed

Re: [HACKERS] Hot Standby and cancelling idle queries

2009-11-25 Thread Heikki Linnakangas
Simon Riggs wrote: Recent change: An idle-in-transaction transaction can also hold a temporary file. Think of an open cursor, for example. Therefore, remove the distinction between CONFLICT_MODE_ERROR and CONFLICT_MODE_ERROR_IF_NOT_IDLE, idle-in-transaction backends need to be killed too

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Sat, 2009-11-21 at 20:20 +0200, Heikki Linnakangas wrote: That causes some headaches for Hot Standby I say leave HS as it is and we can clean up when we do the VFectomy. It isn't really a headache, the code works easily enough. I agree its ugly and it should eventually be removed. Let's

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Sat, 2009-11-21 at 23:00 +0200, Heikki Linnakangas wrote: Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: There's no equivalent of XLogArchivingActive()? XLogArchivingMode() == false enables us to skip WAL-logging in operations like

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Tom Lane wrote: There's no equivalent of XLogArchivingActive()? We've tried hard to have it just work. But I wonder whether we should have a parameter to allow performance testing on the master? If nobody finds any issues then we can remove it again,

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Greg Stark
On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: As long as there's not anything the master actually does differently then I can't see where there'd be any performance testing to do.  What's bothering me about this is that it seems likely that we'll find places where the

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: Well the only thing that's been discussed is having vacuum require a minimum age before considering a transaction visible to all to reduce the chance of conflicts on cleanup records. [ shrug... ] Call me Cassandra. I am not concerned about what has or has

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Wed, 2009-11-25 at 03:12 +, Greg Stark wrote: On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: As long as there's not anything the master actually does differently then I can't see where there'd be any performance testing to do. What's bothering me about this is

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Greg Stark
On Wed, Nov 25, 2009 at 3:26 AM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark gsst...@mit.edu writes: Well the only thing that's been discussed is having vacuum require a minimum age before considering a transaction visible to all to reduce the chance of conflicts on cleanup records. [

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Itagaki Takahiro
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: So I guess what I'm asking is: Does anyone see any show-stoppers in removing VACUUM FULL, and does anyone want to step up to the plate and promise to do it before release? I'm working on New VACUUM FULL patch, that shrinks tables

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Heikki Linnakangas
Itagaki Takahiro wrote: VACUUM FULL is still reserved for system catalogs in my patch because we cannot modify relfilenodes for the catalog tables. Do you have solutions for it? Tom had an idea on that: http://archives.postgresql.org/message-id/19750.1252094...@sss.pgh.pa.us -- Heikki

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Heikki Linnakangas
Greg Smith wrote: Heikki Linnakangas wrote: So I guess what I'm asking is: Does anyone see any show-stoppers in removing VACUUM FULL Here's the disclaimers attached to the new VACUUM REPLACE implementation from Itagaki: We still need traditional VACUUM FULL behavior for system catalog

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Hannu Krosing
On Tue, 2009-11-24 at 09:24 +0200, Heikki Linnakangas wrote: Greg Smith wrote: Heikki Linnakangas wrote: So I guess what I'm asking is: Does anyone see any show-stoppers in removing VACUUM FULL Here's the disclaimers attached to the new VACUUM REPLACE implementation from Itagaki:

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Greg Smith
Heikki Linnakangas wrote: So I guess what I'm asking is: Does anyone see any show-stoppers in removing VACUUM FULL Here's the disclaimers attached to the new VACUUM REPLACE implementation from Itagaki: We still need traditional VACUUM FULL behavior for system catalog because we cannot change

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: So I guess what I'm asking is: Does anyone see any show-stoppers in removing VACUUM FULL, and does anyone want to step up to the plate and promise to do it before release? I don't see much problem with rejecting VAC FULL in a HS

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: So I guess what I'm asking is: Does anyone see any show-stoppers in removing VACUUM FULL, and does anyone want to step up to the plate and promise to do it before release? I don't see much problem with rejecting

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: I don't see much problem with rejecting VAC FULL in a HS master, whether or not it gets removed altogether. Why not just do that rather than write a lot of kluges? Hmm. At the moment, no action is required in the

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: I don't see much problem with rejecting VAC FULL in a HS master, whether or not it gets removed altogether. Why not just do that rather than write a lot of kluges? Hmm. At the moment, no action

<    1   2   3   4   5   6   7   8   9   10   >