Re: [HACKERS] Problem with pg_upgrade?

2011-03-31 Thread Bruce Momjian
Bruce Momjian wrote: Bruce Momjian wrote: It does seem possible that that could happen, but I'm not sure exactly what would be causing autovacuum to fire in the first place. It wouldn't have to be triggered by the anti-wraparound machinery - if the table appeared to be in need of

Re: [HACKERS] Problem with pg_upgrade?

2011-03-31 Thread Heikki Linnakangas
On 31.03.2011 17:55, Bruce Momjian wrote: I will work on code to allow autovacuum_max_workers to be set to zero in HEAD and 9.0, and have pg_upgrade us that. We've intentionally not allowed the user to disable anti-wraparound autovacuum before. Do we really want to allow it now for the sake

Re: [HACKERS] Problem with pg_upgrade?

2011-03-31 Thread Bruce Momjian
Heikki Linnakangas wrote: On 31.03.2011 17:55, Bruce Momjian wrote: I will work on code to allow autovacuum_max_workers to be set to zero in HEAD and 9.0, and have pg_upgrade us that. We've intentionally not allowed the user to disable anti-wraparound autovacuum before. Do we really want

Re: [HACKERS] Problem with pg_upgrade?

2011-03-31 Thread Robert Haas
On Thu, Mar 31, 2011 at 11:32 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote:  I think the maintenance overhead of an invisible variable is too much. A simple GUC or command-line switch isn't much code. I like the idea of a command-line switch. -- Robert Haas EnterpriseDB:

Re: [HACKERS] Problem with pg_upgrade?

2011-03-31 Thread Bruce Momjian
Robert Haas wrote: On Thu, Mar 31, 2011 at 11:32 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: ?I think the maintenance overhead of an invisible variable is too much. A simple GUC or command-line switch isn't much code. I like the idea of a command-line switch. If

Re: [HACKERS] Problem with pg_upgrade?

2011-03-31 Thread Robert Haas
On Thu, Mar 31, 2011 at 12:11 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Thu, Mar 31, 2011 at 11:32 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: ?I think the maintenance overhead of an invisible variable is too much. A simple GUC or command-line

Re: [HACKERS] Problem with pg_upgrade?

2011-03-31 Thread Gurjeet Singh
On Wed, Mar 30, 2011 at 10:57 AM, Bruce Momjian br...@momjian.us wrote: I am hearing only second-hand reports of this problem through Rhodiumtoad on IRC. I don't have IRC access this week If the firewalls allow port 80, you can use Freenode's web interface: webchat.freenode.net Regards, --

Re: [HACKERS] Problem with pg_upgrade?

2011-03-31 Thread Bruce Momjian
Robert Haas wrote: On Thu, Mar 31, 2011 at 12:11 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Thu, Mar 31, 2011 at 11:32 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: ?I think the maintenance overhead of an invisible variable is too much. A

Re: [HACKERS] Problem with pg_upgrade?

2011-03-31 Thread Bruce Momjian
Bruce Momjian wrote: Robert Haas wrote: On Thu, Mar 31, 2011 at 12:11 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Thu, Mar 31, 2011 at 11:32 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: ?I think the maintenance overhead of an invisible

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Bruce Momjian
Alvaro Herrera wrote: Excerpts from Jeff Davis's message of mar mar 29 21:27:34 -0300 2011: On Tue, 2011-03-29 at 15:52 -0400, Bruce Momjian wrote: Does anyone have any other suggestions on how to make sure autovacuum does not run in freeze mode? Can you run in single user mode? I

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Peter Eisentraut
On ons, 2011-03-30 at 10:57 -0400, Bruce Momjian wrote: Also, I am unclear if this is really our bug. At least one of the systems was on Ubuntu/Debian, and they might both have been, and I know Debian changes our source code. Where can I find a copy of the diffs they have made?

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Bruce Momjian
Peter Eisentraut wrote: On ons, 2011-03-30 at 10:57 -0400, Bruce Momjian wrote: Also, I am unclear if this is really our bug. At least one of the systems was on Ubuntu/Debian, and they might both have been, and I know Debian changes our source code. Where can I find a copy of the diffs

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Robert Haas
On Wed, Mar 30, 2011 at 10:57 AM, Bruce Momjian br...@momjian.us wrote: I think we have three options:        o  find if the use of autovacuum_freeze_max_age is safe, or make           it safe        o  document that autovacuum_naptime always happens before           autovacuum does anything

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Jeff Davis
On Wed, 2011-03-30 at 16:46 -0400, Robert Haas wrote: I don't really understand why this is an issue in the first place, though. Surely we must be setting the XID counter on the new cluster to match the one on the old cluster, and migrating the relfrozenxid and datfrozenxid settings, so why

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Peter Eisentraut
On ons, 2011-03-30 at 15:39 -0400, Bruce Momjian wrote: Peter Eisentraut wrote: On ons, 2011-03-30 at 10:57 -0400, Bruce Momjian wrote: Also, I am unclear if this is really our bug. At least one of the systems was on Ubuntu/Debian, and they might both have been, and I know Debian

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Bruce Momjian
Robert Haas wrote: On Wed, Mar 30, 2011 at 10:57 AM, Bruce Momjian br...@momjian.us wrote: I think we have three options: ? ? ? ?o ?find if the use of autovacuum_freeze_max_age is safe, or make ? ? ? ? ? it safe ? ? ? ?o ?document that autovacuum_naptime always happens before ? ? ? ?

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Bruce Momjian
Bruce Momjian wrote: First, I am not sure it is a problem, but based on the IRC reports I felt I should ask here for confirmation. Here is a sample pg_dump output: CREATE TABLE sample ( x integer ); -- For binary upgrade, set relfrozenxid.

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Bruce Momjian
Bruce Momjian wrote: I wonder if the fact that these people never reported the bug means there were doing something odd with their servers. I just updated the C comment about what we are doing: * Using autovacuum=off disables cleanup vacuum and analyze, but * freeze vacuums

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Robert Haas
On Wed, Mar 30, 2011 at 5:27 PM, Bruce Momjian br...@momjian.us wrote: First, I am not sure it is a problem, but based on the IRC reports I felt I should ask here for confirmation.  Here is a sample pg_dump output:        CREATE TABLE sample (            x integer        );        -- For

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Bruce Momjian
Robert Haas wrote: So, we set the cluster xid while we do this schema-only restore. ?I belive it might be possible for autovacuum to run while the schema is restored, see an empty table, and set the relfrozenxid to be the current xid, when in fact we are about to put a heap file in place

Re: [HACKERS] Problem with pg_upgrade?

2011-03-30 Thread Bruce Momjian
Bruce Momjian wrote: It does seem possible that that could happen, but I'm not sure exactly what would be causing autovacuum to fire in the first place. It wouldn't have to be triggered by the anti-wraparound machinery - if the table appeared to be in need of vacuuming, then we'd vacuum

[HACKERS] Problem with pg_upgrade?

2011-03-29 Thread Bruce Momjian
I have gotten two reports via IRC that months after using 9.0 pg_upgrade, some of the clog files have been removed while there is still data in the table needing those clog files. These reports came to me through Rhodiumtoad who analyzed the systems. Looking at pg_upgrade, I am concerned that

Re: [HACKERS] Problem with pg_upgrade?

2011-03-29 Thread Jeff Davis
On Tue, 2011-03-29 at 15:52 -0400, Bruce Momjian wrote: Does anyone have any other suggestions on how to make sure autovacuum does not run in freeze mode? Can you run in single user mode? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Problem with pg_upgrade?

2011-03-29 Thread Alvaro Herrera
Excerpts from Jeff Davis's message of mar mar 29 21:27:34 -0300 2011: On Tue, 2011-03-29 at 15:52 -0400, Bruce Momjian wrote: Does anyone have any other suggestions on how to make sure autovacuum does not run in freeze mode? Can you run in single user mode? I asked the same thing.

Re: [HACKERS] Problem with pg_upgrade?

2011-03-29 Thread Jeff Davis
On Tue, 2011-03-29 at 21:43 -0300, Alvaro Herrera wrote: I think it would be better to have some sort of option to disable autovacuum completely which would be used only during pg_upgrade. Sounds reasonable to me. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-03-09 Thread Bruce Momjian
Robert Treat wrote: On Mon, Feb 28, 2011 at 3:42 AM, Magnus Hagander mag...@hagander.net wrote: On Mon, Feb 28, 2011 at 06:21, Tom Lane t...@sss.pgh.pa.us wrote: Robert Treat r...@xzilla.net writes: Did anything ever come of this discussion? I think it's a TODO --- nothing done about it

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-02-28 Thread Magnus Hagander
On Mon, Feb 28, 2011 at 06:21, Tom Lane t...@sss.pgh.pa.us wrote: Robert Treat r...@xzilla.net writes: Did anything ever come of this discussion? I think it's a TODO --- nothing done about it as yet, AFAIR. On one of the databases I was upgrading, I ran into a similar problem with roles

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-02-28 Thread Robert Treat
On Mon, Feb 28, 2011 at 3:42 AM, Magnus Hagander mag...@hagander.net wrote: On Mon, Feb 28, 2011 at 06:21, Tom Lane t...@sss.pgh.pa.us wrote: Robert Treat r...@xzilla.net writes: Did anything ever come of this discussion? I think it's a TODO --- nothing done about it as yet, AFAIR. On one

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-02-27 Thread Robert Treat
On Thu, Jan 6, 2011 at 10:08 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: We could modify pg_dump to emit RESET AUTHORIZATION in --binary-upgrade mode.  I am unclear if that might cause some other problems though. I finally figured out

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-02-27 Thread Tom Lane
Robert Treat r...@xzilla.net writes: Did anything ever come of this discussion? I think it's a TODO --- nothing done about it as yet, AFAIR. On one of the databases I was upgrading, I ran into a similar problem with roles that are set as roles. The problem seems to stem from pg_dumpall

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-06 Thread Florian Pflug
On Jan6, 2011, at 04:13 , Bruce Momjian wrote: Robert Haas wrote: On Wed, Jan 5, 2011 at 9:44 PM, Bruce Momjian br...@momjian.us wrote: I think pg_dumpall would have failed with this setup too, so I don't see this as a pg_upgrade bug, nor something that I am willing to risk adding to

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-06 Thread Florian Pflug
On Jan6, 2011, at 05:08 , Tom Lane wrote: I think an appropriate response would be to prevent ALTER DATABASE SET ROLE. I really cannot believe that there are any situations where that's a good idea. I explained up-thread why, in my situation, doing this *is* a perfectly good idea. You have

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-06 Thread Bruce Momjian
Florian Pflug wrote: On Jan6, 2011, at 04:13 , Bruce Momjian wrote: Robert Haas wrote: On Wed, Jan 5, 2011 at 9:44 PM, Bruce Momjian br...@momjian.us wrote: I think pg_dumpall would have failed with this setup too, so I don't see this as a pg_upgrade bug, nor something that I am willing

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-06 Thread Bruce Momjian
Robert Haas wrote: On Wed, Jan 5, 2011 at 11:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Or we could take the approach somebody was just espousing about Our job is to prevent the user from *accidentally* shooting themselves in the foot. I don't see how you can compare those two cases

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-06 Thread Robert Haas
On Thu, Jan 6, 2011 at 1:59 PM, Bruce Momjian br...@momjian.us wrote: Well, if everyone who logs in gets the same username, you can easily conclude that trying to dump/restore the database will cause problems if you have objects in there that are not owned by that user. I can't, and neither

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-06 Thread Bruce Momjian
Robert Haas wrote: On Thu, Jan 6, 2011 at 1:59 PM, Bruce Momjian br...@momjian.us wrote: Well, if everyone who logs in gets the same username, you can easily conclude that trying to dump/restore the database will cause problems if you have objects in there that are not owned by that user.

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-06 Thread Robert Haas
On Thu, Jan 6, 2011 at 3:54 PM, Bruce Momjian br...@momjian.us wrote: Well, we usually tell people to restore as super-user, particularly pg_dumpall, but in this case, it is impossible.  Certainly pg_upgrade requires it, which is the root of the problem. True. Although it's not really

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-06 Thread Bruce Momjian
bruce wrote: Robert Haas wrote: On Thu, Jan 6, 2011 at 1:59 PM, Bruce Momjian br...@momjian.us wrote: Well, if everyone who logs in gets the same username, you can easily conclude that trying to dump/restore the database will cause problems if you have objects in there that are not

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-06 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: We could modify pg_dump to emit RESET AUTHORIZATION in --binary-upgrade mode. I am unclear if that might cause some other problems though. I finally figured out what was really bugging me about that proposal: it's a one-shot hack for fixing one problem

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-06 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: We could modify pg_dump to emit RESET AUTHORIZATION in --binary-upgrade mode. I am unclear if that might cause some other problems though. I finally figured out what was really bugging me about that proposal: it's a one-shot hack for

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-05 Thread Bruce Momjian
Florian Pflug wrote: Hi I've just ran into a problem while upgrading from 8.4 to 9.0. pg_upgrade aborted during the step Adding support functions to new cluster with ERROR: permission denied for language c error. Unfortunately, the log didn't include the name of the database where the

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-05 Thread Robert Haas
On Wed, Jan 5, 2011 at 9:44 PM, Bruce Momjian br...@momjian.us wrote: I think pg_dumpall would have failed with this setup too, so I don't see this as a pg_upgrade bug, nor something that I am willing to risk adding to pg_upgrade. If adding RESET SESSION AUTHORIZATION fixes the bug, I think we

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-05 Thread Bruce Momjian
Robert Haas wrote: On Wed, Jan 5, 2011 at 9:44 PM, Bruce Momjian br...@momjian.us wrote: I think pg_dumpall would have failed with this setup too, so I don't see this as a pg_upgrade bug, nor something that I am willing to risk adding to pg_upgrade. If adding RESET SESSION AUTHORIZATION

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jan 5, 2011 at 9:44 PM, Bruce Momjian br...@momjian.us wrote: I think pg_dumpall would have failed with this setup too, so I don't see this as a pg_upgrade bug, nor something that I am willing to risk adding to pg_upgrade. If adding RESET

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-05 Thread Bruce Momjian
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jan 5, 2011 at 9:44 PM, Bruce Momjian br...@momjian.us wrote: I think pg_dumpall would have failed with this setup too, so I don't see this as a pg_upgrade bug, nor something that I am willing to risk adding to pg_upgrade.

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-05 Thread Andrew Dunstan
On 01/05/2011 11:08 PM, Tom Lane wrote: If they want to deliberately shoot themselves in the foot by hosing the login system like that, it's not our job to prevent it. But it's not our job to try to work around it, either. I think this is especially true in this

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-01-05 Thread Robert Haas
On Wed, Jan 5, 2011 at 11:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Or we could take the approach somebody was just espousing about Our job is to prevent the user from *accidentally* shooting themselves in the foot. I don't see how you can compare those two cases with a straight face. In the

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2010-12-13 Thread Tom Lane
Florian Pflug f...@phlo.org writes: On Dec13, 2010, at 00:16 , Robert Haas wrote: And in fact it strikes me that we might not have much choice about how to fix this. I think we are not going to retroactively change the behavior of ALTER DATABASE .. SET ROLE in a released version, but yet we

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2010-12-13 Thread Florian Pflug
On Dec13, 2010, at 16:40 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: On Dec13, 2010, at 00:16 , Robert Haas wrote: And in fact it strikes me that we might not have much choice about how to fix this. I think we are not going to retroactively change the behavior of ALTER DATABASE ..

[HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2010-12-12 Thread Florian Pflug
Hi I've just ran into a problem while upgrading from 8.4 to 9.0. pg_upgrade aborted during the step Adding support functions to new cluster with ERROR: permission denied for language c error. Unfortunately, the log didn't include the name of the database where the error occurred, so it took

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2010-12-12 Thread Tom Lane
Florian Pflug f...@phlo.org writes: pg_upgrade aborted during the step Adding support functions to new cluster with ERROR: permission denied for language c error. Unfortunately, the log didn't include the name of the database where the error occurred, so it took me a while to figure out

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2010-12-12 Thread Florian Pflug
On Dec12, 2010, at 17:01 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: pg_upgrade aborted during the step Adding support functions to new cluster with ERROR: permission denied for language c error. Unfortunately, the log didn't include the name of the database where the error

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2010-12-12 Thread Robert Haas
On Sun, Dec 12, 2010 at 11:01 AM, Tom Lane t...@sss.pgh.pa.us wrote: This is about like arguing that pg_dump and pg_upgrade should still work after you've done delete from pg_proc;.  Superusers are assumed to know what they're doing and not break fundamental operations. No, it isn't like that

Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2010-12-12 Thread Florian Pflug
On Dec13, 2010, at 00:16 , Robert Haas wrote: And in fact it strikes me that we might not have much choice about how to fix this. I think we are not going to retroactively change the behavior of ALTER DATABASE .. SET ROLE in a released version, but yet we do, I think, want to make pg_upgrade