Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-28 Thread Peter Eisentraut
On tor, 2010-07-15 at 10:24 +0100, Simon Riggs wrote: Patch to reduce lock levels for ALTER TABLE CREATE TRIGGER CREATE RULE Tried this out, but $subject is still the case. The problem is that ATRewriteCatalogs() calls AlterTableCreateToastTable() based on what it thinks the subcommands

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-28 Thread Simon Riggs
On Wed, 2010-07-28 at 15:24 +0300, Peter Eisentraut wrote: On tor, 2010-07-15 at 10:24 +0100, Simon Riggs wrote: Patch to reduce lock levels for ALTER TABLE CREATE TRIGGER CREATE RULE Tried this out, but $subject is still the case. The problem is that ATRewriteCatalogs() calls

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-28 Thread Simon Riggs
On Wed, 2010-07-28 at 15:24 +0300, Peter Eisentraut wrote: On tor, 2010-07-15 at 10:24 +0100, Simon Riggs wrote: Patch to reduce lock levels for ALTER TABLE CREATE TRIGGER CREATE RULE Tried this out, but $subject is still the case. The problem is that ATRewriteCatalogs() calls

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-19 Thread Simon Riggs
On Sun, 2010-07-18 at 22:47 -0400, Robert Haas wrote: But it seems that it's far from clear what to do about it, and it's not the job of this patch to fix it anyway. Agreed. Regarding the actual patch, it looks mostly good. Questions: 1. Why in rewriteSupport.c are we adding a call to

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-19 Thread Robert Haas
On Mon, Jul 19, 2010 at 2:46 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sun, 2010-07-18 at 22:47 -0400, Robert Haas wrote:  But it seems that it's far from clear what to do about it, and it's not the job of this patch to fix it anyway. Agreed. Regarding the actual patch, it looks

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Andres Freund
On Saturday 17 July 2010 09:55:37 Simon Riggs wrote: On Fri, 2010-07-16 at 23:03 +0200, Andres Freund wrote: Sure its not that bad, but at least it needs to get documented imho. Likely others should chime in here ;-) Don't understand you. This is a clear bug in join removal, test case

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Simon Riggs
On Sun, 2010-07-18 at 17:28 +0200, Andres Freund wrote: Unfortunately the same issue exists with constraint exclusion - and we can hardly disable that for serializable transactions... Then I think the fix is to look at the xmin values on all of the tables used during planning and ensure that

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Andres Freund
On Sunday 18 July 2010 18:02:26 Simon Riggs wrote: On Sun, 2010-07-18 at 17:28 +0200, Andres Freund wrote: Unfortunately the same issue exists with constraint exclusion - and we can hardly disable that for serializable transactions... Then I think the fix is to look at the xmin values on

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On Sunday 18 July 2010 18:02:26 Simon Riggs wrote: Then I think the fix is to look at the xmin values on all of the tables used during planning and ensure that we only use constraint-based optimisations in a serializable transaction when our top xmin is

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Andres Freund
On Sunday 18 July 2010 19:20:25 Tom Lane wrote: Andres Freund and...@anarazel.de writes: On Sunday 18 July 2010 18:02:26 Simon Riggs wrote: Then I think the fix is to look at the xmin values on all of the tables used during planning and ensure that we only use constraint-based

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Robert Haas
On Sun, Jul 18, 2010 at 1:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@anarazel.de writes: On Sunday 18 July 2010 18:02:26 Simon Riggs wrote: Then I think the fix is to look at the xmin values on all of the tables used during planning and ensure that we only use

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-17 Thread Simon Riggs
On Fri, 2010-07-16 at 23:03 +0200, Andres Freund wrote: Sure its not that bad, but at least it needs to get documented imho. Likely others should chime in here ;-) Don't understand you. This is a clear bug in join removal, test case attached, a minor rework of your original test case. What

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-17 Thread Simon Riggs
On Fri, 2010-07-16 at 20:45 -0400, Tom Lane wrote: Andres Freund and...@anarazel.de writes: Just to help me: The primary reasons for using SnapshotNow is speed and in some cases correctness (referential integrity). Right? Any other reasons? Well, the main point for system catalog accesses

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
Hi Simon, Your patch implements part of a feature I desire greatly - thanks! Some comments: On Thursday 15 July 2010 11:24:27 Simon Riggs wrote: ! LOCKMODE ! AlterTableGreatestLockLevel(List *cmds) ! { !ListCell *lcmd; !LOCKMODE lockmode = ShareUpdateExclusiveLock; /* default

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
On Friday 16 July 2010 20:41:44 Andres Freund wrote: ! */ !case AT_AddColumn: /* may rewrite heap, in some cases and visible to SELECT */ ! case AT_DropColumn: /* change visible to SELECT

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Simon Riggs
On Fri, 2010-07-16 at 20:41 +0200, Andres Freund wrote: You argue above that you cant change SET [NOT] NULL to be less restrictive because it might change plans - isnt that true for some of the above cases as well? For example UNIQUE/PRIMARY might make join removal possible - which could

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Simon Riggs
On Fri, 2010-07-16 at 21:10 +0200, Andres Freund wrote: On Friday 16 July 2010 20:41:44 Andres Freund wrote: ! */ !case AT_AddColumn: /* may rewrite heap, in some cases and visible to SELECT */ ! case

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
On Friday 16 July 2010 21:12:33 Simon Riggs wrote: On Fri, 2010-07-16 at 20:41 +0200, Andres Freund wrote: You argue above that you cant change SET [NOT] NULL to be less restrictive because it might change plans - isnt that true for some of the above cases as well? For example

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
On Friday 16 July 2010 21:15:44 Simon Riggs wrote: On Fri, 2010-07-16 at 21:10 +0200, Andres Freund wrote: On Friday 16 July 2010 20:41:44 Andres Freund wrote: ! */ !case AT_AddColumn: /* may rewrite heap, in some cases

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Simon Riggs
On Fri, 2010-07-16 at 21:38 +0200, Andres Freund wrote: boom Your test case would still occur in the case where the first query had not been executed against the same table. So the test case illustrates a failing of join removal, not of this patch. -- Simon Riggs www.2ndQuadrant.com

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
On Friday 16 July 2010 22:24:32 Simon Riggs wrote: On Fri, 2010-07-16 at 21:38 +0200, Andres Freund wrote: boom Your test case would still occur in the case where the first query had not been executed against the same table. So the test case illustrates a failing of join removal, not of

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Tom Lane
Andres Freund and...@anarazel.de writes: What could the join removal path (and similar places) *possibly* do against such a case? Without stopping to use SnapshotNow I dont see any way :-( But the planner, along with most of the rest of the backend, *does* use SnapshotNow when examining the

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Robert Haas
On Jul 16, 2010, at 6:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@anarazel.de writes: What could the join removal path (and similar places) *possibly* do against such a case? Without stopping to use SnapshotNow I dont see any way :-( But the planner, along with most of

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
On Saturday 17 July 2010 01:53:24 Robert Haas wrote: On Jul 16, 2010, at 6:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@anarazel.de writes: What could the join removal path (and similar places) *possibly* do against such a case? Without stopping to use SnapshotNow I dont

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Tom Lane
Andres Freund and...@anarazel.de writes: Just to help me: The primary reasons for using SnapshotNow is speed and in some cases correctness (referential integrity). Right? Any other reasons? Well, the main point for system catalog accesses is that you *must* have an up-to-date view of the table

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-15 Thread Simon Riggs
On Thu, 2010-07-08 at 07:16 +0100, Simon Riggs wrote: I'll take my previous patch through to completion now Patch to reduce lock levels for ALTER TABLE CREATE TRIGGER CREATE RULE I've completely re-analyzed the required lock levels for sub-commands, so lock levels can now also be these, if

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-15 Thread Josh Berkus
On 7/7/10 6:04 PM, Cédric Villemain wrote: I just faced production issue where it is impossible to alter table to adjust autovacuum settings in a pg8.4. (5K tps, 260M rows table, lock too much) We could try to resolve the COMMENT ON issue with the same mechanism. What we need is a table lock

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-11 Thread Simon Riggs
On Fri, 2010-07-09 at 15:03 -0400, Robert Haas wrote: On Fri, Jul 9, 2010 at 1:18 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-07-09 at 13:04 -0400, Robert Haas wrote: Tom asked what happens when two transactions attempt to do concurrent actions on the same table. Your

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-10 Thread Bruce Momjian
Robert Haas wrote: On Wed, Jul 7, 2010 at 9:04 PM, C?dric Villemain cedric.villemain.deb...@gmail.com wrote: I assume this did not get done for 9.0. ?Do we want a TODO item? Yes. Added: ? ? ? ?Reduce locking required for ALTER commands I just faced production issue where it is

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-09 Thread Robert Haas
On Thu, Jul 8, 2010 at 5:09 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-07-08 at 06:08 -0400, Robert Haas wrote: On Thu, Jul 8, 2010 at 2:16 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-07-07 at 22:26 -0400, Robert Haas wrote: Rereading the thread, I'm a bit

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-09 Thread Simon Riggs
On Fri, 2010-07-09 at 13:04 -0400, Robert Haas wrote: Tom asked what happens when two transactions attempt to do concurrent actions on the same table. Your response was that we should handle it like CREATE INDEX, and handle the update of the pg_class row non-transactionally. But of course,

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-09 Thread Robert Haas
On Fri, Jul 9, 2010 at 1:18 PM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-07-09 at 13:04 -0400, Robert Haas wrote: Tom asked what happens when two transactions attempt to do concurrent actions on the same table.  Your response was that we should handle it like CREATE INDEX, and

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-08 Thread Simon Riggs
On Wed, 2010-07-07 at 22:26 -0400, Robert Haas wrote: Rereading the thread, I'm a bit confused by why we're proposing to use a SHARE lock; it seems to me that a self-conflicting lock type would simplify things. There's a bunch of discussion on the thread about how to handle pg_class updates

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-08 Thread Robert Haas
On Thu, Jul 8, 2010 at 2:16 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-07-07 at 22:26 -0400, Robert Haas wrote: Rereading the thread, I'm a bit confused by why we're proposing to use a SHARE lock; it seems to me that a self-conflicting lock type would simplify things.  There's a

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-08 Thread Simon Riggs
On Thu, 2010-07-08 at 06:08 -0400, Robert Haas wrote: On Thu, Jul 8, 2010 at 2:16 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-07-07 at 22:26 -0400, Robert Haas wrote: Rereading the thread, I'm a bit confused by why we're proposing to use a SHARE lock; it seems to me that a

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-07 Thread Cédric Villemain
2010/3/3 Bruce Momjian br...@momjian.us: Peter Eisentraut wrote: On m?n, 2010-02-22 at 10:32 -0500, Bruce Momjian wrote: Simon Riggs wrote: On Mon, 2009-10-19 at 12:56 -0300, Alvaro Herrera wrote: Simon Riggs wrote: On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote:

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-07 Thread Robert Haas
On Wed, Jul 7, 2010 at 9:04 PM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: I assume this did not get done for 9.0.  Do we want a TODO item? Yes. Added:        Reduce locking required for ALTER commands I just faced production issue where it is impossible to alter table to

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-03-02 Thread Bruce Momjian
Peter Eisentraut wrote: On m?n, 2010-02-22 at 10:32 -0500, Bruce Momjian wrote: Simon Riggs wrote: On Mon, 2009-10-19 at 12:56 -0300, Alvaro Herrera wrote: Simon Riggs wrote: On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: Tom Lane wrote: Peter Eisentraut

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-03-01 Thread Peter Eisentraut
On mån, 2010-02-22 at 10:32 -0500, Bruce Momjian wrote: Simon Riggs wrote: On Mon, 2009-10-19 at 12:56 -0300, Alvaro Herrera wrote: Simon Riggs wrote: On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes:

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-02-22 Thread Bruce Momjian
Simon Riggs wrote: On Mon, 2009-10-19 at 12:56 -0300, Alvaro Herrera wrote: Simon Riggs wrote: On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Is there a good reason for $subject, other than that the code is

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-10-19 Thread Alvaro Herrera
Simon Riggs wrote: On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Is there a good reason for $subject, other than that the code is entangled with other ALTER TABLE code? I think it could be lower, but

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-10-19 Thread Simon Riggs
On Mon, 2009-10-19 at 12:56 -0300, Alvaro Herrera wrote: Simon Riggs wrote: On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Is there a good reason for $subject, other than that the code is entangled

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-08-08 Thread Simon Riggs
On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Is there a good reason for $subject, other than that the code is entangled with other ALTER TABLE code? I think it could be lower, but it would take nontrivial

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-08-07 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: Is there a good reason for $subject, other than that the code is entangled with other ALTER TABLE code? I think it could be lower, but it would take nontrivial restructuring of the ALTER TABLE support. In particular, consider what happens when you have

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-08-07 Thread Alvaro Herrera
Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Is there a good reason for $subject, other than that the code is entangled with other ALTER TABLE code? I think it could be lower, but it would take nontrivial restructuring of the ALTER TABLE support. In particular, consider what