Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-22 Thread Simon Riggs
On 21 March 2014 23:36, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 21 March 2014 20:58, Noah Misch n...@leadboat.com wrote: It's not the behavior I would choose for a new product, but I can't see benefits sufficient to overturn previous decisions to keep

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-22 Thread Simon Riggs
On 21 March 2014 16:11, Simon Riggs si...@2ndquadrant.com wrote: + * Be careful to ensure this function is called for Tables and Indexes only. + * It is not currently safe to be called for Views because security_barrier + * is listed as an option and so would be allowed to be set at a level

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-22 Thread Jim Nasby
On 2/26/14, 9:15 AM, Simon Riggs wrote: On 26 February 2014 13:38, Andres Freundand...@2ndquadrant.com wrote: Hi, On 2014-02-26 07:32:45 +, Simon Riggs wrote: * This definitely should include isolationtester tests actually performing concurrent ALTER TABLEs. All that's currently

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Simon Riggs
On 21 March 2014 03:45, Noah Misch n...@leadboat.com wrote: On Sat, Mar 08, 2014 at 11:14:30AM +, Simon Riggs wrote: On 7 March 2014 09:04, Simon Riggs si...@2ndquadrant.com wrote: The right thing to do here is to not push to the extremes. If we mess too much with the ruleutil stuff it

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Noah Misch
On Fri, Mar 21, 2014 at 04:11:12PM +, Simon Riggs wrote: On 21 March 2014 03:45, Noah Misch n...@leadboat.com wrote: On Sat, Mar 08, 2014 at 11:14:30AM +, Simon Riggs wrote: Thanks for the review. I'll respond to each point on a later email but looks nothing much major, apart from

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Simon Riggs
On 21 March 2014 17:49, Noah Misch n...@leadboat.com wrote: + * Be careful to ensure this function is called for Tables and Indexes only. + * It is not currently safe to be called for Views because security_barrier + * is listed as an option and so would be allowed to be set at a

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Simon Riggs
On 21 March 2014 03:45, Noah Misch n...@leadboat.com wrote: + * Note that Hot Standby only knows about AccessExclusiveLocks on the master + * so any changes that might affect SELECTs running on standbys need to use + * AccessExclusiveLocks even if you think a lesser lock would do, unless you

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Noah Misch
On Fri, Mar 21, 2014 at 06:53:27PM +, Simon Riggs wrote: On 21 March 2014 17:49, Noah Misch n...@leadboat.com wrote: alter table information_schema.triggers set (security_barrier = true); I find it hard to justify why we accept such a statement. Surely its a bug when the named

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Simon Riggs
On 21 March 2014 20:58, Noah Misch n...@leadboat.com wrote: On Fri, Mar 21, 2014 at 06:53:27PM +, Simon Riggs wrote: On 21 March 2014 17:49, Noah Misch n...@leadboat.com wrote: alter table information_schema.triggers set (security_barrier = true); I find it hard to justify why we

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-21 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 21 March 2014 20:58, Noah Misch n...@leadboat.com wrote: It's not the behavior I would choose for a new product, but I can't see benefits sufficient to overturn previous decisions to keep it. Speechless The key argument for not fixing this is that

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-20 Thread Vik Fearing
On 03/18/2014 11:39 AM, Simon Riggs wrote: On 8 March 2014 11:14, Simon Riggs si...@2ndquadrant.com wrote: On 7 March 2014 09:04, Simon Riggs si...@2ndquadrant.com wrote: The right thing to do here is to not push to the extremes. If we mess too much with the ruleutil stuff it will just be

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-20 Thread Noah Misch
On Sat, Mar 08, 2014 at 11:14:30AM +, Simon Riggs wrote: On 7 March 2014 09:04, Simon Riggs si...@2ndquadrant.com wrote: The right thing to do here is to not push to the extremes. If we mess too much with the ruleutil stuff it will just be buggy. A more considered analysis in a later

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-19 Thread Simon Riggs
On 18 March 2014 21:21, Noah Misch n...@leadboat.com wrote: On Tue, Mar 18, 2014 at 10:39:03AM +, Simon Riggs wrote: On 8 March 2014 11:14, Simon Riggs si...@2ndquadrant.com wrote: Implemented in attached patch, v22 I commend this patch to you for final review; I would like to commit

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-18 Thread Simon Riggs
On 8 March 2014 11:14, Simon Riggs si...@2ndquadrant.com wrote: On 7 March 2014 09:04, Simon Riggs si...@2ndquadrant.com wrote: The right thing to do here is to not push to the extremes. If we mess too much with the ruleutil stuff it will just be buggy. A more considered analysis in a later

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-18 Thread Noah Misch
On Tue, Mar 18, 2014 at 10:39:03AM +, Simon Riggs wrote: On 8 March 2014 11:14, Simon Riggs si...@2ndquadrant.com wrote: Implemented in attached patch, v22 I commend this patch to you for final review; I would like to commit this in a few days. I'm planning to commit this today at

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-09 Thread Vik Fearing
On 03/06/2014 10:47 AM, Simon Riggs wrote: On 5 March 2014 09:28, Simon Riggs si...@2ndquadrant.com wrote: So that returns us to solving the catalog consistency problem in pg_dump and similar applications. No answer, guys, and time is ticking away here. Sorry, I've accumulated several days

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-07 Thread Simon Riggs
On 6 March 2014 22:43, Noah Misch n...@leadboat.com wrote: On Tue, Mar 04, 2014 at 06:50:17PM +0100, Andres Freund wrote: On 2014-03-04 11:40:10 -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: I think this is all too late for 9.4, though. I agree with the feeling that a

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-06 Thread Simon Riggs
On 5 March 2014 09:28, Simon Riggs si...@2ndquadrant.com wrote: So that returns us to solving the catalog consistency problem in pg_dump and similar applications. No answer, guys, and time is ticking away here. I'd like to get a communal solution to this rather than just punting the whole

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-06 Thread Noah Misch
On Tue, Mar 04, 2014 at 06:50:17PM +0100, Andres Freund wrote: On 2014-03-04 11:40:10 -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: I think this is all too late for 9.4, though. I agree with the feeling that a meaningful fix for pg_dump isn't going to get done for

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-06 Thread Simon Riggs
On 6 March 2014 22:43, Noah Misch n...@leadboat.com wrote: Good analysis. The hazards arise when pg_dump uses one of the ruleutils.c deparse worker functions. Ah, good. We're thinking along the same lines. I was already working on this analysis also. I'll complete mine and then compare notes.

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-05 Thread Simon Riggs
On 4 March 2014 21:37, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Mar 4, 2014 at 2:39 PM, Simon Riggs si...@2ndquadrant.com wrote: Your earlier claim that the dump is inconsistent just isn't accurate. We now have MVCC catalogs, so any dump is going to

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Simon Riggs
On 4 March 2014 01:07, Andres Freund and...@2ndquadrant.com wrote: On 2014-03-03 19:15:27 -0500, Tom Lane wrote: Noah Misch n...@leadboat.com writes: Just to be clear, that list is not a commentary on the particular patch at hand. Those are merely the kinds of regressions to look for in a

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Atri Sharma
Good points. In most cases, DDL is applied manually after careful thought, so people seldom dump at the same time they upgrade the database. This is especially true for pg_dump since it captures the logical definition of tables. So most people will be happy with the default locking, but we

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Simon Riggs
On 4 March 2014 08:39, Atri Sharma atri.j...@gmail.com wrote: Good points. In most cases, DDL is applied manually after careful thought, so people seldom dump at the same time they upgrade the database. This is especially true for pg_dump since it captures the logical definition of tables.

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Atri Sharma
If its not the case, the user should be more careful about when he is scheduling backups to so that they dont conflict with DDL changes. That is most certainly the wise choice. I am not too comfortable with exposing the locking type to the user. That may be just me though. Why would

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Simon Riggs
On 4 March 2014 09:31, Simon Riggs si...@2ndquadrant.com wrote: On 4 March 2014 08:39, Atri Sharma atri.j...@gmail.com wrote: Good points. In most cases, DDL is applied manually after careful thought, so people seldom dump at the same time they upgrade the database. This is especially true

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Robert Haas
On Tue, Mar 4, 2014 at 6:57 AM, Simon Riggs si...@2ndquadrant.com wrote: The main impact I see is that this would block VACUUM while pg_dump runs. But then, while pg_dump runs VACUUM is ineffective anyway so perhaps that is no bad thing. Well, a vacuum that's already running when pg_dump

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Atri Sharma
I'd really like to see us find a way to apply some version of this patch. I was in favor of the concept 3 years ago when we did this the first time, and I've subsequently done quite a bit of work (viz., MVCC catalog snapshots) to eliminate the main objection that was raised at that time. But

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Simon Riggs
On 4 March 2014 12:18, Robert Haas robertmh...@gmail.com wrote: On Tue, Mar 4, 2014 at 6:57 AM, Simon Riggs si...@2ndquadrant.com wrote: The main impact I see is that this would block VACUUM while pg_dump runs. But then, while pg_dump runs VACUUM is ineffective anyway so perhaps that is no

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Stephen Frost
* Atri Sharma (atri.j...@gmail.com) wrote: If its not the case, the user should be more careful about when he is scheduling backups to so that they dont conflict with DDL changes. I'm not following this as closely as I'd like to, but I wanted to voice my opinion that this is just not acceptable

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Alvaro Herrera
Stephen Frost escribió: * Atri Sharma (atri.j...@gmail.com) wrote: If its not the case, the user should be more careful about when he is scheduling backups to so that they dont conflict with DDL changes. I'm not following this as closely as I'd like to, but I wanted to voice my opinion

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Atri Sharma
On Tue, Mar 4, 2014 at 8:19 PM, Stephen Frost sfr...@snowman.net wrote: * Atri Sharma (atri.j...@gmail.com) wrote: If its not the case, the user should be more careful about when he is scheduling backups to so that they dont conflict with DDL changes. I'm not following this as closely as

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Robert Haas
On Tue, Mar 4, 2014 at 10:17 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: One possible idea would be to create a new lock level which conflicts with DDL changes but not with regular operation including dumps; so it wouldn't self-conflict but it would conflict with ShareUpdateExclusive.

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: One concern is schema changes that make a dump unrestorable, for instance if there's a foreign key relationship between tables A and B, Yeah. Ideally, what pg_dump would produce would be a consistent snapshot of the database state as of its

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Yeah. Ideally, what pg_dump would produce would be a consistent snapshot of the database state as of its transaction snapshot time. We have always had that guarantee so far as user data was concerned, but it's been shaky (and getting worse) so far as

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I think this is all too late for 9.4, though. I agree with the feeling that a meaningful fix for pg_dump isn't going to get done for 9.4. So that leaves us with the alternatives of (1) put off the lock-strength-reduction patch for another year; (2) push

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: I don't have too much of an issue with the above, but I would like to have us figure out a solution to the deadlock problem with parallel pg_dump. The issue arises when pg_dump gets an AccessShareLock and then another process attempts to acquire an

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: I don't have too much of an issue with the above, but I would like to have us figure out a solution to the deadlock problem with parallel pg_dump. The issue arises when pg_dump gets an AccessShareLock and then

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Alvaro Herrera
Tom Lane escribió: I'd like to have lock strength reduction as much as anybody, but it can't come at the price of reduction of reliability. Can we have at least a cut-down version of it? If we can just reduce the lock level required for ALTER TABLE / VALIDATE, that would be an enormous

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Atri Sharma
On Tue, Mar 4, 2014 at 10:20 PM, Stephen Frost sfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: I don't have too much of an issue with the above, but I would like to have us figure out a solution to the deadlock problem with

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-04 Thread Andres Freund
On 2014-03-04 11:40:10 -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: I think this is all too late for 9.4, though. I agree with the feeling that a meaningful fix for pg_dump isn't going to get done for 9.4. So that leaves us with the alternatives of (1) put off the

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Simon Riggs
On 4 March 2014 16:27, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: One concern is schema changes that make a dump unrestorable, for instance if there's a foreign key relationship between tables A and B, Yeah. Ideally, what pg_dump would produce would be

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Andres Freund
On March 4, 2014 8:39:55 PM CET, Simon Riggs si...@2ndquadrant.com wrote: On 4 March 2014 16:27, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: One concern is schema changes that make a dump unrestorable, for instance if there's a foreign key relationship

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-04 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-03-04 11:40:10 -0500, Tom Lane wrote: I don't care for (2). I'd like to have lock strength reduction as much as anybody, but it can't come at the price of reduction of reliability. I am sorry, but I think this is vastly overstating the

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Robert Haas
On Tue, Mar 4, 2014 at 2:39 PM, Simon Riggs si...@2ndquadrant.com wrote: On 4 March 2014 16:27, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: One concern is schema changes that make a dump unrestorable, for instance if there's a foreign key relationship

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Mar 4, 2014 at 2:39 PM, Simon Riggs si...@2ndquadrant.com wrote: Your earlier claim that the dump is inconsistent just isn't accurate. We now have MVCC catalogs, so any dump is going to see a perfectly consistent set of data plus DDL. OK the

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Josh Berkus
On 03/04/2014 11:43 AM, Andres Freund wrote: On March 4, 2014 8:39:55 PM CET, Simon Riggs si...@2ndquadrant.com wrote: I was going to add an option to increase lock level, but I can't see why you'd want it even. The dumps are consistent... Mvcc scans only guarantee that individual scans are

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Andres Freund
On 2014-03-04 14:29:31 -0800, Josh Berkus wrote: On 03/04/2014 11:43 AM, Andres Freund wrote: On March 4, 2014 8:39:55 PM CET, Simon Riggs si...@2ndquadrant.com wrote: I was going to add an option to increase lock level, but I can't see why you'd want it even. The dumps are consistent...

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-04 Thread Greg Stark
On Tue, Mar 4, 2014 at 8:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: CREATE INDEX CONCURRENTLY, otoh, already did break pg_dump, and we had to hack things to fix it; see commit 683abc73dff549e94555d4020dae8d02f32ed78b. Well pg_dump was only broken in that there was a new catalog state to deal

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Andres Freund
On 2014-03-04 16:37:48 -0500, Tom Lane wrote: However, it seems possible that we could have a mode in which a read-only session did all its catalog fetches according to the transaction snapshot. That would get us to a situation where the backend-internal lookups that ruleutils relies on would

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe Reply-To:

2014-03-04 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Tue, Mar 4, 2014 at 8:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: CREATE INDEX CONCURRENTLY, otoh, already did break pg_dump, and we had to hack things to fix it; see commit 683abc73dff549e94555d4020dae8d02f32ed78b. Well pg_dump was only broken in that there

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-04 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-03-04 16:37:48 -0500, Tom Lane wrote: However, it seems possible that we could have a mode in which a read-only session did all its catalog fetches according to the transaction snapshot. That would get us to a situation where the

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Robert Haas
On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs si...@2ndquadrant.com wrote: Removing SELECT privilege while running a SELECT would be a different matter. This is all a matter of definition; we can make up any rules we like. Doing so is IMHO a separate patch and not something to hold up the main

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 15:19, Robert Haas robertmh...@gmail.com wrote: On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs si...@2ndquadrant.com wrote: What I'm really concerned about is whether there are other things like the SnapshotNow issues that can cause stuff to halt and catch fire. I don't know

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 3 March 2014 15:19, Robert Haas robertmh...@gmail.com wrote: What I'm really concerned about is whether there are other things like the SnapshotNow issues that can cause stuff to halt and catch fire. I don't know whether there are or are not, but

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Robert Haas
On Sun, Mar 2, 2014 at 4:50 AM, Simon Riggs si...@2ndquadrant.com wrote: v20 includes slightly re-ordered checks in GetLockLevel, plus more detailed comments on each group of subcommands. Also corrects grammar as noted by Vik. Plus adds an example of usage to the docs. This patch contains a

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 15:53, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 3 March 2014 15:19, Robert Haas robertmh...@gmail.com wrote: What I'm really concerned about is whether there are other things like the SnapshotNow issues that can cause stuff to halt and

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 16:06, Robert Haas robertmh...@gmail.com wrote: On Sun, Mar 2, 2014 at 4:50 AM, Simon Riggs si...@2ndquadrant.com wrote: v20 includes slightly re-ordered checks in GetLockLevel, plus more detailed comments on each group of subcommands. Also corrects grammar as noted by Vik.

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 11:29 AM, Simon Riggs si...@2ndquadrant.com wrote: On 3 March 2014 16:06, Robert Haas robertmh...@gmail.com wrote: On Sun, Mar 2, 2014 at 4:50 AM, Simon Riggs si...@2ndquadrant.com wrote: v20 includes slightly re-ordered checks in GetLockLevel, plus more detailed

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 10:19:55AM -0500, Robert Haas wrote: On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs si...@2ndquadrant.com wrote: Removing SELECT privilege while running a SELECT would be a different matter. This is all a matter of definition; we can make up any rules we like. Doing

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 03:43:46PM +, Simon Riggs wrote: The question is are there any specific areas of concern here? If not, then we commit because we've done a lot of work on it and at the moment the balance is high benefit to users against a non-specific feeling of risk. @Noah -

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 18:57, Noah Misch n...@leadboat.com wrote: On Mon, Mar 03, 2014 at 10:19:55AM -0500, Robert Haas wrote: On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs si...@2ndquadrant.com wrote: Removing SELECT privilege while running a SELECT would be a different matter. This is all a

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 07:19:45PM +, Simon Riggs wrote: On 3 March 2014 18:57, Noah Misch n...@leadboat.com wrote: On Mon, Mar 03, 2014 at 10:19:55AM -0500, Robert Haas wrote: On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs si...@2ndquadrant.com wrote: Removing SELECT privilege while

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Noah Misch n...@leadboat.com writes: Just to be clear, that list is not a commentary on the particular patch at hand. Those are merely the kinds of regressions to look for in a patch affecting this area of the code. A complaint on pgsql-bugs just now reminded me of a specific area that needs

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Andres Freund
On 2014-03-03 19:15:27 -0500, Tom Lane wrote: Noah Misch n...@leadboat.com writes: Just to be clear, that list is not a commentary on the particular patch at hand. Those are merely the kinds of regressions to look for in a patch affecting this area of the code. A complaint on pgsql-bugs

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-03-03 19:15:27 -0500, Tom Lane wrote: This greatly ameliorates the snapshot-skew problems that arise from its habit of doing some things for itself and other things via backend-internal functions (which historically used SnapshotNow and now

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Andres Freund
On 2014-03-03 20:32:13 -0500, Tom Lane wrote: Afair (I really haven't rechecked) all the actions that have a changed locklevels affect things that pg_dump recreates clientside, using a repeatable read snapshot, so there shouldn't be much change there? You're missing the point entirely if

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-03-03 20:32:13 -0500, Tom Lane wrote: You're missing the point entirely if you think pg_dump recreates everything client-side. No, I am not obviously not thinking that. What I mean is that the things that actually change their locking

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-01 Thread Vik Fearing
On 03/01/2014 12:06 PM, Simon Riggs wrote: On 27 February 2014 08:48, Simon Riggs si...@2ndquadrant.com wrote: On 26 February 2014 15:25, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-26 15:15:00 +, Simon Riggs wrote: On 26 February 2014 13:38, Andres Freund and...@2ndquadrant.com

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-27 Thread Simon Riggs
On 26 February 2014 07:32, Simon Riggs si...@2ndquadrant.com wrote: * Are you sure AlterConstraint is generally safe without an AEL? It should be safe to change whether something is deferred, but not necessarily whether it's deferrable? We change the lock levels for individual commands.

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-27 Thread Simon Riggs
On 26 February 2014 15:25, Andres Freund and...@2ndquadrant.com wrote: * Why does ChangeOwner need AEL? Ownership affects privileges, which includes SELECTs, hence AEL. So? That reply could be added to any post. Please explain your concern. I don't understand why that means it

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-27 Thread Simon Riggs
On 26 February 2014 15:25, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-26 15:15:00 +, Simon Riggs wrote: On 26 February 2014 13:38, Andres Freund and...@2ndquadrant.com wrote: Hi, On 2014-02-26 07:32:45 +, Simon Riggs wrote: * This definitely should include

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-26 Thread Andres Freund
Hi, On 2014-02-26 07:32:45 +, Simon Riggs wrote: * This definitely should include isolationtester tests actually performing concurrent ALTER TABLEs. All that's currently there is tests that the locklevel isn't too high, but not that it actually works. There is no concurrent

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-26 Thread Simon Riggs
On 26 February 2014 13:38, Andres Freund and...@2ndquadrant.com wrote: Hi, On 2014-02-26 07:32:45 +, Simon Riggs wrote: * This definitely should include isolationtester tests actually performing concurrent ALTER TABLEs. All that's currently there is tests that the locklevel isn't

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-26 Thread Andres Freund
On 2014-02-26 15:15:00 +, Simon Riggs wrote: On 26 February 2014 13:38, Andres Freund and...@2ndquadrant.com wrote: Hi, On 2014-02-26 07:32:45 +, Simon Riggs wrote: * This definitely should include isolationtester tests actually performing concurrent ALTER TABLEs. All that's

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-25 Thread Simon Riggs
On 24 February 2014 16:01, Andres Freund and...@2ndquadrant.com wrote: Hi, I took a quick peek at this, and noticed the following things: * I am pretty sure this patch doesn't compile anymore after the latest set of releases. Refreshed to v18, will post shortly. * This definitely should

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-02-24 Thread Andres Freund
Hi, I took a quick peek at this, and noticed the following things: * I am pretty sure this patch doesn't compile anymore after the latest set of releases. * This definitely should include isolationtester tests actually performing concurrent ALTER TABLEs. All that's currently there is tests

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-29 Thread Simon Riggs
On 29 January 2014 05:43, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 28, 2014 at 12:36 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Honestly, I would prefer that we push a patch that has been thoroughly reviewed and in which we have more

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Bruce Momjian
On Mon, Jan 27, 2014 at 08:57:26PM +, Simon Riggs wrote: We get the new behaviour by default and I expect we'll be very happy with it. A second thought is that if we have problems of some kind in the field as a result of the new lock modes then we will be able to turn them off. I'm happy

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Simon Riggs
On 28 January 2014 14:55, Bruce Momjian br...@momjian.us wrote: On Mon, Jan 27, 2014 at 08:57:26PM +, Simon Riggs wrote: We get the new behaviour by default and I expect we'll be very happy with it. A second thought is that if we have problems of some kind in the field as a result of the

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Bruce Momjian
On Tue, Jan 28, 2014 at 04:36:39PM +, Simon Riggs wrote: For me, reducing the strength of DDL locking is a major change in RDBMS behaviour that could both delight and surprise our users. Maybe a few actually depend upon the locking behaviour, maybe. After some years of various people

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Heikki Linnakangas
On 01/28/2014 07:15 PM, Bruce Momjian wrote: On Tue, Jan 28, 2014 at 04:36:39PM +, Simon Riggs wrote: For me, reducing the strength of DDL locking is a major change in RDBMS behaviour that could both delight and surprise our users. Maybe a few actually depend upon the locking behaviour,

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Bruce Momjian
On Tue, Jan 28, 2014 at 07:21:50PM +0200, Heikki Linnakangas wrote: I have no problem removing the parameter if required to. In that case, I would like to leave the parameter in until mid beta, to allow greater certainty. In any case, I would wish to retain as a minimum an extern bool variable

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Heikki Linnakangas
On 01/28/2014 07:26 PM, Bruce Momjian wrote: On Tue, Jan 28, 2014 at 07:21:50PM +0200, Heikki Linnakangas wrote: I have no problem removing the parameter if required to. In that case, I would like to leave the parameter in until mid beta, to allow greater certainty. In any case, I would wish to

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Bruce Momjian
On Tue, Jan 28, 2014 at 07:30:47PM +0200, Heikki Linnakangas wrote: On 01/28/2014 07:26 PM, Bruce Momjian wrote: On Tue, Jan 28, 2014 at 07:21:50PM +0200, Heikki Linnakangas wrote: I have no problem removing the parameter if required to. In that case, I would like to leave the parameter in

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Honestly, I would prefer that we push a patch that has been thoroughly reviewed and in which we have more confidence, so that we can push without a GUC. This means mark in CF as needs-review, then some other developer reviews it and marks it as

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Alvaro Herrera
Bruce Momjian escribió: I have no problem removing the parameter if required to. In that case, I would like to leave the parameter in until mid beta, to allow greater certainty. Uhm. If we remove a GUC during beta we don't need to force an initdb. At worst we will need to keep a no-op GUC

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Simon Riggs
On 28 January 2014 17:15, Bruce Momjian br...@momjian.us wrote: On Tue, Jan 28, 2014 at 04:36:39PM +, Simon Riggs wrote: For me, reducing the strength of DDL locking is a major change in RDBMS behaviour that could both delight and surprise our users. Maybe a few actually depend upon the

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Andres Freund
On 2014-01-27 15:25:22 -0500, Robert Haas wrote: On Mon, Jan 27, 2014 at 12:58 PM, Simon Riggs si...@2ndquadrant.com wrote: This version adds a GUC called ddl_exclusive_locks which allows us to keep the 9.3 behaviour if we wish it. Some people may be surprised that their programs don't wait

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Simon Riggs
On 28 January 2014 17:21, Heikki Linnakangas hlinnakan...@vmware.com wrote: I don't understand why anyone would want to turn this feature off, ie. require stronger locks than necessary for a DDL change. Nobody would *want* to turn it off. They might need to, as explained. If we're not

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Robert Haas
On Tue, Jan 28, 2014 at 12:36 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Bruce Momjian escribió: I have no problem removing the parameter if required to. In that case, I would like to leave the parameter in until mid beta, to allow greater certainty. Uhm. If we remove a GUC

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 28, 2014 at 12:36 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Honestly, I would prefer that we push a patch that has been thoroughly reviewed and in which we have more confidence, so that we can push without a GUC. This means mark

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Simon Riggs
On 24 January 2014 08:33, Simon Riggs si...@2ndquadrant.com wrote: On 24 January 2014 07:08, Peter Geoghegan p...@heroku.com wrote: On Wed, Jan 15, 2014 at 6:57 AM, Simon Riggs si...@2ndquadrant.com wrote: v15 to fix the above problem. v16 attached v17 attached This version adds a GUC

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Simon Riggs
On 27 January 2014 17:58, Simon Riggs si...@2ndquadrant.com wrote: On 24 January 2014 08:33, Simon Riggs si...@2ndquadrant.com wrote: On 24 January 2014 07:08, Peter Geoghegan p...@heroku.com wrote: On Wed, Jan 15, 2014 at 6:57 AM, Simon Riggs si...@2ndquadrant.com wrote: v15 to fix the above

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Robert Haas
On Mon, Jan 27, 2014 at 12:58 PM, Simon Riggs si...@2ndquadrant.com wrote: On 24 January 2014 08:33, Simon Riggs si...@2ndquadrant.com wrote: On 24 January 2014 07:08, Peter Geoghegan p...@heroku.com wrote: On Wed, Jan 15, 2014 at 6:57 AM, Simon Riggs si...@2ndquadrant.com wrote: v15 to fix

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Peter Geoghegan
On Mon, Jan 27, 2014 at 12:25 PM, Robert Haas robertmh...@gmail.com wrote: I haven't reviewed the patch, but -1 for adding a GUC. I'm pretty surprised that it's been suggested that some people might prefer AccessExclusiveLocks. Why would anyone prefer that? -- Peter Geoghegan -- Sent via

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: On Mon, Jan 27, 2014 at 12:25 PM, Robert Haas robertmh...@gmail.com wrote: I haven't reviewed the patch, but -1 for adding a GUC. I'm pretty surprised that it's been suggested that some people might prefer AccessExclusiveLocks. Why would anyone prefer

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Simon Riggs
On 27 January 2014 20:35, Peter Geoghegan p...@heroku.com wrote: On Mon, Jan 27, 2014 at 12:25 PM, Robert Haas robertmh...@gmail.com wrote: I haven't reviewed the patch, but -1 for adding a GUC. I'm pretty surprised that it's been suggested that some people might prefer AccessExclusiveLocks.

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-27 Thread Simon Riggs
On 27 January 2014 20:47, Tom Lane t...@sss.pgh.pa.us wrote: Peter Geoghegan p...@heroku.com writes: On Mon, Jan 27, 2014 at 12:25 PM, Robert Haas robertmh...@gmail.com wrote: I haven't reviewed the patch, but -1 for adding a GUC. I'm pretty surprised that it's been suggested that some people

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-01-24 Thread Simon Riggs
On 24 January 2014 07:08, Peter Geoghegan p...@heroku.com wrote: On Wed, Jan 15, 2014 at 6:57 AM, Simon Riggs si...@2ndquadrant.com wrote: v15 to fix the above problem. Minor quibble: I get a compiler warning with the patch applied. relcache.c: In function 'RememberToFreeTupleDescAtEOX':

  1   2   3   >