Re: [HACKERS] Pending 9.4 patches
On 4/7/14 2:59 AM, Craig Ringer wrote: On 04/05/2014 03:57 AM, Andres Freund wrote: c07) Updatable security barrier views. This needs a serious look by a committer. I've been exercising it via row security and it's been looking pretty solid. It isn't a huge or intrusive patch, and it's seen several rounds of discussion during its development and refinement. (Thanks Dean). Same here, nothing but good feedback from testing. The updatable security barrier views has been sitting in Ready For Committer since late January. Unfortunately, I just learned that some of the people who might commit in this area--Stephen Frost for example--thought there were still major oustanding issues with that part. I (and I think Craig too) been waiting for that to be picked up by a committer, Stephen was waiting for me or Craig to fix unrelated bugs, and that's where the CF process has been deadlocked on this one. A core bugfix with locking in security barrier views is required before the regression tests can be fixed up properly, for one thing. Tom also expressed concerns about how plan invalidation works, though it's not yet clear whether that was just miscommunication about how it works on my part or whether there's a concrete problem there. This is similarly stuck. I'm not out of resources, there's just nothing I can do here myself. For this to move forward, a committer needs to pick up the security barrier views part. We also need a bug fix for the issue that's breaking the regression tests. Once all that's done, RLS on top of updateable security barrier views might be commitable. But there's no way to tell for sure until those other two bits are sorted out. I'd really love to finish this off for 9.4, but other projects have to come first. I have no other projects ahead of this for the remainder of this month. I just can't figure out what to do next until there's a committer (or committers, if someone else is going to take on the locking bug) identified. I looked at the locking problem enough to know that one is over my head. -- Greg Smith greg.sm...@crunchydatasolutions.com Chief PostgreSQL Evangelist - http://crunchydatasolutions.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Pending 9.4 patches
On Tue, Apr 8, 2014 at 11:59 AM, Gregory Smith gregsmithpg...@gmail.com wrote: I have no other projects ahead of this for the remainder of this month. I just can't figure out what to do next until there's a committer (or committers, if someone else is going to take on the locking bug) identified. I looked at the locking problem enough to know that one is over my head. What post or thread should I read for details about this locking bug? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Pending 9.4 patches
* Robert Haas (robertmh...@gmail.com) wrote: On Tue, Apr 8, 2014 at 11:59 AM, Gregory Smith gregsmithpg...@gmail.com wrote: I have no other projects ahead of this for the remainder of this month. I just can't figure out what to do next until there's a committer (or committers, if someone else is going to take on the locking bug) identified. I looked at the locking problem enough to know that one is over my head. What post or thread should I read for details about this locking bug? I think it was discussed a couple times, but it's here: http://www.postgresql.org/message-id/531d3355.6010...@2ndquadrant.com Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Pending 9.4 patches
* Tom Lane (t...@sss.pgh.pa.us) wrote: Craig Ringer cr...@2ndquadrant.com writes: On 04/05/2014 03:57 AM, Andres Freund wrote: r04) Row-security based on Updatable security barrier views This one's fate seems to be hard to judge without c07. Open issues remain with this patch, and resources for working on it in 9.4 have run out. It is not ready for commit. A core bugfix with locking in security barrier views is required before the regression tests can be fixed up properly, for one thing. Tom also expressed concerns about how plan invalidation works, though it's not yet clear whether that was just miscommunication about how it works on my part or whether there's a concrete problem there. I'd really love to finish this off for 9.4, but other projects have to come first. Given that, I think we should go ahead and mark this one Returned With Feedback. It's past time to be punting anything that doesn't have a serious chance of getting committed for 9.4. I'm a bit confused on this point- is the only issue the *preexisting* bug with security barrier views? I agree we need to fix that, but I'd really like to see that fixed and backpatched to address the risk in back-branches. I had understood there to be *other* issues with this, which is why I hadn't spent time on it. Craig, in general, I'd argue that a pre-existing bug isn't a reason that a patch isn't ready for commit. The bug may need to be fixed before the patch goes in, but saying a patch isn't ready implied, to me at least, issues with the *patch*, which it sounds like isn't the case here. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Pending 9.4 patches
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 02:00 AM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Craig Ringer cr...@2ndquadrant.com writes: On 04/05/2014 03:57 AM, Andres Freund wrote: r04) Row-security based on Updatable security barrier views This one's fate seems to be hard to judge without c07. Open issues remain with this patch, and resources for working on it in 9.4 have run out. It is not ready for commit. A core bugfix with locking in security barrier views is required before the regression tests can be fixed up properly, for one thing. Tom also expressed concerns about how plan invalidation works, though it's not yet clear whether that was just miscommunication about how it works on my part or whether there's a concrete problem there. I'd really love to finish this off for 9.4, but other projects have to come first. Given that, I think we should go ahead and mark this one Returned With Feedback. It's past time to be punting anything that doesn't have a serious chance of getting committed for 9.4. I'm a bit confused on this point- is the only issue the *preexisting* bug with security barrier views? This thread discusses two patches. The above refers to row security (per quoted text at top), not updatable security barrier views. Updatable security barrier views are ready. There's a pre-existing bug with security barrier views, but updatable s.b. views don't make it any worse and it can be fixed separately. Row security is not. It could possibly be committed w/o a fix for the security barrier bug by deleting the relevant regression tests, but Tom had reservations about plan invalidation in it, the docs need updating, and it needs a bunch more testing. It's possible I could have it ready in a few days - or it might be a couple of weeks. I ran out of time to work on it for 9.4. Craig, in general, I'd argue that a pre-existing bug isn't a reason that a patch isn't ready for commit. The bug may need to be fixed before the patch goes in, but saying a patch isn't ready implied, to me at least, issues with the *patch*, which it sounds like isn't the case here. I tend to agree, and for that reason want updatable security barrier views to make it in for 9.4. - -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTRKCAAAoJELBXNkqjr+S2kwQH+gP9+sNyEnE2HiKpRkEgFn0C g+rIfhjJl0ANPMAt6DIBNbns/1t38xqhpkbmirT8cS0RVplAETV6ynYngdzcQQOk GVeoOylSr75Hh3PWC82qRBHtgMZ7tV8RChNXgW6p4qekpAhqmAMJzBwq+bVhKXmZ +Wfpc1u5wTTc0aw9pmQVmr3ZpjibI+C54+eYrq97+JmC7kFHQWrLAmM/stiGeJpW nzOCADfQolpjCWDts/flwKDu+F2y4aUNhOUEiMo+LtPqPRgYioZwIUMeF5HBz+Ng CQTnjDeC/ROBFMvD1Jk1wBKvNl5lPd3ikdaLIaCmjav4hX2B35fbmuQLKgkxOwM= =AaWD -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Pending 9.4 patches
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 01:54 AM, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Tue, Apr 8, 2014 at 11:59 AM, Gregory Smith gregsmithpg...@gmail.com wrote: I have no other projects ahead of this for the remainder of this month. I just can't figure out what to do next until there's a committer (or committers, if someone else is going to take on the locking bug) identified. I looked at the locking problem enough to know that one is over my head. What post or thread should I read for details about this locking bug? I think it was discussed a couple times, but it's here: http://www.postgresql.org/message-id/531d3355.6010...@2ndquadrant.com Yep. The follow-ups are important though - showing that it's actually a pre-existing bug, and while it might also occur in updatable security barrier views, there's already a locking problem in the existing security barrier views that needs to be fixed. The short version is: When you have a RowMark (SELECT ... FOR UPDATE/SHARE, UPDATE, or DELETE) on the results of a query, security barrier views are incorrectly pushing this row mark down into the security_barrier subquery they generate. That means that if you: SELECT * FROM some_view FOR UPDATE WHERE id % 2 = 0; you should be locking only *even numbered* rows that match the predicate of some_view, but in fact, what'll get run looks like the pseudo-SQL: SELECT * FROM ( SELECT * FROM underlying_table WHERE view_predicate FOR UPDATE ) some_view FOR UPDATE; i.e. we're locking *all rows that match the view predicate*, failing to apply the *user* predicate before locking. - -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTRKFtAAoJELBXNkqjr+S2pa8IAI1UJPstG1EIcoT5szB7BXWT FBnRpe5zECM1faZuHAjx9dRYXGjv/u5E+wq2jwocXLqRPIf4Cu90KDmwx3O2gCPO psv8lpfkmjX7MGtuz4Y1A8OkcB+Q3m+4neV+NpFnPA5A3Dx7WLjiFCdHTurlvtD1 BZxK0YkUWw3S40v67MZtcOIrCRwVPQP9NS+PEt3WfTydRryXecOKnJxdolH6H4A8 1inCLvIfphkCChFMiLV6r+mzzi4JxRiPEwWg3uccLRhCwcTf1BQJcXbiKdbcTYBW XT5CSyVm76gpd3WkFfxahlXkaSLOrzGney0LGHUI4ItunlxQzPgQhx2ghc97P9w= =H425 -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Pending 9.4 patches
* Craig Ringer (cr...@2ndquadrant.com) wrote: On 04/09/2014 02:00 AM, Stephen Frost wrote: I'm a bit confused on this point- is the only issue the *preexisting* bug with security barrier views? This thread discusses two patches. The above refers to row security (per quoted text at top), not updatable security barrier views. Right, I understood that. Updatable security barrier views are ready. There's a pre-existing bug with security barrier views, but updatable s.b. views don't make it any worse and it can be fixed separately. Ok. Row security is not. It could possibly be committed w/o a fix for the security barrier bug by deleting the relevant regression tests, but Tom had reservations about plan invalidation in it, the docs need updating, and it needs a bunch more testing. It's possible I could have it ready in a few days - or it might be a couple of weeks. I ran out of time to work on it for 9.4. So- row security makes the *existing bug* worse; I get that. The question regarding plan invalidation may be something we can work out. As for docs and testing, those are things we would certainly be better off with and may mean that this isn't able to make it into 9.4, which is fair, but I wouldn't toss it out solely due to that. Craig, in general, I'd argue that a pre-existing bug isn't a reason that a patch isn't ready for commit. The bug may need to be fixed before the patch goes in, but saying a patch isn't ready implied, to me at least, issues with the *patch*, which it sounds like isn't the case here. I tend to agree, and for that reason want updatable security barrier views to make it in for 9.4. Ok. I'm going to make a serious effort to find time to work on this, at least. Right now I'm busy preparing to launch a new site (you'll see the announce in a couple days...), etc, etc, but I should have time this weekend... Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Pending 9.4 patches
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 09:28 AM, Stephen Frost wrote: Ok. I'm going to make a serious effort to find time to work on this, at least. Right now I'm busy preparing to launch a new site (you'll see the announce in a couple days...), etc, etc, but I should have time this weekend... That'd be fantastic. I'm committed on other work, but I'll make the time to get back on this somehow. I've put a lot of time and work into it, and would love to see it make 9.4 against the odds. If just updatable security barrier views make it, that's a big plus for next time around. - -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTRKqGAAoJELBXNkqjr+S28MgIAJ7uD5gYMS+zX1VKhtB104qH 9+iO1KeK/JKvNlVitwVQ+rpAUcCt13VU0CIKH3mn/5+hRQLiM/8mffdl33oCdh4L RRtx3zMiM74NiJk8H0Z9awjdAAAEe5IpcQuac57sFn8+NjQJAykpv03AwltLgbd7 s+ZK90kqGHtQTRAvxJqGfObRa/rc7IP1iASxk26xiRR/fTBxjGIJ0T+ihbjf/XI0 gYobmaUUQJFxoKTprhLL+MZHBf2UZntbJJBuL7VS9b6NlgyeS6rVai7f5MyREW0m giIpKWRJTROW7o8syAy7jjCpuKgbuxVvAHOFdo8EIyX4u6tkZ4NakUwdn1Zimgg= =QPHO -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Pending 9.4 patches
On 04/05/2014 03:57 AM, Andres Freund wrote: c07) Updatable security barrier views. This needs a serious look by a committer. I've been exercising it via row security and it's been looking pretty solid. It isn't a huge or intrusive patch, and it's seen several rounds of discussion during its development and refinement. (Thanks Dean). In some ways it'd be nicer to do it by splitting resultRelation into two (row read source, and relation to write modified tuples into), but this turns out to be rather complex and exceedingly intrusive. We might need to do it someday - at that point, it wouldn't be overly hard to change updatable s.b. views over to working that way, but only once the major surgery required to remove the assumptions about resultRelation was done. Having this in place in 9.4 would allow people to build row-security like features in applications, and ease the path of row security into 9.5. r04) Row-security based on Updatable security barrier views This one's fate seems to be hard to judge without c07. Open issues remain with this patch, and resources for working on it in 9.4 have run out. It is not ready for commit. A core bugfix with locking in security barrier views is required before the regression tests can be fixed up properly, for one thing. Tom also expressed concerns about how plan invalidation works, though it's not yet clear whether that was just miscommunication about how it works on my part or whether there's a concrete problem there. I'd really love to finish this off for 9.4, but other projects have to come first. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Pending 9.4 patches
Craig Ringer cr...@2ndquadrant.com writes: On 04/05/2014 03:57 AM, Andres Freund wrote: r04) Row-security based on Updatable security barrier views This one's fate seems to be hard to judge without c07. Open issues remain with this patch, and resources for working on it in 9.4 have run out. It is not ready for commit. A core bugfix with locking in security barrier views is required before the regression tests can be fixed up properly, for one thing. Tom also expressed concerns about how plan invalidation works, though it's not yet clear whether that was just miscommunication about how it works on my part or whether there's a concrete problem there. I'd really love to finish this off for 9.4, but other projects have to come first. Given that, I think we should go ahead and mark this one Returned With Feedback. It's past time to be punting anything that doesn't have a serious chance of getting committed for 9.4. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Pending 9.4 patches
Hi, I today tried to cleanup the state of the pending patches a bit. I hope I haven't bloodied too many toes. Here's a summary of all patches that aren't either committed, returned or rejected: Pending patches waiting for committer are: c01) Custom Scan APIs This really seems to need Tom's attention. c02) cache-only table scan This unfortunately also seems to need Tom's attention. c03) ALTER TABLE lock strength reduction Simon should cleanup the cosmetic issues noticed by Noah and commit it. c04) Foreign table inheritance. No idea. Too big to look. c05) Widening application of indices. Is this really ready? I am rather doubtful. c06) Enable CREATE FOREIGN TABLE (... LIKE ... ) I *personally* don't like some parts of the approach, but a committer should decide. My concerns are more around aestetics and maintainability than any real problems. c07) Updatable security barrier views. This needs a serious look by a committer. c08) Bugfix for timeout in LDAP connection parameter resolution. Looks pretty straightforward to me. c09) Problem with displaying wide tables in psql Not really sure. I wonder if there are situations where the new output will be more confusing than the old. c10) PostgreSQL fails to start on Windows if it crashes after tablespace creation Looks good to me, trivial code formatting issue. c11) pg_ctl fails with config-only directory Not a big fan, but I don't have a better idea. c12) pg_ctl always uses the same event source I personally think this is pointless complication. But I am also not a windows person. c12) pg_ctl stop times out when it should respond quickly c13) PostgreSQL Service on Windows does not start if data directory given is relative path I unfortunately couldn't convince myself to care enough to have an opinion on these. The complicated patches that seem to need serious committer time are: c07, c01, c02, c04, c05. My feeling is that 01 and 02 came in pretty darn late in their current form to be fully considered for 9.4, independent of their state. 07 seems to be somewhat large this late, but there really hasn't been much change recently. Pending patches waiting for review are: r01) Inverse Aggregate Transition Functions There's some argument about whether a already useful subset should be committed in 9.4 or whether everything should be in 9.5. The 9.4 thing doesn't seem to be too far away from being ready. r02) Using indices for UNION Tom seems to be looking at atm. Not that big. r03) Optimization in regexp handling in pg_trgm Unsure whether the complications are worth the gains. r04) Row-security based on Updatable security barrier views This one's fate seems to be hard to judge without c07. r05) WAL rate limiting I don't think we can get concensus on this for 9.4, thus it probably should be returned with feedback. r06) Add min, max, and stdev execute statement time in pg_stat_statement The discussion here seems to have stalled on the questions which additional columns should be added. Doesn't look too hard to make committable once agreement has been reached. r07) ECPG cursor readahead I have no friggin idea. r08) vacuumdb: Add option --analyze-in-stages Should imo be committed after some absolutely minor cleanup. r09) transforms It's a bit sad to see so this patch limping along for more than a year now. But it still doesn't seem to be ready :(. Peter wrote more than two months back that he wants to make another pass after a couple nights of sleep. I'll have a look so there's some progress, but I don't think it's 9.4 material at this point. r10) PL/pgSQL Warnings - plpgsql.warn_shadow I haven't followed, and the thread is too long. r11) extension_control_path Minefield. r12) Create function prototype as part of PG_FUNCTION_INFO_V1 Some potential issues, but it'd reduce the amount extension authors have to type. r13) Correctly place DLLs for ECPG apps in bin folder msvc issue, doesn't seem to be fully ready. Seems like a bug worthy of fixing tho. r14) Issue with PGC_BACKEND parameters on Windows Looks simple enough. r15) multibyte messages are displayed incorrectly on the client Seems like a bigger can of worms than its worth. r16) tests for client programs Should imo be committed after some simple cleanup. The bigger things that might have a bearing for 9.4 seem to be: r01, r04. Lots of the other stuff just need someone paying a bit of attention. Waiting for author: w01) GiST support for inet datatypes Needs a rebase as noticed today, apparently ready for committer otherwise. w02) variant of regclass etc. Robert noticed some issues today when he wanted to commit. I hope that we can prune this list to the patches that we think have a chance for 9.4, so we can avoid delaying the feature freeze. The CF is officially closed,
