Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-03-06 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: On 26 February 2015 at 09:50, Dean Rasheed dean.a.rash...@gmail.com wrote: On 26 February 2015 at 05:43, Stephen Frost sfr...@snowman.net wrote: I wonder if there are some documentation updates which need to be done for this also? I'm

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-03-06 Thread Dean Rasheed
On 26 February 2015 at 09:50, Dean Rasheed dean.a.rash...@gmail.com wrote: On 26 February 2015 at 05:43, Stephen Frost sfr...@snowman.net wrote: I wonder if there are some documentation updates which need to be done for this also? I'm planning to look as I vauguely recall mentioning the

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-03-02 Thread Dean Rasheed
On 26 February 2015 at 09:27, Dean Rasheed dean.a.rash...@gmail.com wrote: I think this should probably be committed as 2 separate patches... On closer inspection, the 2 parts are interrelated since the new self-join test that tests the inheritance planner changes also requires the rewriter

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-02-26 Thread Dean Rasheed
On 26 February 2015 at 05:41, Stephen Frost sfr...@snowman.net wrote: Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: Here's an updated patch with a new test for this bug. I've been developing the fixes for these RLS issues as one big patch, but I suppose it would be easy to split up,

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-02-26 Thread Dean Rasheed
On 26 February 2015 at 05:43, Stephen Frost sfr...@snowman.net wrote: Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: Attached is a patch to make RLS checks run before attempting to insert/update any data rather than afterwards. Excellent, this I really like and it's a pretty

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-02-25 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: Here's an updated patch with a new test for this bug. I've been developing the fixes for these RLS issues as one big patch, but I suppose it would be easy to split up, if that's preferred. Thanks for working on all of this! I've brought

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-02-25 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: Attached is a patch to make RLS checks run before attempting to insert/update any data rather than afterwards. Excellent, this I really like and it's a pretty straight-forward change. I wonder if there are some documentation updates which

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-19 Thread Dean Rasheed
On 10 January 2015 at 15:12, Stephen Frost sfr...@snowman.net wrote: * Dean Rasheed (dean.a.rash...@gmail.com) wrote: Currently we're applying RLS CHECKs after the INSERT or UPDATE, like WITH CHECK OPTIONs on views. The SQL spec says that WITH CHECK OPTIONs on views have to be applied after

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-14 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: Turns out it wasn't as simple as that. prepend_row_security_policies() really could get called multiple times for the same RTE, because the call to query_tree_walker() at the end of fireRIRrules() would descend into the just-added quals again.

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-14 Thread Dean Rasheed
On 14 January 2015 at 08:43, Dean Rasheed dean.a.rash...@gmail.com wrote: On 12 January 2015 at 14:24, Stephen Frost sfr...@snowman.net wrote: Interesting, thanks for the work! I had been suspicious that there was an issue with the recursion handling. So continuing to review the RLS code, I

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-14 Thread Dean Rasheed
On 12 January 2015 at 14:24, Stephen Frost sfr...@snowman.net wrote: Interesting, thanks for the work! I had been suspicious that there was an issue with the recursion handling. So continuing to review the RLS code, I spotted the following in prepend_row_security_policies(): /* * We

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-12 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: On 10 January 2015 at 21:38, Dean Rasheed dean.a.rash...@gmail.com wrote: OK, I'll take a look at it. I started reading the existing code that expands RLS policies and spotted a couple of bugs that should be fixed first:- 1). In

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-12 Thread Dean Rasheed
On 10 January 2015 at 21:38, Dean Rasheed dean.a.rash...@gmail.com wrote: OK, I'll take a look at it. I started reading the existing code that expands RLS policies and spotted a couple of bugs that should be fixed first:- 1). In prepend_row_security_policies(), defaultDeny should be

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-12 Thread Dean Rasheed
On 12 January 2015 at 14:24, Stephen Frost sfr...@snowman.net wrote: Looking at the regression tests a bit though, I notice that this removes the lower-level LockRows.. There had been much discussion about that last spring which I believe you were a part of..? I specifically recall

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-10 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: On 9 January 2015 at 20:26, Stephen Frost sfr...@snowman.net wrote: Where this leaves me, at least, is feeling like we should always apply the INSERT WITH CHECK policy, then if there is a conflict, check the UPDATE USING policy and

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-10 Thread Dean Rasheed
On 9 January 2015 at 20:26, Stephen Frost sfr...@snowman.net wrote: Where this leaves me, at least, is feeling like we should always apply the INSERT WITH CHECK policy, then if there is a conflict, check the UPDATE USING policy and throw an error if the row isn't visible but otherwise perform

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-10 Thread Dean Rasheed
On 10 January 2015 at 15:12, Stephen Frost sfr...@snowman.net wrote: NOTE: If we do change RLS CHECKs to be executed prior to modifying any data, that's potentially a change that could be made independently of the UPSERT patch. We should also probably then stop referring to them as WITH CHECK

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Dean Rasheed
On 9 January 2015 at 00:49, Stephen Frost sfr...@snowman.net wrote: Peter, * Peter Geoghegan (p...@heroku.com) wrote: For column level privileges, you wouldn't expect to only get an error about not having the relevant update permissions at runtime, when the update path happens to be taken.

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Peter Geoghegan
On Fri, Jan 9, 2015 at 12:19 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: I was trying to think up an example where you might actually have different INSERT and UPDATE policies, and the best I can think of is some sort of mod_count column where you have an INSERT CHECK (mod_count = 0) and

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Peter Geoghegan
On Fri, Jan 9, 2015 at 2:22 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: Whoa, hang on. I think you're being a bit quick to dismiss that example. Why shouldn't I want an upsert where the majority of the table columns follow the usual make it so pattern of an upsert, but there is also this

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: On Fri, Jan 9, 2015 at 1:09 PM, Stephen Frost sfr...@snowman.net wrote: Therefore, I'm not sure that I see the point in checking the INSERT tuple against the UPDATE policy. I guess it wouldn't be hard to modify the struct WithCheckOption to

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Stephen Frost
Dean, Peter, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: On 9 January 2015 at 08:49, Peter Geoghegan p...@heroku.com wrote: On Fri, Jan 9, 2015 at 12:19 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: I was trying to think up an example where you might actually have different

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Stephen Frost
Peter, * Peter Geoghegan (p...@heroku.com) wrote: On Fri, Jan 9, 2015 at 2:22 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: Whoa, hang on. I think you're being a bit quick to dismiss that example. Why shouldn't I want an upsert where the majority of the table columns follow the usual

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Peter Geoghegan
On Fri, Jan 9, 2015 at 12:53 PM, Stephen Frost sfr...@snowman.net wrote: I'm not sure how that would work exactly though, since the tuple the UPDATE results in might be different from what the INSERT has, as Dean pointed out. The INSERT tuple might even pass the UPDATE policy where the

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Peter Geoghegan
On Fri, Jan 9, 2015 at 1:09 PM, Stephen Frost sfr...@snowman.net wrote: To flip it around a bit, I don't think we can avoid checking the *resulting* tuple from the UPDATE against the UPDATE policy. We can avoid it - by not updating. What I'm suggesting is that an enforcement occurs that is more

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Peter Geoghegan
On Fri, Jan 9, 2015 at 12:26 PM, Stephen Frost sfr...@snowman.net wrote: Where this leaves me, at least, is feeling like we should always apply the INSERT WITH CHECK policy, then if there is a conflict, check the UPDATE USING policy and throw an error if the row isn't visible but otherwise

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: On Fri, Jan 9, 2015 at 12:26 PM, Stephen Frost sfr...@snowman.net wrote: Where this leaves me, at least, is feeling like we should always apply the INSERT WITH CHECK policy, then if there is a conflict, check the UPDATE USING policy and throw an

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-09 Thread Dean Rasheed
On 9 January 2015 at 08:49, Peter Geoghegan p...@heroku.com wrote: On Fri, Jan 9, 2015 at 12:19 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: I was trying to think up an example where you might actually have different INSERT and UPDATE policies, and the best I can think of is some sort of

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-08 Thread Stephen Frost
Peter, * Peter Geoghegan (p...@heroku.com) wrote: For column level privileges, you wouldn't expect to only get an error about not having the relevant update permissions at runtime, when the update path happens to be taken. And so it is for RLS. Right, that's the precedent we should be

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Dean Rasheed
On 6 January 2015 at 20:44, Peter Geoghegan p...@heroku.com wrote: Another issue that I see is that there is only one resultRelation between the INSERT and UPDATE. That means that without some additional special measure, both UPDATE and INSERT WITH CHECK ( ... ) options are applied regardless

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Peter Geoghegan
On Wed, Jan 7, 2015 at 12:01 PM, Stephen Frost sfr...@snowman.net wrote: Other databases have this capability and have triggers and at least one ends up firing both INSERT and UPDATE triggers, with many complaints from users about how that ends up making the performance suck. Perhaps we could

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Peter Geoghegan
On Wed, Jan 7, 2015 at 12:26 PM, Stephen Frost sfr...@snowman.net wrote: I agree with that up to a point- this case feels more like a POLA violation though rather than a run-of-the-mill you need to test all that. I'm a bit worried this might lead to cases where users can't use UPSERT because

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 7, 2015 at 4:04 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: I think the policies applied should depend on the path taken, so if it does an INSERT, then only the INSERT CHECK policy should be applied (after the insert), but if it

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 7, 2015 at 3:01 PM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 7, 2015 at 4:04 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: I think the policies applied should depend on

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Robert Haas
On Wed, Jan 7, 2015 at 3:01 PM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 7, 2015 at 4:04 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: I think the policies applied should depend on the path taken, so if it does an INSERT, then only

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: On 6 January 2015 at 20:44, Peter Geoghegan p...@heroku.com wrote: Another issue that I see is that there is only one resultRelation between the INSERT and UPDATE. That means that without some additional special measure, both UPDATE and

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-07 Thread Robert Haas
On Wed, Jan 7, 2015 at 4:04 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: I think the policies applied should depend on the path taken, so if it does an INSERT, then only the INSERT CHECK policy should be applied (after the insert), but if it ends up doing an UPDATE, I would expect the

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-06 Thread Peter Geoghegan
On Tue, Jan 6, 2015 at 9:39 AM, Robert Haas robertmh...@gmail.com wrote: I think the INSERT .. ON CONFLICT UPDATE shouldn't be able to attempt an update unless the UPDATE policies of the table are such that a regular UPDATE would find the affected row. The post-image of the row needs to

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-06 Thread Peter Geoghegan
Another issue that I see is that there is only one resultRelation between the INSERT and UPDATE. That means that without some additional special measure, both UPDATE and INSERT WITH CHECK ( ... ) options are applied regardless of whether the INSERT path was taken, or the UPDATE path. Maybe that's

[HACKERS] INSERT ... ON CONFLICT UPDATE and RLS

2015-01-05 Thread Peter Geoghegan
The patch that implements INSERT ... ON CONFLICT UPDATE has support and tests for per-column privileges (which are not relevant to the IGNORE variant, AFAICT). However, RLS support is another thing entirely. It has not been properly thought out, and unlike per-column privileges requires careful