Re: [HACKERS] Parallel safety tagging of extension functions

Robert Haas writes: > On Sun, Jun 19, 2016 at 4:21 PM, Andreas Karlsson wrote: >> Here they are. Shelve them if you like. They are some of the least important >> extensions that still should be fixed so they can end up in 10 if you do not >> find the

Re: [HACKERS] Parallel safety tagging of extension functions

On Sun, Jun 19, 2016 at 4:21 PM, Andreas Karlsson wrote: > On 06/17/2016 09:11 PM, Robert Haas wrote: >> I was kind of hoping you'd have a new version of this posted already. >> beta2 is wrapping on Monday, and I'm inclined to shelve anything that >> isn't done by then for the

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/17/2016 09:11 PM, Robert Haas wrote: I was kind of hoping you'd have a new version of this posted already. beta2 is wrapping on Monday, and I'm inclined to shelve anything that isn't done by then for the next release. And I don't really plan to work much this weekend. Here they are.

Re: [HACKERS] Parallel safety tagging of extension functions

On Thu, Jun 16, 2016 at 9:15 AM, Andreas Karlsson wrote: >> That was the only clear mistake I found, but I tend >> to think that changing the markings on anything defined by >> UNSUPPORTED_FUNCTION() is pretty silly, because there's no point in >> going to extra planner effort

Re: [HACKERS] Parallel safety tagging of extension functions

On 01-06-2016 20:52, Andreas Karlsson wrote: > I think at least three of the four aggregate functions are little used, > so I do not think many users would be affected. And only min(citext) and > max(citext) can make use of the parallel aggregation. > > The functions are: > > min(citext) >

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/14/2016 09:55 PM, Robert Haas wrote: On Tue, Jun 14, 2016 at 1:51 PM, Robert Haas wrote: On Tue, Jun 14, 2016 at 6:37 AM, Andreas Karlsson wrote: I have rebased all my patches on the current master now (and skipped the extensions I previously

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/14/2016 07:51 PM, Robert Haas wrote: On Tue, Jun 14, 2016 at 6:37 AM, Andreas Karlsson wrote: I have rebased all my patches on the current master now (and skipped the extensions I previously listed). Thanks, this is really helpful. It was starting to get hard to

Re: [HACKERS] Parallel safety tagging of extension functions

On Tue, Jun 14, 2016 at 1:51 PM, Robert Haas wrote: > On Tue, Jun 14, 2016 at 6:37 AM, Andreas Karlsson wrote: >> I have rebased all my patches on the current master now (and skipped the >> extensions I previously listed). > > Thanks, this is really

Re: [HACKERS] Parallel safety tagging of extension functions

On Tue, Jun 14, 2016 at 6:37 AM, Andreas Karlsson wrote: > I have rebased all my patches on the current master now (and skipped the > extensions I previously listed). Thanks, this is really helpful. It was starting to get hard to keep track of what hadn't been applied yet. I

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/07/2016 05:54 PM, Andreas Karlsson wrote: On 06/07/2016 05:44 PM, Robert Haas wrote: cube: I think we need a new extension version. hstore: Does not apply for me. intarray: Does not apply for me. Those three and ltree, pg_trgm, and seg depend on my patch with fixes for the GiST/GIN

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/10/2016 01:44 PM, Andreas Karlsson wrote: I have attached a patch which adds the shcema, plus an updated patch for tseach2. Forgot adding schema to the tables. Here are new versions. Andreas parallel-contrib-v4-tsearch2.patch.gz Description: application/gzip diff --git

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/09/2016 10:48 PM, Tom Lane wrote: Robert Haas writes: On Sat, May 21, 2016 at 11:45 AM, Tom Lane wrote: Yes, let's fix it. This will also take care of the questions about whether the GIN/GIST opclass tweaks I made a few months ago require

Re: [HACKERS] Parallel safety tagging of extension functions

On Thu, Jun 9, 2016 at 5:45 PM, Andres Freund wrote: > On 2016-06-09 17:40:24 -0400, Robert Haas wrote: >> On Thu, Jun 9, 2016 at 4:48 PM, Tom Lane wrote: >> > Robert Haas writes: >> >> On Sat, May 21, 2016 at 11:45 AM, Tom Lane

Re: [HACKERS] Parallel safety tagging of extension functions

On Tue, Jun 7, 2016 at 11:44 AM, Robert Haas wrote: > More later. ltree: Skipped per your other email. ltree_plpython: No point. pageinspect: Committed. pg_buffercache: Committed. pgcrypto: Committed. pg_freespacemap: Committed. pg_prewarm: Committed. pgrowlocks:

Re: [HACKERS] Parallel safety tagging of extension functions

On 2016-06-09 17:40:24 -0400, Robert Haas wrote: > On Thu, Jun 9, 2016 at 4:48 PM, Tom Lane wrote: > > Robert Haas writes: > >> On Sat, May 21, 2016 at 11:45 AM, Tom Lane wrote: > >>> Yes, let's fix it. This will also take care of

Re: [HACKERS] Parallel safety tagging of extension functions

On Thu, Jun 9, 2016 at 4:48 PM, Tom Lane wrote: > Robert Haas writes: >> On Sat, May 21, 2016 at 11:45 AM, Tom Lane wrote: >>> Yes, let's fix it. This will also take care of the questions about >>> whether the GIN/GIST opclass

Re: [HACKERS] Parallel safety tagging of extension functions

Robert Haas writes: > On Sat, May 21, 2016 at 11:45 AM, Tom Lane wrote: >> Yes, let's fix it. This will also take care of the questions about >> whether the GIN/GIST opclass tweaks I made a few months ago require >> module version bumps. > Tom,

Re: [HACKERS] Parallel safety tagging of extension functions

On Thu, Jun 9, 2016 at 1:51 PM, Tom Lane wrote: > Robert Haas writes: >> On Sat, May 21, 2016 at 11:45 AM, Tom Lane wrote: >>> Yes, let's fix it. This will also take care of the questions about >>> whether the GIN/GIST opclass

Re: [HACKERS] Parallel safety tagging of extension functions

Robert Haas writes: > On Sat, May 21, 2016 at 11:45 AM, Tom Lane wrote: >> Yes, let's fix it. This will also take care of the questions about >> whether the GIN/GIST opclass tweaks I made a few months ago require >> module version bumps. > Tom,

Re: [HACKERS] Parallel safety tagging of extension functions

On Sat, May 21, 2016 at 11:45 AM, Tom Lane wrote: > Peter Eisentraut writes: >> On 5/20/16 7:37 PM, Robert Haas wrote: >>> I guess my first question is whether we have consensus on the release >>> into which we should put this. Some people

Re: [HACKERS] Parallel safety tagging of extension functions

On Wed, Jun 8, 2016 at 9:24 AM, Andreas Karlsson wrote: >> dblink: Isn't changing dblink_fdw_validator pointless? The others I get. > > Yeah, but since it is just one function I think it makes sense to change it > when we already are bumping the version of the extension. I

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/07/2016 05:44 PM, Robert Haas wrote: adminpack: Doesn't seem useful. The case I imagined was if someone would use these functions on the result from a slow CTE and would want the CTE to be executed in parallel. I have no idea if that is a realistic case, but I rarely use adminpack in

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/07/2016 05:44 PM, Robert Haas wrote: cube: I think we need a new extension version. hstore: Does not apply for me. intarray: Does not apply for me. Those three and ltree, pg_trgm, and seg depend on my patch with fixes for the GiST/GIN function signatures in

Re: [HACKERS] Parallel safety tagging of extension functions

On Fri, Jun 3, 2016 at 8:37 PM, Andreas Karlsson wrote: > Here is the patch split into many small patches as you suggested. The > current patches are based on top of the patch which fixes the signatures for > gin and gist functions. Generally, I think that there's no point in

Re: [HACKERS] Parallel safety tagging of extension functions

Hi, Here is the patch split into many small patches as you suggested. The current patches are based on top of the patch which fixes the signatures for gin and gist functions. These patches only touch functions which never should be called directly, so they are fine to skip. I decided to

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/02/2016 01:41 AM, Michael Paquier wrote: > On Thu, Jun 2, 2016 at 7:36 AM, Andreas Karlsson wrote: >> Looked at this quickly and I do not think adding it would be what I consider >> a small patch since we would essentially need to copy the validation logic >> from

Re: [HACKERS] Parallel safety tagging of extension functions

On Thu, Jun 2, 2016 at 7:36 AM, Andreas Karlsson wrote: > On 05/25/2016 03:32 AM, Tom Lane wrote: >> >> Andreas Karlsson writes: >>> >>> On 05/25/2016 02:41 AM, Robert Haas wrote: I'd rather extend see us ALTER AGGREGATE to do this. >> >> >>>

Re: [HACKERS] Parallel safety tagging of extension functions

On 05/25/2016 03:32 AM, Tom Lane wrote: Andreas Karlsson writes: On 05/25/2016 02:41 AM, Robert Haas wrote: I'd rather extend see us ALTER AGGREGATE to do this. Wouldn't that prevent this from going into 9.6? I do not think changing ALTER AGGREGATE is 9.6 materials.

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/01/2016 05:15 PM, Tom Lane wrote: Andreas Karlsson writes: On 06/01/2016 04:44 PM, Tom Lane wrote: I don't understand why you think you need the CREATE OR REPLACE FUNCTION commands? We only need to change proargtypes, and the updates did that. The catcache can take

Re: [HACKERS] Parallel safety tagging of extension functions

Andreas Karlsson writes: > On 06/01/2016 04:44 PM, Tom Lane wrote: >> I don't understand why you think you need the CREATE OR REPLACE FUNCTION >> commands? We only need to change proargtypes, and the updates did that. >> The catcache can take care of itself. > Maybe I did

Re: [HACKERS] Parallel safety tagging of extension functions

On 06/01/2016 04:44 PM, Tom Lane wrote: Andreas Karlsson writes: It is the least ugly of all the ugly solutions I could think of. I have attached a patch which fixes the signatures using this method. I use CREATE OR REPLACE FUNCTION to update to catcache. What do you think?

Re: [HACKERS] Parallel safety tagging of extension functions

Andreas Karlsson writes: > It is the least ugly of all the ugly solutions I could think of. I have > attached a patch which fixes the signatures using this method. I use > CREATE OR REPLACE FUNCTION to update to catcache. What do you think? Is > it too ugly? I don't

Re: [HACKERS] Parallel safety tagging of extension functions

On 05/31/2016 06:47 PM, Tom Lane wrote: Given that, your original approach of manually updating proargtypes in the existing pg_proc row for the functions may be the best way. Anything else is going to be more complicated and ultimately will still require at least one direct catalog update. It

Re: [HACKERS] Parallel safety tagging of extension functions

Andreas Karlsson writes: > So how to best change the function signatures? I do not think it is > possible without locking indexes by just using the SQL commands. You > cannot drop a function from the operator family without dropping the > operator class first. Ugh. I had

Re: [HACKERS] Parallel safety tagging of extension functions

On 05/25/2016 03:37 AM, Tom Lane wrote: Andreas Karlsson writes: Ok, then I can avoid touching all functions which are only called by operator classes, tsearch, pls, fdws, etc. Which also means that there is no need to care about Tom's changes to the signatures of GIN and

Re: [HACKERS] Parallel safety tagging of extension functions

All, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Andreas Karlsson writes: > > On 05/25/2016 02:41 AM, Robert Haas wrote: > >> I'd rather extend see us ALTER AGGREGATE to do this. > > > Wouldn't that prevent this from going into 9.6? I do not think changing > > ALTER AGGREGATE

Re: [HACKERS] Parallel safety tagging of extension functions

Andreas Karlsson writes: > Ok, then I can avoid touching all functions which are only called by > operator classes, tsearch, pls, fdws, etc. Which also means that there > is no need to care about Tom's changes to the signatures of GIN and GiST > support functions. I think

Re: [HACKERS] Parallel safety tagging of extension functions

Andreas Karlsson writes: > On 05/25/2016 02:41 AM, Robert Haas wrote: >> I'd rather extend see us ALTER AGGREGATE to do this. > Wouldn't that prevent this from going into 9.6? I do not think changing > ALTER AGGREGATE is 9.6 materials. Well, it's debatable --- but if the

Re: [HACKERS] Parallel safety tagging of extension functions

On 05/25/2016 03:09 AM, Robert Haas wrote: On Tue, May 24, 2016 at 8:41 PM, Robert Haas wrote: - Do you think we should add PARALLEL UNSAFE to the functions which we know are unsafe to make it obvious that it is intentional? That seems likely unnecessary churn from

Re: [HACKERS] Parallel safety tagging of extension functions

On 05/25/2016 02:41 AM, Robert Haas wrote: On Thu, May 19, 2016 at 5:50 PM, Andreas Karlsson wrote: - How should we modify the aggregate functions when upgrading extensions? ALTER AGGREGATE cannot change COMBINEFUNC or PARALLEL. My current patch updates the system catalogs

Re: [HACKERS] Parallel safety tagging of extension functions

On 05/25/2016 02:34 AM, Robert Haas wrote: On Fri, May 20, 2016 at 11:45 PM, Michael Paquier wrote: Sounds to me that this is part of the cleanup of a 9.6 feature and should be in that release. Yes, I agree. By the way, the patch completely ignores the fact that

Re: [HACKERS] Parallel safety tagging of extension functions

On Tue, May 24, 2016 at 8:41 PM, Robert Haas wrote: >> - Do you think we should add PARALLEL UNSAFE to the functions which we know >> are unsafe to make it obvious that it is intentional? > > That seems likely unnecessary churn from here. A general point here is that

Re: [HACKERS] Parallel safety tagging of extension functions

On Thu, May 19, 2016 at 5:50 PM, Andreas Karlsson wrote: > - How should we modify the aggregate functions when upgrading extensions? > ALTER AGGREGATE cannot change COMBINEFUNC or PARALLEL. My current patch > updates the system catalogs directly, which should be safe in this

Re: [HACKERS] Parallel safety tagging of extension functions

On Sat, May 21, 2016 at 1:01 PM, Andreas Karlsson wrote: > Another question which I thought of is what we should do with functions like > pg_file_write() in adminpack. > > While it is perfectly fine to modify files from the parallel workers, a user > could get race conditions

Re: [HACKERS] Parallel safety tagging of extension functions

On Fri, May 20, 2016 at 11:45 PM, Michael Paquier wrote: >> Sounds to me that this is part of the cleanup of a 9.6 feature and should be >> in that release. > > Yes, I agree. By the way, the patch completely ignores the fact that > some of the modules already had a

Re: [HACKERS] Parallel safety tagging of extension functions

Alvaro Herrera writes: > Tom Lane wrote: >> You'd have to alter the index opfamily to disconnect the function from it, >> drop/recreate the function, then re-add it to the opfamily. Kind of icky, >> but probably better than the alternatives. > What happens if the

Re: [HACKERS] Parallel safety tagging of extension functions

Tom Lane wrote: > Michael Paquier writes: > > On Sat, May 21, 2016 at 6:16 PM, Andreas Karlsson wrote: > >> My immediate thought is first doing an UPDATE of pg_proc and then updating > >> the catcache with CREATE OR REPLACE with the new arguments.

Re: [HACKERS] Parallel safety tagging of extension functions

Michael Paquier writes: > On Sat, May 21, 2016 at 6:16 PM, Andreas Karlsson wrote: >> My immediate thought is first doing an UPDATE of pg_proc and then updating >> the catcache with CREATE OR REPLACE with the new arguments. Does that work? >> Is

Re: [HACKERS] Parallel safety tagging of extension functions

On Sat, May 21, 2016 at 6:16 PM, Andreas Karlsson wrote: > My immediate thought is first doing an UPDATE of pg_proc and then updating > the catcache with CREATE OR REPLACE with the new arguments. Does that work? > Is there a less ugly way to accomplish this? > > Example: > >

Re: [HACKERS] Parallel safety tagging of extension functions

On 05/21/2016 11:45 AM, Tom Lane wrote: Yes, let's fix it. This will also take care of the questions about whether the GIN/GIST opclass tweaks I made a few months ago require module version bumps. Do you have any idea what the best way to add these tweaks to the upgrade scripts would be?

Re: [HACKERS] Parallel safety tagging of extension functions

On Sat, May 21, 2016 at 1:01 PM, Andreas Karlsson wrote: > Another question which I thought of is what we should do with functions like > pg_file_write() in adminpack. > > While it is perfectly fine to modify files from the parallel workers, a user > could get race conditions

Re: [HACKERS] Parallel safety tagging of extension functions

Another question which I thought of is what we should do with functions like pg_file_write() in adminpack. While it is perfectly fine to modify files from the parallel workers, a user could get race conditions if he tries to modify the same file multiple times. Is this a kind of problem the

Re: [HACKERS] Parallel safety tagging of extension functions

Peter Eisentraut writes: > On 5/20/16 7:37 PM, Robert Haas wrote: >> I guess my first question is whether we have consensus on the release >> into which we should put this. Some people (Noah, among others) >> thought it should wait because we're after feature

Re: [HACKERS] Parallel safety tagging of extension functions

On 05/20/2016 11:45 PM, Michael Paquier wrote: Yes, I agree. By the way, the patch completely ignores the fact that some of the modules already had a version bump in the 9.6 development cycle, like pageinpect. You don't need to create a new version script in such cases. I assumed this was too

Re: [HACKERS] Parallel safety tagging of extension functions

On Fri, May 20, 2016 at 10:30 PM, Peter Eisentraut wrote: > On 5/20/16 7:37 PM, Robert Haas wrote: >> >> I guess my first question is whether we have consensus on the release >> into which we should put this. Some people (Noah, among others) >> thought it should

Re: [HACKERS] Parallel safety tagging of extension functions

On 5/20/16 7:37 PM, Robert Haas wrote: I guess my first question is whether we have consensus on the release into which we should put this. Some people (Noah, among others) thought it should wait because we're after feature freeze, while others thought we should do it now. If we're going to

Re: [HACKERS] Parallel safety tagging of extension functions

On Thu, May 19, 2016 at 5:50 PM, Andreas Karlsson wrote: > I have gone through all our extensions and tried to tag all functions > correctly according to their parallel safety. > > I also did the same for the aggregate functions in a second patch, and for >

Re: [HACKERS] Parallel safety tagging of extension functions

On Thu, May 19, 2016 at 05:50:01PM -0400, Andreas Karlsson wrote: > Hi, > > I have gone through all our extensions and tried to tag all functions > correctly according to their parallel safety. > > I also did the same for the aggregate functions in a second patch, and for >