Re: make MaxBackends available in _PG_init

2022-05-16 Thread Robert Haas
On Sun, May 15, 2022 at 8:58 AM Julien Rouhaud wrote: > For the record I submitted patches or pull requests this weekend for all > repositories I was aware of to fix the compatibility with this patch, hoping > that it will save some time to the packagers when they will take care of the > beta.

Re: make MaxBackends available in _PG_init

2022-05-15 Thread Julien Rouhaud
On Fri, May 13, 2022 at 09:49:54AM -0400, Robert Haas wrote: > > Committed. Thanks! For the record I submitted patches or pull requests this weekend for all repositories I was aware of to fix the compatibility with this patch, hoping that it will save some time to the packagers when they will

Re: make MaxBackends available in _PG_init

2022-05-13 Thread Nathan Bossart
On Fri, May 13, 2022 at 09:49:54AM -0400, Robert Haas wrote: > Committed. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: make MaxBackends available in _PG_init

2022-05-13 Thread Robert Haas
On Thu, May 12, 2022 at 8:15 PM Michael Paquier wrote: > On Thu, May 12, 2022 at 06:51:59PM +0300, Anton A. Melnikov wrote: > > Maybe remove the first part from the patchset? > > Because now the Patch Tester is giving apply error for the first part and > > can't test the other. > >

Re: make MaxBackends available in _PG_init

2022-05-12 Thread Michael Paquier
On Thu, May 12, 2022 at 06:51:59PM +0300, Anton A. Melnikov wrote: > Maybe remove the first part from the patchset? > Because now the Patch Tester is giving apply error for the first part and > can't test the other. > http://cfbot.cputube.org/patch_38_3614.log Yep. As simple as the attached. --

Re: make MaxBackends available in _PG_init

2022-05-12 Thread Anton A. Melnikov
On 12.05.2022 00:01, Nathan Bossart wrote: On Wed, May 11, 2022 at 04:18:10PM -0400, Robert Haas wrote: OK, I have committed 0001 now. Thanks! Maybe remove the first part from the patchset? Because now the Patch Tester is giving apply error for the first part and can't test the other.

Re: make MaxBackends available in _PG_init

2022-05-11 Thread Nathan Bossart
On Wed, May 11, 2022 at 04:18:10PM -0400, Robert Haas wrote: > OK, I have committed 0001 now. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: make MaxBackends available in _PG_init

2022-05-11 Thread Robert Haas
On Wed, May 11, 2022 at 12:12 AM Nathan Bossart wrote: > I swapped 0001 and 0002 in v8. > > Ah, good catch. I adjusted the docs as you suggested. OK, I have committed 0001 now. -- Robert Haas EDB: http://www.enterprisedb.com

Re: make MaxBackends available in _PG_init

2022-05-10 Thread Nathan Bossart
On Wed, May 11, 2022 at 11:18:29AM +0900, Michael Paquier wrote: > 0002 looks pretty good from here. If it were me, I would apply 0002 > first to avoid the extra tweak in pg_stat_statements with _PG_fini in > 0001. I swapped 0001 and 0002 in v8. > Regarding 0001, I have spotted an extra issue

Re: make MaxBackends available in _PG_init

2022-05-10 Thread Michael Paquier
On Tue, May 10, 2022 at 08:56:28AM -0700, Nathan Bossart wrote: > On Tue, May 10, 2022 at 05:55:12PM +0900, Michael Paquier wrote: >> I agree that removing support for the unloading part would be nice to >> clean up now on HEAD. Note that 0002 is missing the removal of one >> reference to

Re: make MaxBackends available in _PG_init

2022-05-10 Thread Nathan Bossart
On Tue, May 10, 2022 at 05:55:12PM +0900, Michael Paquier wrote: > I agree that removing support for the unloading part would be nice to > clean up now on HEAD. Note that 0002 is missing the removal of one > reference to _PG_fini in xfunc.sgml ( markup). Oops, sorry about that. -- Nathan

Re: make MaxBackends available in _PG_init

2022-05-10 Thread Michael Paquier
On Fri, May 06, 2022 at 07:51:43PM -0400, Robert Haas wrote: > On Fri, May 6, 2022 at 5:15 PM Nathan Bossart > wrote: >> On Fri, May 06, 2022 at 08:27:11AM -0700, Nathan Bossart wrote: >> > +1, I'll put together a new patch set. >> >> As promised... > > Looks reasonable to me. 0001 looks

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Robert Haas
On Fri, May 6, 2022 at 5:15 PM Nathan Bossart wrote: > On Fri, May 06, 2022 at 08:27:11AM -0700, Nathan Bossart wrote: > > +1, I'll put together a new patch set. > > As promised... Looks reasonable to me. Anyone else have thoughts? -- Robert Haas EDB: http://www.enterprisedb.com

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Nathan Bossart
On Fri, May 06, 2022 at 08:27:11AM -0700, Nathan Bossart wrote: > +1, I'll put together a new patch set. As promised... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 831218d6e0c04d6342bf593dbf6799efdd831b6b Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 18 Apr

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Nathan Bossart
On Fri, May 06, 2022 at 10:43:21AM -0400, Tom Lane wrote: > Robert Haas writes: >> On Fri, May 6, 2022 at 9:57 AM Tom Lane wrote: >>> I agree that _PG_fini functions as they stand are worthless. >>> What I'm not getting is why we should care enough about that >>> to break just about everybody's

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Tom Lane
Robert Haas writes: > On Fri, May 6, 2022 at 9:57 AM Tom Lane wrote: >> I agree that _PG_fini functions as they stand are worthless. >> What I'm not getting is why we should care enough about that >> to break just about everybody's extension. Even if unloading >> extensions were feasible, who

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Robert Haas
On Fri, May 6, 2022 at 9:57 AM Tom Lane wrote: > Robert Haas writes: > > perhaps for v16 or some future release we can think about redoing a > > bunch of hooks this way. There would be some migration pain for > > extension authors for sure, but then we might get to a point where > > extensions

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Tom Lane
Robert Haas writes: > perhaps for v16 or some future release we can think about redoing a > bunch of hooks this way. There would be some migration pain for > extension authors for sure, but then we might get to a point where > extensions can be cleanly unloaded in at least some circumstances, >

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Robert Haas
On Tue, Apr 19, 2022 at 11:47 AM Nathan Bossart wrote: > Okay, I did it this way in v5. I pushed 0001. Regarding 0002, I think there's no point in adding a _PG_fini(). The code to call _PG_fini() has been surrounded by #ifdef NOT_USED forever, and that seems unlikely to change any time soon as

Re: make MaxBackends available in _PG_init

2022-05-06 Thread Anton A. Melnikov
Hello! On 19.04.2022 18:46, Nathan Bossart wrote: Okay, I did it this way in v5. Recently i ran into a problem that it would be preferable in our extended pg_stat_statements to use MaxBackEnds in _PG_Init() but it's equal to zero here. And i was very happy to find this patch and this

Re: make MaxBackends available in _PG_init

2022-04-19 Thread Nathan Bossart
On Tue, Apr 19, 2022 at 05:49:13PM +0800, Julien Rouhaud wrote: > On Mon, Apr 18, 2022 at 08:17:04PM -0400, Tom Lane wrote: >> Nathan Bossart writes: >> > I'm looking for a clean way to ERROR if someone attempts to call >> > RequestAddinShmemSpace() or RequestNamedLWLockTranche() outside of the

Re: make MaxBackends available in _PG_init

2022-04-19 Thread Julien Rouhaud
Hi, On Mon, Apr 18, 2022 at 08:17:04PM -0400, Tom Lane wrote: > Nathan Bossart writes: > > I'm looking for a clean way to ERROR if someone attempts to call > > RequestAddinShmemSpace() or RequestNamedLWLockTranche() outside of the > > hook. Currently, we are using static variables in ipci.c and

Re: make MaxBackends available in _PG_init

2022-04-18 Thread Tom Lane
Nathan Bossart writes: > I'm looking for a clean way to ERROR if someone attempts to call > RequestAddinShmemSpace() or RequestNamedLWLockTranche() outside of the > hook. Currently, we are using static variables in ipci.c and lwlock.c to > silently ignore invalid requests. I could add a new

Re: make MaxBackends available in _PG_init

2022-04-18 Thread Nathan Bossart
On Mon, Apr 18, 2022 at 07:33:54PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> I noticed that requests for more LWLocks follow a similar pattern as >> regular shared memory requests, and I figured that we would want to do >> something similar for those, but I wasn't sure exactly how to

Re: make MaxBackends available in _PG_init

2022-04-18 Thread Tom Lane
Nathan Bossart writes: > I noticed that requests for more LWLocks follow a similar pattern as > regular shared memory requests, and I figured that we would want to do > something similar for those, but I wasn't sure exactly how to proceed. I > saw two options: 1) use shmem_request_hook for both

Re: make MaxBackends available in _PG_init

2022-04-18 Thread Nathan Bossart
On Thu, Apr 14, 2022 at 12:39:46PM -0400, Tom Lane wrote: > Robert Haas writes: >> On Thu, Apr 14, 2022 at 12:22 PM Nathan Bossart >> wrote: I'd be in favor of a hard break. > >>> Yeah, this is a good point. If we're okay with breaking existing >>> extensions like this, I will work on a

Re: make MaxBackends available in _PG_init

2022-04-14 Thread Tom Lane
Robert Haas writes: > On Thu, Apr 14, 2022 at 12:22 PM Nathan Bossart > wrote: >>> I'd be in favor of a hard break. >> Yeah, this is a good point. If we're okay with breaking existing >> extensions like this, I will work on a patch. > I tend to think it's a good idea. I've come around to

Re: make MaxBackends available in _PG_init

2022-04-14 Thread Robert Haas
On Thu, Apr 14, 2022 at 12:22 PM Nathan Bossart wrote: > > I'd be in favor of a hard break. There are already multiple extensions that > > relies on non final value of GUCs to size their shmem request. And as an > > extension author it can be hard to realize that, as those extensions work > >

Re: make MaxBackends available in _PG_init

2022-04-14 Thread Nathan Bossart
On Thu, Apr 14, 2022 at 01:50:24PM +0800, Julien Rouhaud wrote: > On Wed, Apr 13, 2022 at 11:30:40AM -0700, Nathan Bossart wrote: >> If we do move forward with the shmem request hook, do we want to disallow >> shmem requests anywhere else, or should we just leave it up to extension >> authors to

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Julien Rouhaud
On Wed, Apr 13, 2022 at 11:30:40AM -0700, Nathan Bossart wrote: > On Wed, Apr 13, 2022 at 12:05:08PM -0400, Tom Lane wrote: > > Robert Haas writes: > >> It may be too much to hope that we're going to completely get rid of > >> process_shared_preload_libraries_in_progress tests. > > > > Perhaps,

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Nathan Bossart
On Wed, Apr 13, 2022 at 12:05:08PM -0400, Tom Lane wrote: > Robert Haas writes: >> It may be too much to hope that we're going to completely get rid of >> process_shared_preload_libraries_in_progress tests. > > Perhaps, but this is definitely an area that could stand to have some > serious

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Tom Lane
Robert Haas writes: > What's a little wonky right now is that it's fairly common for > extensions to just return straight-off without doing *anything at all* > if !process_shared_preload_libraries_in_progress. See > pg_stat_statements for example. It seems kind of strange to me to make >

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 11:39 AM Tom Lane wrote: > Is there anything else typically done in _PG_init that has to be > conditional on process_shared_preload_libraries_in_progress? I recall > something about reserving LWLocks, which probably should get the same > treatment. If we can get to a

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Tom Lane
Robert Haas writes: > I guess my feeling about this is that _PG_init() is sort of a grab > bag, and that's not very good. You're supposed to register new GUCs > there, and request shared memory space, and install any hook functions > you want to use, and maybe some other stuff. But why is it

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 10:43 AM Tom Lane wrote: > Yeah, there is something to be said for preventing subtle interactions > between extensions. > > > So in the end I see basically four possibilities here: > > > A. No hard compatibility break, but invent a set-a-GUC hook that runs > > earlier. >

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Tom Lane
Robert Haas writes: > I wouldn't say that approach is wrong. I think we're just prioritizing > different things. I think that the pattern of wanting to request > shared memory in _PG_init() is common, and there are lots of > extensions that already do it that way, and if we pursue this plan, >

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 9:27 AM Tom Lane wrote: > Robert Haas writes: > > Hmm. I suppose I was thinking that we'd go the other way around: move > > the call to InitializeMaxBackends() earlier, as proposed previously, > > and add a hook to allow extensions to get control before that point. > >

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Tom Lane
Robert Haas writes: > Hmm. I suppose I was thinking that we'd go the other way around: move > the call to InitializeMaxBackends() earlier, as proposed previously, > and add a hook to allow extensions to get control before that point. > The reason that I like that approach is that I suspect that

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Robert Haas
On Tue, Apr 12, 2022 at 6:26 PM Nathan Bossart wrote: > Okay. So maybe we only need the attached patches. 0001 is just 5ecd018 > un-reverted, and 0002 is Julien's patch from a few weeks ago with some > minor edits. Hmm. I suppose I was thinking that we'd go the other way around: move the call

Re: make MaxBackends available in _PG_init

2022-04-13 Thread Michael Paquier
On Tue, Apr 12, 2022 at 05:43:17PM -0400, Tom Lane wrote: > It probably is. I'm just offering this as a solution if people want to > insist on a mechanism to prevent unsafe GUC changes. If we drop the > idea of trying to forcibly prevent extensions from Doing It Wrong, > then we don't need this,

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Nathan Bossart
On Tue, Apr 12, 2022 at 05:46:36PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> If we allow changing GUCs in _PG_init() and provide another hook where >> MaxBackends will be initialized, do we need to introduce another GUC flag, >> or can we get away with just blocking all GUC changes when

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Tom Lane
Nathan Bossart writes: > If we allow changing GUCs in _PG_init() and provide another hook where > MaxBackends will be initialized, do we need to introduce another GUC flag, > or can we get away with just blocking all GUC changes when the new hook is > called? I'm slightly hesitant to add a GUC

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 12, 2022 at 4:58 PM Tom Lane wrote: >> It seems after a bit of reflection that what we ought to do is identify >> the subset of PGC_POSTMASTER GUCs that feed into shared memory sizing, >> mark those with a new GUC flag, and not allow them to be changed after >>

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Robert Haas
On Tue, Apr 12, 2022 at 4:58 PM Tom Lane wrote: > It's reasonable to allow changing *most* GUCs in _PG_init(); let us > please not break cases that work fine today. > > It seems after a bit of reflection that what we ought to do is identify > the subset of PGC_POSTMASTER GUCs that feed into

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Nathan Bossart
On Tue, Apr 12, 2022 at 04:58:42PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> I think it'd be reasonable to allow changing custom GUCs in _PG_init(). >> Those aren't going to impact things like MaxBackends. > > It's reasonable to allow changing *most* GUCs in _PG_init(); let us > please

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Nathan Bossart
On Tue, Apr 12, 2022 at 04:33:39PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> On Tue, Apr 12, 2022 at 03:12:42PM -0400, Robert Haas wrote: >>> But if there's even one use case where adjusting GUCs at this phase is >>> reasonable, then 0003 isn't really good enough. We need an 0004 that

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Tom Lane
Nathan Bossart writes: > I think it'd be reasonable to allow changing custom GUCs in _PG_init(). > Those aren't going to impact things like MaxBackends. It's reasonable to allow changing *most* GUCs in _PG_init(); let us please not break cases that work fine today. It seems after a bit of

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Tom Lane
Nathan Bossart writes: > On Tue, Apr 12, 2022 at 03:12:42PM -0400, Robert Haas wrote: >> But if there's even one use case where adjusting GUCs at this phase is >> reasonable, then 0003 isn't really good enough. We need an 0004 that >> provides a new hook in a place where such changes can safely

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Nathan Bossart
On Tue, Apr 12, 2022 at 03:30:23PM -0400, Robert Haas wrote: > On Tue, Apr 12, 2022 at 3:22 PM Tom Lane wrote: >> Yeah. It's a very long way from "changing shared memory sizes here >> doesn't work" to "no extension is allowed to change any GUC at load >> time". A counterexample is that it's not

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Nathan Bossart
On Tue, Apr 12, 2022 at 03:12:42PM -0400, Robert Haas wrote: > But if there's even one use case where adjusting GUCs at this phase is > reasonable, then 0003 isn't really good enough. We need an 0004 that > provides a new hook in a place where such changes can safely be made. I think that is

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Robert Haas
On Tue, Apr 12, 2022 at 3:22 PM Tom Lane wrote: > Yeah. It's a very long way from "changing shared memory sizes here > doesn't work" to "no extension is allowed to change any GUC at load > time". A counterexample is that it's not clear why an extension > shouldn't be allowed to define a GUC

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Tom Lane
Robert Haas writes: > Gah, I hate putting this off for another year, but I guess I'm also > not convinced that 0003 has the right idea, so maybe it's for the > best. Here's what's bugging me: 0003 decrees that _PG_init() is the > Wrong Place to Adjust GUC Settings. Now, that begs the question:

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Robert Haas
On Tue, Apr 12, 2022 at 2:03 PM Nathan Bossart wrote: > On Tue, Apr 12, 2022 at 10:44:27AM -0700, Andres Freund wrote: > > On 2022-04-11 14:14:35 -0700, Nathan Bossart wrote: > >> Here's a new patch set. I've only changed 0003 as described above. My > >> apologies for the unnecessary round

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Nathan Bossart
On Tue, Apr 12, 2022 at 10:44:27AM -0700, Andres Freund wrote: > On 2022-04-11 14:14:35 -0700, Nathan Bossart wrote: >> Here's a new patch set. I've only changed 0003 as described above. My >> apologies for the unnecessary round trip. > > ISTM we shouldn't apply 0002, 0003 to master before

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Andres Freund
Hi, On 2022-04-11 14:14:35 -0700, Nathan Bossart wrote: > On Mon, Apr 11, 2022 at 01:44:42PM -0700, Nathan Bossart wrote: > > On Mon, Apr 11, 2022 at 04:36:36PM -0400, Robert Haas wrote: > >> If we throw an error while defining_custom_guc is true, how will it > >> ever again become false? > > >

Re: make MaxBackends available in _PG_init

2022-04-12 Thread Nathan Bossart
On Tue, Apr 12, 2022 at 01:08:35PM +0800, Julien Rouhaud wrote: > It looks sensible to me, although I think I would apply 0003 before 0002. Great. > + if (process_shared_preload_libraries_in_progress && > + !allow_when_loading_preload_libs) > + ereport(ERROR, > +

Re: make MaxBackends available in _PG_init

2022-04-11 Thread Julien Rouhaud
Hi, On Mon, Apr 11, 2022 at 02:14:35PM -0700, Nathan Bossart wrote: > On Mon, Apr 11, 2022 at 01:44:42PM -0700, Nathan Bossart wrote: > > On Mon, Apr 11, 2022 at 04:36:36PM -0400, Robert Haas wrote: > >> If we throw an error while defining_custom_guc is true, how will it > >> ever again become

Re: make MaxBackends available in _PG_init

2022-04-11 Thread Nathan Bossart
On Mon, Apr 11, 2022 at 01:44:42PM -0700, Nathan Bossart wrote: > On Mon, Apr 11, 2022 at 04:36:36PM -0400, Robert Haas wrote: >> If we throw an error while defining_custom_guc is true, how will it >> ever again become false? > > Ah, I knew I was forgetting something this morning. > > It won't,

Re: make MaxBackends available in _PG_init

2022-04-11 Thread Nathan Bossart
On Mon, Apr 11, 2022 at 04:36:36PM -0400, Robert Haas wrote: > On Mon, Apr 11, 2022 at 12:44 PM Nathan Bossart > wrote: >> Here are some patches. 0001 reverts all the recent commits in this area, >> 0002 is the patch I posted in August, and 0003 is an attempt at blocking >> GUC changes in

Re: make MaxBackends available in _PG_init

2022-04-11 Thread Robert Haas
On Mon, Apr 11, 2022 at 12:44 PM Nathan Bossart wrote: > Here are some patches. 0001 reverts all the recent commits in this area, > 0002 is the patch I posted in August, and 0003 is an attempt at blocking > GUC changes in preloaded libraries' _PG_init() functions. If we throw an error while

Re: make MaxBackends available in _PG_init

2022-04-11 Thread Nathan Bossart
On Mon, Apr 11, 2022 at 10:47:17AM -0400, Robert Haas wrote: > On Sat, Apr 9, 2022 at 9:24 AM Julien Rouhaud wrote: >> > FWIW I would be on board with reverting all the GetMaxBackends() stuff if >> > we made the value available in _PG_init() and stopped supporting GUC >> > overrides by extensions

Re: make MaxBackends available in _PG_init

2022-04-11 Thread Julien Rouhaud
Hi, On Mon, Apr 11, 2022 at 10:47:17AM -0400, Robert Haas wrote: > On Sat, Apr 9, 2022 at 9:24 AM Julien Rouhaud wrote: > > > On the bright side, I see that citus is using SetConfigOption() to increase > > max_prepared_transactions [1]. That's the only extension mentioned in that > > thread

Re: make MaxBackends available in _PG_init

2022-04-11 Thread Robert Haas
On Sat, Apr 9, 2022 at 9:24 AM Julien Rouhaud wrote: > > FWIW I would be on board with reverting all the GetMaxBackends() stuff if > > we made the value available in _PG_init() and stopped supporting GUC > > overrides by extensions (e.g., ERROR-ing in SetConfigOption() when > >

Re: make MaxBackends available in _PG_init

2022-04-10 Thread Michael Paquier
On Sat, Apr 09, 2022 at 09:24:42PM +0800, Julien Rouhaud wrote: > Yeah I would prefer this approach too, although it couldn't prevent extension > from directly modifying the underlying variables so I don't know how effective > it would be. > > On the bright side, I see that citus is using

Re: make MaxBackends available in _PG_init

2022-04-09 Thread Julien Rouhaud
On Wed, Mar 30, 2022 at 09:30:51AM -0700, Nathan Bossart wrote: > On Tue, Mar 29, 2022 at 12:22:21PM -0400, Robert Haas wrote: > > It's not, though, because the original proposal was to change things > > around so that the value of MaxBackends would have been reliable in > > _PG_init(). If we'd

Re: make MaxBackends available in _PG_init

2022-03-30 Thread Nathan Bossart
On Tue, Mar 29, 2022 at 12:22:21PM -0400, Robert Haas wrote: > It's not, though, because the original proposal was to change things > around so that the value of MaxBackends would have been reliable in > _PG_init(). If we'd done that, then extensions that are using it in > _PG_init() would have

Re: make MaxBackends available in _PG_init

2022-03-29 Thread Robert Haas
On Sat, Mar 26, 2022 at 1:23 PM Andres Freund wrote: > > And indeed, any third party code that previously needed to access what > > MaxBackends is supposed to store should already be using that formula, and > > the new GetMaxBackends() doesn't do anything about it. > > It couldn't rely on

Re: make MaxBackends available in _PG_init

2022-03-26 Thread Julien Rouhaud
On Sat, Mar 26, 2022 at 10:23:16AM -0700, Andres Freund wrote: > > On 2022-03-26 15:22:03 +0800, Julien Rouhaud wrote: > > On Fri, Mar 25, 2022 at 03:23:17PM -0700, Andres Freund wrote: > > > > > > I don't really understand. The issue that started this thread was bugs in > > > extensions due to

Re: make MaxBackends available in _PG_init

2022-03-26 Thread Andres Freund
Hi, On 2022-03-26 15:22:03 +0800, Julien Rouhaud wrote: > On Fri, Mar 25, 2022 at 03:23:17PM -0700, Andres Freund wrote: > > > > I don't really understand. The issue that started this thread was bugs in > > extensions due to accessing MaxBackends before it is initialized - which the > > patch

Re: make MaxBackends available in _PG_init

2022-03-26 Thread Julien Rouhaud
Hi, On Fri, Mar 25, 2022 at 03:23:17PM -0700, Andres Freund wrote: > > I don't really understand. The issue that started this thread was bugs in > extensions due to accessing MaxBackends before it is initialized - which the > patch prevents. Well, the patch prevents accessing a 0-valued

Re: make MaxBackends available in _PG_init

2022-03-25 Thread Andres Freund
Hi, On 2022-03-25 14:35:42 +0800, Julien Rouhaud wrote: > On Fri, Mar 25, 2022 at 02:08:09PM +0900, Michael Paquier wrote: > > On Fri, Mar 25, 2022 at 11:11:46AM +0800, Julien Rouhaud wrote: > > > As an example, here's a POC for a new shmem_request_hook hook after > > > _PG_init(). > > > With it

Re: make MaxBackends available in _PG_init

2022-03-25 Thread Julien Rouhaud
On Fri, Mar 25, 2022 at 02:08:09PM +0900, Michael Paquier wrote: > On Fri, Mar 25, 2022 at 11:11:46AM +0800, Julien Rouhaud wrote: > > As an example, here's a POC for a new shmem_request_hook hook after > > _PG_init(). > > With it I could easily fix pg_wait_sampling shmem allocation (and checked

Re: make MaxBackends available in _PG_init

2022-03-24 Thread Michael Paquier
On Fri, Mar 25, 2022 at 11:11:46AM +0800, Julien Rouhaud wrote: > As an example, here's a POC for a new shmem_request_hook hook after > _PG_init(). > With it I could easily fix pg_wait_sampling shmem allocation (and checked that > it's indeed requesting the correct size). Are you sure that the

Re: make MaxBackends available in _PG_init

2022-03-24 Thread Julien Rouhaud
On Fri, Mar 25, 2022 at 10:39:51AM +0800, Julien Rouhaud wrote: > On Thu, Mar 24, 2022 at 04:27:36PM -0400, Robert Haas wrote: > > On Thu, Mar 24, 2022 at 4:20 PM Nathan Bossart > > wrote: > > > Another possibility could be to add a hook that is called _before_ > > > _PG_init() where libraries

Re: make MaxBackends available in _PG_init

2022-03-24 Thread Julien Rouhaud
On Thu, Mar 24, 2022 at 04:27:36PM -0400, Robert Haas wrote: > On Thu, Mar 24, 2022 at 4:20 PM Nathan Bossart > wrote: > > Another possibility could be to add a hook that is called _before_ > > _PG_init() where libraries are permitted to adjust GUCs. After the library > > is loaded, we first

Re: make MaxBackends available in _PG_init

2022-03-24 Thread Robert Haas
On Thu, Mar 24, 2022 at 4:20 PM Nathan Bossart wrote: > Another possibility could be to add a hook that is called _before_ > _PG_init() where libraries are permitted to adjust GUCs. After the library > is loaded, we first call this _PG_change_GUCs() function, then we > initialize MaxBackends,

Re: make MaxBackends available in _PG_init

2022-03-24 Thread Nathan Bossart
On Wed, Mar 23, 2022 at 09:03:18PM +0800, Julien Rouhaud wrote: > On Wed, Mar 23, 2022 at 08:32:39AM -0400, Robert Haas wrote: >> Well, the conclusion upthread was that extensions might change the >> values of those GUCs from _PG_init(). If that's a real thing, then >> what you're asking for here

Re: make MaxBackends available in _PG_init

2022-03-23 Thread Julien Rouhaud
On Wed, Mar 23, 2022 at 08:32:39AM -0400, Robert Haas wrote: > On Wed, Mar 23, 2022 at 12:53 AM Julien Rouhaud wrote: > > Unless I'm missing something, the new situation is that the system is > > supposed > > to prevent access to MaxBackends during s_p_l_pg_init, for reasons I totally > > agree

Re: make MaxBackends available in _PG_init

2022-03-23 Thread Robert Haas
On Wed, Mar 23, 2022 at 12:53 AM Julien Rouhaud wrote: > Unless I'm missing something, the new situation is that the system is supposed > to prevent access to MaxBackends during s_p_l_pg_init, for reasons I totally > agree with, but without doing anything for extensions that actually need to >

Re: make MaxBackends available in _PG_init

2022-03-22 Thread Julien Rouhaud
Hi, Sorry for showing up this late, but I'm a bit confused with the new situation. Unless I'm missing something, the new situation is that the system is supposed to prevent access to MaxBackends during s_p_l_pg_init, for reasons I totally agree with, but without doing anything for extensions

Re: make MaxBackends available in _PG_init

2022-02-10 Thread Nathan Bossart
On Thu, Feb 10, 2022 at 10:47:39AM +0900, Michael Paquier wrote: > On Wed, Feb 09, 2022 at 09:53:38AM -0800, Nathan Bossart wrote: >> This is fine with me as well. I only left these out because the extra >> variable felt unnecessary to me for these functions. > > Okay, done, then. > >> While

Re: make MaxBackends available in _PG_init

2022-02-09 Thread Michael Paquier
On Wed, Feb 09, 2022 at 09:53:38AM -0800, Nathan Bossart wrote: > On Wed, Feb 09, 2022 at 12:31:16PM -0500, Robert Haas wrote: >> On Wed, Feb 9, 2022 at 12:18 AM Michael Paquier wrote: >>> Okay, I'd rather apply the same rule everywhere for consistency, then, >>> like in the attached. That's

Re: make MaxBackends available in _PG_init

2022-02-09 Thread Nathan Bossart
On Wed, Feb 09, 2022 at 12:31:16PM -0500, Robert Haas wrote: > On Wed, Feb 9, 2022 at 12:18 AM Michael Paquier wrote: >> Okay, I'd rather apply the same rule everywhere for consistency, then, >> like in the attached. That's minimal, still. > > That's fine with me. In the interest of full

Re: make MaxBackends available in _PG_init

2022-02-09 Thread Nathan Bossart
On Tue, Feb 08, 2022 at 04:12:26PM -0500, Robert Haas wrote: > After some investigation I've determined that it's no longer Friday > afternoon. This matches my analysis. > I also spent time investigating whether the patch had > problems that would make me uncomfortable with the idea of

Re: make MaxBackends available in _PG_init

2022-02-09 Thread Robert Haas
On Wed, Feb 9, 2022 at 12:18 AM Michael Paquier wrote: > Okay, I'd rather apply the same rule everywhere for consistency, then, > like in the attached. That's minimal, still. That's fine with me. In the interest of full disclosure, I did kind of notice this when reviewing the patch, though

Re: make MaxBackends available in _PG_init

2022-02-08 Thread Michael Paquier
On Tue, Feb 08, 2022 at 10:57:37PM -0500, Robert Haas wrote: > Well I didn't do anything myself except review and commit Nathan's > patch, so I suppose you mean he could have done that, but fair enough. > I don't mind if you want to change it around. Okay, I'd rather apply the same rule

Re: make MaxBackends available in _PG_init

2022-02-08 Thread Robert Haas
On Tue, Feb 8, 2022 at 9:38 PM Michael Paquier wrote: > On Tue, Feb 08, 2022 at 04:12:26PM -0500, Robert Haas wrote: > > After some investigation I've determined that it's no longer Friday > > afternoon. I also spent time investigating whether the patch had > > problems that would make me

Re: make MaxBackends available in _PG_init

2022-02-08 Thread Michael Paquier
On Tue, Feb 08, 2022 at 04:12:26PM -0500, Robert Haas wrote: > After some investigation I've determined that it's no longer Friday > afternoon. I also spent time investigating whether the patch had > problems that would make me uncomfortable with the idea of committing > it, and I did not find

Re: make MaxBackends available in _PG_init

2022-02-08 Thread Robert Haas
On Fri, Feb 4, 2022 at 3:27 PM Robert Haas wrote: > Great. I'll take a look at this next week, but not right now, mostly > because it's Friday afternoon and if I commit it and anything breaks I > don't want to end up having to fix it on the weekend if I can avoid > it. After some investigation

Re: make MaxBackends available in _PG_init

2022-02-04 Thread Robert Haas
On Fri, Feb 4, 2022 at 3:13 PM Nathan Bossart wrote: > On Fri, Feb 04, 2022 at 08:46:39AM -0500, Robert Haas wrote: > > For multixact.c, I think you should invent GetMaxOldestSlot() to avoid > > confusion. Maybe it could be a static inline rather than a macro. > > > > Likewise, I think

Re: make MaxBackends available in _PG_init

2022-02-04 Thread Nathan Bossart
On Fri, Feb 04, 2022 at 08:46:39AM -0500, Robert Haas wrote: > For multixact.c, I think you should invent GetMaxOldestSlot() to avoid > confusion. Maybe it could be a static inline rather than a macro. > > Likewise, I think PROCARRAY_MAXPROCS, NumProcSignalSlots, and > NumBackendStatSlots should

Re: make MaxBackends available in _PG_init

2022-02-04 Thread Robert Haas
On Fri, Feb 4, 2022 at 12:55 AM Nathan Bossart wrote: > Here is a new patch with fewer calls to GetMaxBackends(). For multixact.c, I think you should invent GetMaxOldestSlot() to avoid confusion. Maybe it could be a static inline rather than a macro. Likewise, I think PROCARRAY_MAXPROCS,

Re: make MaxBackends available in _PG_init

2022-02-03 Thread Nathan Bossart
On Wed, Feb 02, 2022 at 08:15:02AM -0500, Robert Haas wrote: > On Tue, Feb 1, 2022 at 5:36 PM Nathan Bossart > wrote: >> I can work on a new patch if this is the direction we want to go. There >> were a couple of functions that called GetMaxBackends() repetitively that I >> should probably fix

Re: make MaxBackends available in _PG_init

2022-02-02 Thread Robert Haas
On Tue, Feb 1, 2022 at 5:36 PM Nathan Bossart wrote: > I can work on a new patch if this is the direction we want to go. There > were a couple of functions that called GetMaxBackends() repetitively that I > should probably fix before the patch should be seriously considered. Sure, that sort of

Re: make MaxBackends available in _PG_init

2022-02-01 Thread Nathan Bossart
On Mon, Jan 31, 2022 at 09:32:21AM -0500, Robert Haas wrote: > The main reason why this doesn't work for MaxBackends is that > MaxBackends depends on the values of multiple GUCs. There is a further > wrinkle too, which is that none of those GUCs can change, and > therefore code does things with

Re: make MaxBackends available in _PG_init

2022-01-31 Thread Robert Haas
On Fri, Jan 28, 2022 at 9:19 PM Michael Paquier wrote: > As of the issues of this thread, we really have two things to think > about: > 1) How do we want to control the access of some parameters in a > context or another? One idea would be more control through GUCs, say > with a set of

Re: make MaxBackends available in _PG_init

2022-01-30 Thread Michael Paquier
On Sat, Jan 29, 2022 at 02:24:24PM +0900, Michael Paquier wrote: > Yeah, it would be good to know the scope before defining the limits > of what could be done. Another thing may be the interactions with > session_preload_libraries and local_preload_libraries. Worth noting that I have marked

Re: make MaxBackends available in _PG_init

2022-01-28 Thread Michael Paquier
On Fri, Jan 28, 2022 at 08:33:42PM -0800, Nathan Bossart wrote: > Hm. Perhaps we should understand the full scope of the problem first. > What else besides MaxBackends and the shared memory GUCs won't be properly > initialized when the shared_preload_libraries' _PG_init() functions are > called?

Re: make MaxBackends available in _PG_init

2022-01-28 Thread Nathan Bossart
On Sat, Jan 29, 2022 at 11:19:12AM +0900, Michael Paquier wrote: > On Thu, Jan 27, 2022 at 10:18:15AM -0800, Nathan Bossart wrote: >> Alright. I think the comment adjustments still apply, so I split those out >> to a new patch. > > Looks fine after a second look, so applied. Thanks! > As of

Re: make MaxBackends available in _PG_init

2022-01-28 Thread Michael Paquier
On Thu, Jan 27, 2022 at 10:18:15AM -0800, Nathan Bossart wrote: > Alright. I think the comment adjustments still apply, so I split those out > to a new patch. Looks fine after a second look, so applied. As of the issues of this thread, we really have two things to think about: 1) How do we want

  1   2   >