On 29/07/2024 19:56, Robert Haas wrote:
On Wed, Jul 24, 2024 at 8:58 AM Heikki Linnakangas <hlinn...@iki.fi> wrote:
a) assign every child process a PGPROC entry, and make postmaster
responsible for assigning them like you suggest. We'll need more PGPROC
entries, because currently a process doesn't reserve one until
authentication has happened. Or we change that behavior.
I wonder how this works right now. Is there something that limits the
number of authentication requests that can be in flight concurrently,
or is it completely uncapped (except by machine resources)?
My first thought when I read this was that it would be bad to have to
put a limit on something that's currently unlimited. But then I
started thinking that, even if it is currently unlimited, that might
be a bug rather than a feature. If you have hundreds of pending
authentication requests, that just means you're using a lot of machine
resources on something that doesn't really help anybody. A machine
with hundreds of authentication-pending connections is possibly
getting DDOS'd and probably getting buried. You'd be better off
focusing the machine's limited resources on the already-established
connections and a more limited number of new connection attempts. If
you accept so many connection attempts that you don't actually have
enough memory/CPU/kernel scheduling firepower to complete the
authentication process with any of them, it does nobody any good.
I'm not sure what's best to do here; just thinking out loud.
Yes, there's a limit, roughly 2x max_connections. see
MaxLivePostmasterChildren().
There's another issue with that that I was about to post in another
thread, but here goes: the MaxLivePostmasterChildren() limit is shared
by all regular backends, bgworkers and autovacuum workers. If you open a
lot of TCP connections to postmaster and don't send anything to the
server, you exhaust those slots, and the server won't be able to start
any autovacuum workers or background workers either. That's not great. I
started to work on approach b), with separate pools of slots for
different kinds of child processes, which fixes that. Stay tuned for a
patch.
In addition to that, you can have an unlimited number of "dead-end"
backends, which are doomed to just respond with "sorry, too many
clients" error. The only limit on those is the amount of resources
needed for all the processes and a little memory to track them.
--
Heikki Linnakangas
Neon (https://neon.tech)
