Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Andres Freund
Hi, On 2024-02-19 13:54:01 -0500, Joe Conway wrote: > On 2/19/24 13:13, Andres Freund wrote: > > On 2024-02-19 09:19:16 -0500, Joe Conway wrote: > > > Couldn't we scale the rounding, e.g. allow small allocations as we do now, > > > but above some number always round? E.g. maybe >= 2GB round to

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Joe Conway
On 2/19/24 13:13, Andres Freund wrote: On 2024-02-19 09:19:16 -0500, Joe Conway wrote: Couldn't we scale the rounding, e.g. allow small allocations as we do now, but above some number always round? E.g. maybe >= 2GB round to the nearest 256MB, >= 4GB round to the nearest 512MB, >= 8GB round to

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Andres Freund
Hi, On 2024-02-19 09:19:16 -0500, Joe Conway wrote: > On 2/18/24 15:35, Andres Freund wrote: > > On 2024-02-18 17:06:09 +0530, Robert Haas wrote: > > > How many people set shared_buffers to something that's not a whole > > > number of GB these days? > > > > I'd say the vast majority of postgres

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Joe Conway
On 2/18/24 15:35, Andres Freund wrote: On 2024-02-18 17:06:09 +0530, Robert Haas wrote: How many people set shared_buffers to something that's not a whole number of GB these days? I'd say the vast majority of postgres instances in production run with less than 1GB of s_b. Just because numbers

Re: PGC_SIGHUP shared_buffers?

2024-02-18 Thread Robert Haas
On Mon, Feb 19, 2024 at 2:05 AM Andres Freund wrote: > We probably should address that independently of making shared_buffers > PGC_SIGHUP. The queue gets absurdly large once s_b hits a few GB. It's not > that much memory compared to the buffer blocks themselves, but a sync queue of > many

Re: PGC_SIGHUP shared_buffers?

2024-02-18 Thread Andres Freund
Hi, On 2024-02-18 17:06:09 +0530, Robert Haas wrote: > On Sat, Feb 17, 2024 at 12:38 AM Andres Freund wrote: > > IMO the ability to *shrink* shared_buffers dynamically and cheaply is more > > important than growing it in a way, except that they are related of > > course. Idling hardware is

Re: PGC_SIGHUP shared_buffers?

2024-02-18 Thread Matthias van de Meent
On Sun, 18 Feb 2024 at 02:03, Andres Freund wrote: > > Hi, > > On 2024-02-17 23:40:51 +0100, Matthias van de Meent wrote: > > > 5. Re-map the shared_buffers when needed. > > > > > > Between transactions, a backend should not hold any buffer pins. When > > > there are no pins, you can munmap() the

Re: PGC_SIGHUP shared_buffers?

2024-02-18 Thread Konstantin Knizhnik
On 16/02/2024 10:37 pm, Thomas Munro wrote: On Fri, Feb 16, 2024 at 5:29 PM Robert Haas wrote: 3. Reserve lots of address space and then only use some of it. I hear rumors that some forks of PG have implemented something like this. The idea is that you convince the OS to give you a whole

Re: PGC_SIGHUP shared_buffers?

2024-02-18 Thread Robert Haas
On Sat, Feb 17, 2024 at 1:54 AM Heikki Linnakangas wrote: > A variant of this approach: > > 5. Re-map the shared_buffers when needed. > > Between transactions, a backend should not hold any buffer pins. When > there are no pins, you can munmap() the shared_buffers and mmap() it at > a different

Re: PGC_SIGHUP shared_buffers?

2024-02-18 Thread Robert Haas
On Sat, Feb 17, 2024 at 12:38 AM Andres Freund wrote: > IMO the ability to *shrink* shared_buffers dynamically and cheaply is more > important than growing it in a way, except that they are related of > course. Idling hardware is expensive, thus overcommitting hardware is very > attractive (I

Re: PGC_SIGHUP shared_buffers?

2024-02-17 Thread Andres Freund
Hi, On 2024-02-17 23:40:51 +0100, Matthias van de Meent wrote: > > 5. Re-map the shared_buffers when needed. > > > > Between transactions, a backend should not hold any buffer pins. When > > there are no pins, you can munmap() the shared_buffers and mmap() it at > > a different address. I hadn't

Re: PGC_SIGHUP shared_buffers?

2024-02-17 Thread Matthias van de Meent
On Fri, 16 Feb 2024 at 21:24, Heikki Linnakangas wrote: > > On 16/02/2024 06:28, Robert Haas wrote: > > 3. Reserve lots of address space and then only use some of it. I hear > > rumors that some forks of PG have implemented something like this. The > > idea is that you convince the OS to give you

Re: PGC_SIGHUP shared_buffers?

2024-02-16 Thread Thomas Munro
On Fri, Feb 16, 2024 at 5:29 PM Robert Haas wrote: > 3. Reserve lots of address space and then only use some of it. I hear > rumors that some forks of PG have implemented something like this. The > idea is that you convince the OS to give you a whole bunch of address > space, but you try to avoid

Re: PGC_SIGHUP shared_buffers?

2024-02-16 Thread Heikki Linnakangas
On 16/02/2024 06:28, Robert Haas wrote: 3. Reserve lots of address space and then only use some of it. I hear rumors that some forks of PG have implemented something like this. The idea is that you convince the OS to give you a whole bunch of address space, but you try to avoid having all of it

Re: PGC_SIGHUP shared_buffers?

2024-02-16 Thread Andres Freund
Hi, On 2024-02-16 09:58:43 +0530, Robert Haas wrote: > I remember Magnus making a comment many years ago to the effect that > every setting that is PGC_POSTMASTER is a bug, but some of those bugs > are very difficult to fix. Perhaps the use of the word bug is > arguable, but I think the sentiment

PGC_SIGHUP shared_buffers?

2024-02-15 Thread Robert Haas
Hi, I remember Magnus making a comment many years ago to the effect that every setting that is PGC_POSTMASTER is a bug, but some of those bugs are very difficult to fix. Perhaps the use of the word bug is arguable, but I think the sentiment is apt, especially with regard to shared_buffers.