On Sat, Feb 17, 2024 at 1:54 AM Heikki Linnakangas <hlinn...@iki.fi> wrote: > A variant of this approach: > > 5. Re-map the shared_buffers when needed. > > Between transactions, a backend should not hold any buffer pins. When > there are no pins, you can munmap() the shared_buffers and mmap() it at > a different address.
I really like this idea, but I think Andres has latched onto the key issue, which is that it supposes that the underlying shared memory object upon which shared_buffers is based can be made bigger and smaller, and that doesn't work for anonymous mappings AFAIK. Maybe that's not really a problem any more, though. If we don't depend on the address of shared_buffers anywhere, we could move it into a DSM. Now that the stats collector uses DSM, it's surely already a requirement that DSM works on every machine that runs PostgreSQL. We'd still need to do something about the buffer mapping table, though, and I bet dshash is not a reasonable answer on performance grounds. Also, it would be nice if the granularity of resizing could be something less than a whole transaction, because transactions can run for a long time. We don't really need to wait for a transaction boundary, probably -- a time when we hold zero buffer pins will probably happen a lot sooner, and at least some of those should be safe points at which to remap. Then again, somebody can open a cursor, read from it until it holds a pin, and then either idle the connection or make it do arbitrary amounts of unrelated work, forcing the remapping to be postponed for an arbitrarily long time. But some version of this problem will exist in any approach to this problem, and long-running pins are a nuisance for other reasons, too. We probably just have to accept this sort of issue as a limitation of our implementation. -- Robert Haas EDB: http://www.enterprisedb.com