On Sun, Sep 01, 2013 at 05:08:38PM +0200, Andres Freund wrote: > On 2013-09-01 09:24:00 -0400, Noah Misch wrote: > > The difficulty depends on whether processes other than the segment's creator > > will attach anytime or only as they start. Attachment at startup is enough > > for parallel query, but it's not enough for something like lock table > > expansion. I'll focus on the attach-anytime case since it's more general. > > Even on startup it might get more complicated than one immediately > imagines on EXEC_BACKEND type platforms because their memory layout > doesn't need to be the same. The more shared memory you need, the harder > that will be. Afair
Non-Windows EXEC_BACKEND is already facing a dead end that way. > > On a system supporting MAP_FIXED, implement this by having the postmaster > > reserve address space under a PROT_NONE mapping, then carving out from that > > mapping for each fixed-address dynamic segment. The size of the reservation > > would be controlled by a GUC; one might set it to several times anticipated > > peak usage. (The overhead of doing that depends on the kernel.) Windows > > permits the same technique with its own primitives. > > Note that allocating a large mapping, even without using it, has > noticeable cost, at least under linux. The kernel has to create & copy > data to track each pages state (without copying the memory content's > itself due to COW) for every fork afterwards. If you don't believe me, > check the whole discussion about go's (the language) memory > management... I believe you, but I'd appreciate a link to the discussion you have in mind. > If that's the solution we go for why don't we just always include heaps > more shared memory and use that (remapping part of it as PROT_NONE) > instead of building the infrastructure in this patch? There would be no freeing of the memory; a usage high water mark would stand for the life of the postmaster. > > I don't foresee fundamental differences on 32-bit. All the allocation > > maximums scale down, but that's the usual story for 32-bit. > > If you actually want to allocate memory after starting up, without > carving a section out for that from the beginning, the memory > fragmentation will make it very hard to find memory addresses of the > same across processes. True. I wouldn't feel bad if total dynamic shared memory usage above, say, 256 MiB were unreliable on 32-bit. If you're still running 32-bit in 2015, you probably have a low-memory platform. I think the take-away is that we have a lot of knobs available, not a bright line between possible and impossible. Robert opted to omit provision for reliable fixed addresses, and the upsides of that decision are the absence of a DBA-unfriendly space-reservation GUC, trivial overhead when the APIs are not used, and a clearer portability outlook. -- Noah Misch EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
