Another way of looking at it is memory mapped files. This probably
most
closely resembles unix shared memory and is the de facto standard
way
for interprocess memory block sharing. Sadly, performance will
suffer
because you have to rely on the virtual memory system (think:
writing to
files) to do a lot of stupid stuff you don't necessarily want or
need.
To the contrary, for the majority of the shared memory usage of
postgres, which is cached file data, the virtual memory system is
doing
exactly what you want it to: managing the movement of data between
memory and disk, and caching the more frequently accessed data to
reduce
the chances you will actually need to access the disk for it.
Yes. Generally, I was trying to point out the disadvantages of memory
mapped files compared to shared memory. In windows, there is no direct
equivalent so shared memory. MMFs are very similar in usage. I suspect
they might not perform quite as well as the shared memory functions.
For example, if used in place of shared memory to cache static file
data, you are maintaining:
1. the file itself,
2. the file cache handled by the os.
3. the MMF memory side cache (following a page fault).
4. the virtual memory space set aside for the os to swap it out should
the os need more memory.
MMFs are efficient when memory allocations are relatively static: they
work especially well with a freestore memory allocation system (this
minimizes movement inside the virtual memory pagefile). For example,
the MMF is allocated at the startup of the backend and doled out to
processes through an internal 'as needed' basis. This is equivalent in
function to memory allocations using the VirtualAlloc() family except
its good for IPC. (IMHO, it will still run slower).
If memory allocations are frequent and dynamic, you start to run into
problems with fragmentation of the pagefile and such problems. This is
very undesirable. Also, if memory allocations are large, you could
potentially run into the worst possible scenario: your file cache system
is competing with the virtual memory system. This will cause the server
to thrash.
One workaround for that is to set up the files for sequential access:
this minimizes os caching of files. This also more or less removes
'double dipping' into the memory system to cache your static file data.
The down side is that the work of maintaining an intelligent file cache
has been offloaded from the OS to you, the programmer. I am not
experienced enough with the postgres memory allocation system to say how
well this would work for PostgreSQL.
For shared memory used only for IPC, typically a VM system treats it
no
differently from any other non-shared memory, so if it's doing
something
you don't want or need (a proposition I quite heartily disagree
with),
it's going to be doing that very every piece of memory your
application
allocates and uses, shared or not.
The OS has to guarantee that the memory can be swapped out to file
at
any time and therefore mirrors the pagefile to the allocated memory
blocks.
The OS does not need to write the pagefile. On modern Unix systems
that
are not allowing overcommit, the space will be allocated but never
written unless there's a need to free up some physical memory, and the
pages in question are used infrequently enough that the system decides
that they are good candidates to be paged out. I would imagine that
Windows does the same.
In windows, things are backwards: the space is allocated in virtual
memory *first* (i.e. the page file), then following a page fault it gets
swapped into memory. The overhead I spoke of was related to the fact
the windows always has to ensure space exists in the page file (or some
user defined file) to swap the file back out. IMHO, *nix has a much
superior approach to IPC in this context. It's much simpler and very
straightforward.
It also exlains why in windows, most server apps are multi threaded, not
multi process. I agree with you on most salient points. The question
is: are MMFs the proper analog of SHHMEM on native port of postgres? My
answer to that question is: it is by no means certain, but what else is
there to use?
Merlin
cjs
--
Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974
http://www.netbsd.org
Don't you know, in this new Dark Age, we're all light. --XTC
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
