On Fri, 22 Oct 2010 12:06:37 +0100 Sad Clouds <cryintotheblue...@googlemail.com> wrote:
> Well if you're allocating memory yourself, then you've just created your > own application cache. Say many files were mapped in the process's address space, the OS would still be responsible of keeping frequently used ones pages active, possibly swapping out long-unused ones, unless of course MAP_WIRED was used. A syscall per access would be eliminated however, i.e. read(2), and I think that zero-copy could be used (with page loaning) if writing 64KB blocks out to a socket from a memory-mapped file. > On the other hand if you mmap() those files > directly, what happens if another process truncates some of those files > while you're reading them? I didn't do a test (it's definitely worth testing), but I think that a SIGSEGV could occur if a previously available page disappeared unless MAP_COPY, and file need to be remapped. I could see a problem where a siginfo-provided address might need to be easily matched with the file so that the process can efficiently know which file to remap... and for many files the current kqueue(2) EVFILT_VNODE isn't very useful either to detect that a file was recently modified, as it'd require too many open file descriptors :( There was some discussion made years ago about a kqueue(2) filter that could be set on a directory under which any modified file (possibly for the whole involved filesystem for the superuser) would generate an event with information about which file is modified by inode, but this seems non-trivial and wasn't yet implemented. There also are issues with inode to file string lookup (multiple files could point to a common destination, and a reverse name cache is needed). Anyway, I like this kind of discussion and have nothing against NIH personally (it fuels variety and competition, in fact), so thanks for sharing your custom cache experiments and performance numbers. If you happen to do achieve interesting performance along the above lines with mmap(2) as well, I'd also like to know how it went. Thanks, -- Matt
