Timo Sirainen put forth on 10/21/2010 9:52 AM: > On Wed, 2010-10-20 at 23:50 -0500, Stan Hoeppner wrote: >>> Oh, interesting. I didn't know that was possible. And even better: Linux >>> has fallocate() that can do it for other filesystems than just XFS. Or >>> looks like it's only XFS and ext4 (ext3 doesn't support it). I don't >>> know if other OSes support this. >> There exists posix_fallocate() which would widen the platforms that >> would support this Timo. > > I know about posix_fallocate(), but that does a different thing. It > would work only when this works:
Ahh, ok, didn't realize that it was different than the Linux specific call. >> Maybe in future I could make mdbox >>> support writing to files whose size has been preallocated by actually >>> writing NUL bytes, but that requires some extra code. > >> You may also want to look at posix_fadvise() >> as well (if you're not using it already) which might increase Dovecot's >> overall disk performance a bit. > > I once wrote a patch for that and asked if anyone could confirm if it > made things worse or better, but no one ever did. > http://dovecot.org/list/dovecot/2007-November/026819.html I wasn't aware of this. I don't really have the resources, skill set, or time to test patches, or I'd do so. :( >> NOTE: I don't believe fallocate() in either posix or linux only form >> will actually accomplish decreased m[d]box file fragmentation. I don't >> believe it actually increases the file size on disk, i.e. physically >> allocating additional free extents tailing the end of the file. >> fallocate() is _speculative_ preallocation, which isn't what you want. > > Why do you believe that? The man pages and everything tell me that it's > about physical preallocation: > > fallocate is used to preallocate blocks to a file. For filesystems > which support the fallocate system call, this is done quickly by allo‐ > cating blocks and marking them as uninitialized, requiring no IO to the > data blocks. This is much faster than creating a file by filling it > with zeros. > > Also "du" shows that the file's size is actually the preallocated size. > That's how I checked which filesystems supported it. > > And I think Linux fallocate() with XFS internally does exactly what > XFS_IOC_RESVSP does, which your quoted mail mentioned being about > physical preallocation. Well that's cool then. >>>> I don't know if it changes anything in the sequence above, but Dovecot >>>> uses mmap i/o. > > mmap is used only for index files, not for mdbox files. Ahh, that's really good to know. IIRC the comments for enabling mmap in dovecot.conf don't state this. I thus assumed enabling mmap was for all files. >> It would be beneficial I think if you'd sub to the xfs list Timo and >> pick some brains. > > Maybe, but there is still so much other stuff to do, like bug fixing. :) Looks like you've got a handle on it. Like I said, I know just enough about this stuff to make me dangerous. ;) -- Stan
