On Sep 13, 2010, at 9:41 PM, Haudy Kazemi wrote: > Richard Elling wrote: >> On Sep 13, 2010, at 5:14 AM, Edward Ned Harvey wrote: >>>> From: Richard Elling [mailto:rich...@nexenta.com >>>> ] >>>> >>>> This operational definition of "fragmentation" comes from the single- >>>> user, >>>> single-tasking world (PeeCees). In that world, only one thread writes >>>> files >>>> from one application at one time. In those cases, there is a reasonable >>>> expectation that a single file's blocks might be contiguous on a single >>>> disk. >>>> That isn't the world we live in, where have RAID, multi-user, or multi- >>>> threaded >>>> environments. >>>> >>>> >>> I don't know what you're saying, but I'm quite sure I disagree with it. >>> >>> Regardless of multithreading, multiprocessing, it's absolutely possible to >>> have contiguous files, and/or file fragmentation. That's not a >>> characteristic which depends on the threading model. >>> >>> >> >> Possible, yes. Probable, no. Consider that a file system is allocating >> space for multiple, concurrent file writers. > > With appropriate write caching and grouping or re-ordering of writes > algorithms, it should be possible to minimize the amount of file interleaving > and fragmentation on write that takes place.
To some degree, ZFS already does this. The dynamic block sizing tries to ensure that a file is written into the largest block[1] > (Or at least optimize the amount of file interleaving. Years ago MFM hard > drives had configurable sector interleave factors to better optimize > performance when no interleaving meant the drive had spun the platter far > enough to be ready to give the next sector to the CPU before the CPU was > ready with the result that the platter had to be spun a second time around to > wait for the CPU to catch up.) Reason #526 why SSDs kill HDDs on performance. >>> Also regardless of raid, it's possible to have contiguous or fragmented >>> files. The same concept applies to multiple disks. >>> >>> >> >> RAID works against the efforts to gain performance by contiguous access >> because the access becomes non-contiguous. >> > > From what I've seen, defragmentation offers its greatest benefit when the > tiniest reads are eliminated by grouping them into larger contiguous reads. > Once the contiguous areas reach a certain size (somewhere in the few Mbytes > to a few hundred Mbytes range), further defragmentation offers little > additional benefit. For the wikipedia definition of defragmentation, this can only occur when the files themselves are hundreds of megabytes in size. This is not the general case for which I see defragmentation used. Also, ZFS has an intelligent prefetch algorithm that can hide some performance aspects of "defragmentation" on HDDs. > Full defragmentation is a useful goal when the option of using file carving > based data recovery is desirable. Also remember that defragmentation is not > limited to space used by files. It can also apply to free, unused space, > which should also be defragmented to prevent future writes from being > fragmented on write. This is why ZFS uses a first fit algorithm until space becomes too low, when it changes to a best fit algorithm. As long as available space is big enough for the block, then it will be used. > With regard to multiuser systems and how that negates the need to defragment, > I think that is only partially true. As long as the files are defragmented > enough so that each particular read request only requires one seek before it > is time to service the next read request, further defragmentation may offer > only marginal benefit. On the other hand, if files from been fragmented down > to each sector being stored separately on the drive, then each read request > is going to take that much longer to be completed (or will be interrupted by > another read request because it has taken too long).. Yes, so try to avoid running your ZFS pool at more than 96% full. -- richard -- OpenStorage Summit, October 25-27, Palo Alto, CA http://nexenta-summit2010.eventbrite.com Richard Elling rich...@nexenta.com +1-760-896-4422 Enterprise class storage for everyone www.nexenta.com _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
