Re: defragmentation in FreeBSD 4.11
On Fri, 29 Jul 2005 21:13:49 +0800, MikeM <[EMAIL PROTECTED]> wrote: On 7/28/2005 at 11:57 AM Bob Johnson wrote: |Microsoft used to claim that NTFS doesn't need defragmentation. |Compared to MSDOSFS, that's a reasonably accurate statement, but |if you push it hard enough, it will still become fragmented. = The process of installing Windows on a clean disk leaves the disk in need of defragmenting. As he said, "if you push it hard enough, it will still become fragmented" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
On 7/28/2005 at 11:57 AM Bob Johnson wrote: |Microsoft used to claim that NTFS doesn't need defragmentation. |Compared to MSDOSFS, that's a reasonably accurate statement, but |if you push it hard enough, it will still become fragmented. = The process of installing Windows on a clean disk leaves the disk in need of defragmenting. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
On Thursday 28 July 2005 03:07 pm, Lowell Gilbert wrote: > Bob Johnson <[EMAIL PROTECTED]> writes: > > From: Victor Semionov <[EMAIL PROTECTED]> [...] > > > Why is it unnecessary to defragment UFS? > > > > In normal use, files never become fragmented enough to affect > > performance. In a (loose) sense, files are intentionally fragmented in a > > controlled way so that fragmentation doesn't cause problems. If you run > > fsck on a partition, you will typically see fragmentation levels of less > > than one percent. > > Careful, there; "fragmentation" on a UFS is measuring a completely > different thing than the same term applied to a Microsoft filesystem. > For UFS, it refers to non-contiguous free blocks (fragments, > actually), as opposed to the Microsoft terminology, where it refers to > non-contiguous blocks within the same file. > > Everything you are saying is correct, but it will confuse people who > don't realize the difference. Yeah, I was trying to keep a long response from getting even longer. And I didn't really know what fsck is measuring when it reports fragmentation, so I got lazy and glossed over it instead of digging up the information. Thanks for keeping me honest, - Bob ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
Freminlins <[EMAIL PROTECTED]> writes: > Gayn Winters wrote: > > > What I get from reading this article is that if the use of the file > > system is to store lots of small files, then use a small block size. Am > > I missing something? > > No and yes! There is a minimum block and fragment size. In this case > there were not enough contiguous fragments to enable an 8K file to be > created. Without checking I believe Solaris uses 8K blocks. That's correct, and also note that Solaris does not offer the flexibility in block sizes that is available from FreeBSD. And it is correct that a small block size will vastly reduce the ability to hit this fully-fragmented condition. At a cost of higher overhead, of course. > > Also, in most situations, buying a big enough disk is far better than > > worrying about what happens when a not-big-enough disk starts to get > > full. > > Indeed. But... in the case I linked to there was apparently plenty of > free space, just not enough free contiguous space. The author also > implies that a bigger disk would not solve the problem: > > "it creates and deletes tons of small files and thus the fragmentation > over a period of time." Where "plenty of free space" is a large absolute number, but still a small fraction of the disk. Full fragmentation of the empty space becomes exponentially less likely as the amount of free space increases. Note that when deciding where to allocate a fragment, there are two possible policies the filesystem can follow: either a space-efficient method that is more likely to require moving data when a file expands, or a time-efficient method that will create new fragments more often. FreeBSD can automatically shift between the two as the filesystem moves over or under a threshold "percent full" value. Solaris, I believe, needs to be explicitly changed between the two. > As mentioned though I have never seen this myself despite running very > busy mail and web servers with what must be billions of files being > created/deleted in that time. > > It certainly grabbed my attention so I thought it may be of interest to > others. It's an academic curiosity, but not an issue to take into account in planning your data center. Planning your filesystem layout, however, may well be. Be well. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
Bob Johnson <[EMAIL PROTECTED]> writes: > Message: 6 > Date: Thu, 28 Jul 2005 11:20:31 +0300 > From: Victor Semionov <[EMAIL PROTECTED]> > Subject: Re: defragmentation in FreeBSD 4.11 > To: freebsd-questions@freebsd.org > Message-ID: <[EMAIL PROTECTED]> > Content-Type: text/plain; charset="windows-1251" > > > > > This is one of the things I find really hard to get Windows users to > > > understand. They just won't believe that a company like Microsoft would > > > still be using a filesystem that needs defragmenting if it were possible > > > to design one that didn't. I often wonder why myself - after all, they > > > must have put a fair amount of work into NTFS, which at least doesn't > > > seem to get corrupted in a power failure. Did they make a trade-off I > > > don't understand, or is it just incompetence - or worse, a deal with > > > disk manufacturers to sell more disk? > > > > Microsoft used to claim that NTFS doesn't need defragmentation. Compared to > MSDOSFS, that's a reasonably accurate statement, but if you push it hard > enough, it will still become fragmented. > > > Why is it unnecessary to defragment UFS? > > > > In normal use, files never become fragmented enough to affect performance. > In > a (loose) sense, files are intentionally fragmented in a controlled way so > that fragmentation doesn't cause problems. If you run fsck on a partition, > you will typically see fragmentation levels of less than one percent. Careful, there; "fragmentation" on a UFS is measuring a completely different thing than the same term applied to a Microsoft filesystem. For UFS, it refers to non-contiguous free blocks (fragments, actually), as opposed to the Microsoft terminology, where it refers to non-contiguous blocks within the same file. Everything you are saying is correct, but it will confuse people who don't realize the difference. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
Gayn Winters wrote: > What I get from reading this article is that if the use of the file > system is to store lots of small files, then use a small block size. Am > I missing something? No and yes! There is a minimum block and fragment size. In this case there were not enough contiguous fragments to enable an 8K file to be created. Without checking I believe Solaris uses 8K blocks. > Also, in most situations, buying a big enough disk is far better than > worrying about what happens when a not-big-enough disk starts to get > full. Indeed. But... in the case I linked to there was apparently plenty of free space, just not enough free contiguous space. The author also implies that a bigger disk would not solve the problem: "it creates and deletes tons of small files and thus the fragmentation over a period of time." As mentioned though I have never seen this myself despite running very busy mail and web servers with what must be billions of files being created/deleted in that time. It certainly grabbed my attention so I thought it may be of interest to others. > -gayn Frem. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
On 7/28/05, Bob Johnson <[EMAIL PROTECTED]> wrote: > > Why is it unnecessary to defragment UFS? > > > > In normal use, files never become fragmented enough to affect performance. In > a (loose) sense, files are intentionally fragmented in a controlled way so > that fragmentation doesn't cause problems. If you run fsck on a partition, > you will typically see fragmentation levels of less than one percent. > > Also, keep in mind that in the default formatting, a FreeBSD partition has 8% > of the disk space withheld from normal users to help keep the disk from > becoming so full the system can't operate, and it has the side effect of > helping to prevent fragmentation as well. It is why df can show a disk being > as much as 108% full. It is possible to make this space available for normal > use if, for example, you are using a partition only for data storage and you > want to squeeze every last bit of space out of it, but of course there will > be some performance penalty as it starts to get full. You can also adjust > other disk parameters to optimize for your particular needs. See tunefs(8). > > If the disk gets close enough to full, the OS has no choice but to start > fragmenting things. Try to keep your disks less than about 90% full (that's > a number I remember from somewhere -- it's just a guideline and not a firm > limit). My /home partition is 95% full according to df (which means it is > actually a little under 90% full including the reserved capacity), and fsck > shows 0.1% fragmentation. Of course, it's a fairly big partition, so it > still has over a gigabyte of free space. Even the ISO CD images I downloaded > a few days ago probably didn't get much fragmentation. Apparently UFS can get fragmented even when there is lots of apparently free space: http://blogs.sun.com/roller/page/sprakki?entry=ufs_file_system_defragmentation Obviously this is Sun UFS, but there is a common heritage. I've never needed to do any sort of defrag on either FreeBSD or Solaris though. > - Bob Frem. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
Message: 6 Date: Thu, 28 Jul 2005 11:20:31 +0300 From: Victor Semionov <[EMAIL PROTECTED]> Subject: Re: defragmentation in FreeBSD 4.11 To: freebsd-questions@freebsd.org Message-ID: <[EMAIL PROTECTED]> Content-Type: text/plain; charset="windows-1251" > > This is one of the things I find really hard to get Windows users to > > understand. They just won't believe that a company like Microsoft would > > still be using a filesystem that needs defragmenting if it were possible > > to design one that didn't. I often wonder why myself - after all, they > > must have put a fair amount of work into NTFS, which at least doesn't > > seem to get corrupted in a power failure. Did they make a trade-off I > > don't understand, or is it just incompetence - or worse, a deal with > > disk manufacturers to sell more disk? > Microsoft used to claim that NTFS doesn't need defragmentation. Compared to MSDOSFS, that's a reasonably accurate statement, but if you push it hard enough, it will still become fragmented. > Why is it unnecessary to defragment UFS? > In normal use, files never become fragmented enough to affect performance. In a (loose) sense, files are intentionally fragmented in a controlled way so that fragmentation doesn't cause problems. If you run fsck on a partition, you will typically see fragmentation levels of less than one percent. Also, keep in mind that in the default formatting, a FreeBSD partition has 8% of the disk space withheld from normal users to help keep the disk from becoming so full the system can't operate, and it has the side effect of helping to prevent fragmentation as well. It is why df can show a disk being as much as 108% full. It is possible to make this space available for normal use if, for example, you are using a partition only for data storage and you want to squeeze every last bit of space out of it, but of course there will be some performance penalty as it starts to get full. You can also adjust other disk parameters to optimize for your particular needs. See tunefs(8). If the disk gets close enough to full, the OS has no choice but to start fragmenting things. Try to keep your disks less than about 90% full (that's a number I remember from somewhere -- it's just a guideline and not a firm limit). My /home partition is 95% full according to df (which means it is actually a little under 90% full including the reserved capacity), and fsck shows 0.1% fragmentation. Of course, it's a fairly big partition, so it still has over a gigabyte of free space. Even the ISO CD images I downloaded a few days ago probably didn't get much fragmentation. - Bob ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
RE: defragmentation in FreeBSD 4.11
On 7/27/2005 at 7:30 PM Mike Jeays wrote: |This is one of the things I find really hard to get Windows users to |understand. They just won't believe that a company like Microsoft would |still be using a filesystem that needs defragmenting if it were possible |to design one that didn't. I often wonder why myself - after all, they |must have put a fair amount of work into NTFS, which at least doesn't |seem to get corrupted in a power failure. Did they make a trade-off I |don't understand, or is it just incompetence - or worse, a deal with |disk manufacturers to sell more disk? = When NTFS was new, Microsoft was saying that it did not need to be defragmented, ever. I attended two technical sessions at conferences where Microsoft actually stated that. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
> This is one of the things I find really hard to get Windows users to > understand. They just won't believe that a company like Microsoft would > still be using a filesystem that needs defragmenting if it were possible > to design one that didn't. I often wonder why myself - after all, they > must have put a fair amount of work into NTFS, which at least doesn't > seem to get corrupted in a power failure. Did they make a trade-off I > don't understand, or is it just incompetence - or worse, a deal with > disk manufacturers to sell more disk? Why is it unnecessary to defragment UFS? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
RE: defragmentation in FreeBSD 4.11
On Wed, 2005-07-27 at 07:54, Norbert Koch wrote: > > How one could defragment a partition on FreeBSD ? Which tools are > > available > > for this ? > > None as I know. Usually there is no reason to defragment a ufs partition. > > Norbert > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "[EMAIL PROTECTED]" This is one of the things I find really hard to get Windows users to understand. They just won't believe that a company like Microsoft would still be using a filesystem that needs defragmenting if it were possible to design one that didn't. I often wonder why myself - after all, they must have put a fair amount of work into NTFS, which at least doesn't seem to get corrupted in a power failure. Did they make a trade-off I don't understand, or is it just incompetence - or worse, a deal with disk manufacturers to sell more disk? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
On 7/27/05, demigor <[EMAIL PROTECTED]> wrote: > > > > > How one could defragment a partition on FreeBSD ? Which tools are > > available > > > for this ? > > > > Why should you want to do this? There's no harmful fragmentation in > > the UFS of FreeBSD unless you exceed the max capacity of 100% which is 92% > > when looking closer but reported as 100% by df. > > > > Thanks for the information. Asked just of curiosity :) Anyway, you can just back up files on that partition, recreate the partition, restore the files (sequentially) - voila, they'll be defragmented. -- Dmitry Mityugov, St. Petersburg, Russia I ignore all messages with confidentiality statements "We live less by imagination than despite it" - Rockwell Kent, "N by E" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
> > > How one could defragment a partition on FreeBSD ? Which tools are > available > > for this ? > > Why should you want to do this? There's no harmful fragmentation in > the UFS of FreeBSD unless you exceed the max capacity of 100% which is 92% > when looking closer but reported as 100% by df. > Thanks for the information. Asked just of curiosity :) br, ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
RE: defragmentation in FreeBSD 4.11
> How one could defragment a partition on FreeBSD ? Which tools are > available > for this ? None as I know. Usually there is no reason to defragment a ufs partition. Norbert ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: defragmentation in FreeBSD 4.11
On Wed, 27 Jul 2005, demigor wrote: > How one could defragment a partition on FreeBSD ? Which tools are available > for this ? Why should you want to do this? There's no harmful fragmentation in the UFS of FreeBSD unless you exceed the max capacity of 100% which is 92% when looking closer but reported as 100% by df. Regards Konrad Heuer GWDG, Am Fassberg, 37077 Goettingen, Germany, [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
