Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Friday 15 February 2008, Alan McKinnon wrote: On Friday 15 February 2008, Dale wrote: Alan McKinnon wrote: On Thursday 14 February 2008, Uwe Thiem wrote: That aside, how would gaps *between* files ever translate into fragmentation unless the author of that particular piece of software managed to kill his very last brain cell? Oops. I had a brain fart there. You two are so funny. Thank you. We try to please :-) I second that. Africa makes you so. I mean funny and trying to please. ;-) I found this too: http://www.oo-software.com/home/en/products/oodefrag/ Seems someone is trying to make money. I have also read that most Linux file systems do this automatically somehow. After doing my test, I tend to agree. So why have a commercial product for this? Is it just money? Yeah, pretty much just money. Microsoft's business model is to trap the market, never perform at any level higher than mediocrity, and create an ecosystem that needs thousands of support apps just to keep the OS limping along. Then shaft all of them with vendor-lockin Coping with file fragmentation has to be one of the easiest algorithms around, it isn't even hard. Write a file, and look to see how the blocks are distributed. If it can be improved, then do so. Otherwise leave it as is But then again, if you have written a file system so that everything is just mushed onto the same device, all higeldypigeldy with no sane structure at all ... then I suppose you would need stuff like defrag to come along once a week and save your ass :-) Back in the days when I still used DOS, one certainly wanted to defragment periodically. The system became significantly more performant for a while. On Linux/Unix, I never bothered. Uwe -- Informal Linux Group Namibia: http://www.linux.org.na/ SysEx (Pty) Ltd.: http://www.SysEx.com.na/ -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
Uwe Thiem wrote: Back in the days when I still used DOS, one certainly wanted to defragment periodically. The system became significantly more performant for a while. On Linux/Unix, I never bothered. Uwe Yea, I remember those days too. Put the disk and get it started then wait until it finishes the next day. Sometimes it would take more time than that. I mean, when you got a puter with 64K of ram and maybe 200 or 300MBs of hard drive space, well, it takes a while. Oh, that 4MHz blazingly fast CPU helped a lot too. ;-) Dale :-) :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Friday 15 February 2008, Dale wrote: Uwe Thiem wrote: Back in the days when I still used DOS, one certainly wanted to defragment periodically. The system became significantly more performant for a while. On Linux/Unix, I never bothered. Uwe Yea, I remember those days too. Put the disk and get it started then wait until it finishes the next day. Sometimes it would take more time than that. I mean, when you got a puter with 64K of ram and maybe 200 or 300MBs of hard drive space, well, it takes a while. Oh, that 4MHz blazingly fast CPU helped a lot too. ;-) I remember using PCTools for defrag then switching to Norton. Took 2 days on a 20M MFM drive - Norton used a different layout scheme to PCTools and wanted to change *everything* around. Heck, I remember changing the interleaving on that disk from 1 to 3 and getting a massive performance increase... -- Alan McKinnon alan dot mckinnon at gmail dot com -- gentoo-user@lists.gentoo.org mailing list
[gentoo-user] Linux, reiserfs and file fragmentation
Hi, I'm not wanting to start a flame or anything but I have a question, or two, on file fragmentation. I have three hard drives here. This is how they are partitioned at the moment: [EMAIL PROTECTED] / # df Filesystem 1K-blocksUsed Available Use% Mounted on /dev/hda619530340768940011840940 40% / /dev/hda11893397741 171822 5%/boot /dev/hda748825321805104 3077428 37%/usr/portage /dev/hda848825321402172 3480360 29%/home /dev/hdb17814576815171752 6297401620%/data [EMAIL PROTECTED] / # So you won't freak out, I removed the system generated file systems. I use reiserfs on everything except /boot. It has ext2. I found a program, script really, that will tell how fragmented a file system is. This is what it reports: [EMAIL PROTECTED] / # /root/fragck.pl /usr/portage/ 2.93016639526831% non contiguous files, 1.10476912422558 average fragments. [EMAIL PROTECTED] / # /root/fragck.pl /data/ 4.64526022181977% non contiguous files, 1.08609726757175 average fragments. [EMAIL PROTECTED] / # /root/fragck.pl /home/ statfs: No such file or directory Use of uninitialized value in pattern match (m//) at /root/fragck.pl line 32. Use of uninitialized value in concatenation (.) or string at /root/fragck.pl line 41. : not understand for /home/dale/.googleearth/Registry/google/gecommonsettings/User/layers/hiking\ and\ mountain\ bike\ trails\ . 6.4034151547492% non contiguous files, 2.34827463536108 average fragments. [EMAIL PROTECTED] / # The big one: [EMAIL PROTECTED] / # /root/fragck.pl / statfs: No such file or directory sh: en_1ca_4278190280: command not found sh: wm_6: command not found sh: wp_#usr#kde#3.5#share#wallpapers#here-gear.svgz_1196427186: command not found sh: blm_0: command not found sh: 01.png: command not found Use of uninitialized value in pattern match (m//) at /root/fragck.pl line 32. Use of uninitialized value in concatenation (.) or string at /root/fragck.pl line 41. : not understand for /var/tmp/kdecache-root/background/143x115_bm_0;en_1ca_4278190280;wm_6;wp_\#usr\#kde\#3.5\#share\#wallpapers\#here-gear.svgz_1196427186;blm_0;01.png . statfs: No such file or directory sh: en_1ca_4278190280: command not found sh: wm_6: command not found sh: wp_#usr#kde#3.5#share#wallpapers#here-gear.svgz_1192962680: command not found sh: blm_0: command not found sh: 01.png: command not found Use of uninitialized value in pattern match (m//) at /root/fragck.pl line 32. Use of uninitialized value in concatenation (.) or string at /root/fragck.pl line 41. : not understand for /var/tmp/kdecache-root/background/143x115_bm_0;en_1ca_4278190280;wm_6;wp_\#usr\#kde\#3.5\#share\#wallpapers\#here-gear.svgz_1192962680;blm_0;01.png . 7.03979903669298% non contiguous files, 1.22497799280025 average fragments. [EMAIL PROTECTED] / # Please pardon the error message. I use the script, I didn't write it. :/ I understand that doing the root directory is sort of a sum of all the others so it may be a little misleading to say the least. My questions; is this badly fragmented? How can I unfragment all the files and not bork something up badly? My opinion on this tho, considering this install is about 4 years old, not to bad. I've seen worse on a windoze rig shortly after a install. ;-) Thanks for any advice/info you can provide. Let me know if you need more info too. Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
Hello On Thu, Feb 14, 2008 at 05:06:43AM -0600, Dale wrote: My questions; is this badly fragmented? How can I unfragment all the files and not bork something up badly? My opinion on this tho, considering this install is about 4 years old, not to bad. I've seen worse on a windoze rig shortly after a install. ;-) I would guess the fragmented files are the big ones. And, with average of 2 fragments per file, it is not too much. If you have a movie with 30MB fragments, then it is no problem. Unless you hear lot of rattling noise from the HDD, you could leave it as is. And the surest way to defragment a filesystem is take everything out and put it back again. It will write the files one after another and will have no reason to split them. -- XML is like violence. If it doesn't solve your problem, use more. Michal 'vorner' Vaner -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
Michal 'vorner' Vaner wrote: Hello On Thu, Feb 14, 2008 at 05:06:43AM -0600, Dale wrote: My questions; is this badly fragmented? How can I unfragment all the files and not bork something up badly? My opinion on this tho, considering this install is about 4 years old, not to bad. I've seen worse on a windoze rig shortly after a install. ;-) I would guess the fragmented files are the big ones. And, with average of 2 fragments per file, it is not too much. If you have a movie with 30MB fragments, then it is no problem. Unless you hear lot of rattling noise from the HDD, you could leave it as is. And the surest way to defragment a filesystem is take everything out and put it back again. It will write the files one after another and will have no reason to split them. So if for example I copied everything over to a different hard drive and then copied everything back, it would be defragmented then? I would think of something like this: Boot some live CD. Mount old and backup drives. Copy old drive to a backup drive using cp -av yada yada. Make a new file system on the old drive to make sure all is clean. Copy everything back over from the backup to the old drive using cp -av yada yada. I would also take the opportunity to redo a few partitions while I was able to. The biggest slow down by the way is when logging into KDE the first time. It takes a long while and that drive is just a getting it. The light just stays on while loading everything up. Your thoughts and others if needed. Dale :-) :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, my 2 cents: | So if for example I copied everything over to a different hard drive and | then copied everything back, it would be defragmented then? I think so yes, but still I would not do it as I think you will hardly notice the difference, but there is a good chance to screw things up. | I would think of something like this: | | Boot some live CD. | Mount old and backup drives. | Copy old drive to a backup drive using cp -av yada yada. Its very important to do this as root and preserve all the file permissions and symbolic links exactly as they are on the drive. In particular the backup file system must support all this. (You cannot backup to a FAT file system, etc.) the cp option -b could help, but surely you should read man cp and man mount | Make a new file system on the old drive to make sure all is clean. | Copy everything back over from the backup to the old drive using cp -av | yada yada. | | I would also take the opportunity to redo a few partitions while I was | able to. If you do so don't forget to update /etc/fstab and the configuration of the bootloader ! | The biggest slow down by the way is when logging into KDE the first | time. It takes a long while and that drive is just a getting it. The | light just stays on while loading everything up. I personally think this is not due to fragmentation. On loading KDE just preloads some big libraries (it is a big program :) and this takes some time. Furthermore the libraries are loaded with LD_BIND_NOW=true, which makes the linker resolve all the symbols when KDE starts. (KDE takes longer to load, but later the programs are loaded faster). You can google for that to learn what it means. Hope it helps a little Thomas -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHtDqdrpEWPKIUt7MRAkV6AKCLrm/tVj7KjM4ElHCqc0Zf/gRoxwCdEI1F jtPhK0NUmeRBcP5giKMb2JI= =Dg/q -END PGP SIGNATURE- -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Donnerstag, 14. Februar 2008, Dale wrote: The biggest slow down by the way is when logging into KDE the first time. It takes a long while and that drive is just a getting it. The light just stays on while loading everything up. do you use prelink? -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
Thomas Kahle wrote: Hi, my 2 cents: | So if for example I copied everything over to a different hard drive and | then copied everything back, it would be defragmented then? I think so yes, but still I would not do it as I think you will hardly notice the difference, but there is a good chance to screw things up. | I would think of something like this: | | Boot some live CD. | Mount old and backup drives. | Copy old drive to a backup drive using cp -av yada yada. Its very important to do this as root and preserve all the file permissions and symbolic links exactly as they are on the drive. In particular the backup file system must support all this. (You cannot backup to a FAT file system, etc.) the cp option -b could help, but surely you should read man cp and man mount | Make a new file system on the old drive to make sure all is clean. | Copy everything back over from the backup to the old drive using cp -av | yada yada. | | I would also take the opportunity to redo a few partitions while I was | able to. If you do so don't forget to update /etc/fstab and the configuration of the bootloader ! | The biggest slow down by the way is when logging into KDE the first | time. It takes a long while and that drive is just a getting it. The | light just stays on while loading everything up. I personally think this is not due to fragmentation. On loading KDE just preloads some big libraries (it is a big program :) and this takes some time. Furthermore the libraries are loaded with LD_BIND_NOW=true, which makes the linker resolve all the symbols when KDE starts. (KDE takes longer to load, but later the programs are loaded faster). You can google for that to learn what it means. Hope it helps a little Thomas I did a little test. Something fishy here. I did a test with the /data partition. I store pictures and documents there and it was fragmented. I cp -av to another reiserfs formatted partition then remade the file system and copied it back using basically the same command just in reverse. This is what I got now: [EMAIL PROTECTED] / # /root/fragck.pl /data/ 3.88457269700333% non contiguous files, 1.04344379261138 average fragments. [EMAIL PROTECTED] / # That is not a lot better than it was before. It was 4.6% before. How is that? I copied it over then ran the command right after without even touching the files. Any ideas? Is there a limit to the fragmenting smallness? Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thu, 14 Feb 2008 06:01:16 -0600 Dale [EMAIL PROTECTED] wrote: Michal 'vorner' Vaner wrote: Hello On Thu, Feb 14, 2008 at 05:06:43AM -0600, Dale wrote: My questions; is this badly fragmented? How can I unfragment all the files and not bork something up badly? My opinion on this tho, considering this install is about 4 years old, not to bad. I've seen worse on a windoze rig shortly after a install. ;-) I would guess the fragmented files are the big ones. And, with average of 2 fragments per file, it is not too much. If you have a movie with 30MB fragments, then it is no problem. Unless you hear lot of rattling noise from the HDD, you could leave it as is. And the surest way to defragment a filesystem is take everything out and put it back again. It will write the files one after another and will have no reason to split them. So if for example I copied everything over to a different hard drive and then copied everything back, it would be defragmented then? I would think of something like this: Boot some live CD. Mount old and backup drives. Copy old drive to a backup drive using cp -av yada yada. Make a new file system on the old drive to make sure all is clean. Copy everything back over from the backup to the old drive using cp -av yada yada. I would also take the opportunity to redo a few partitions while I was able to. The biggest slow down by the way is when logging into KDE the first time. It takes a long while and that drive is just a getting it. The light just stays on while loading everything up. Your thoughts and others if needed. Dale :-) :-) :-) If you haven't already done this, you could try [1] for faster KDE boot. I believe it'll bring you much bigger application start-up boost than defragmenting your FS. Please, notice that I'm not saying that defragmentation is pointless. Just the opposite: I believe fragmentation leads to a perceivable (and actually measurable) performance hit. [1] http://www.gentoo.org/doc/en/prelink-howto.xml -- Best regards, Daniel -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
Volker Armin Hemmann wrote: On Donnerstag, 14. Februar 2008, Dale wrote: The biggest slow down by the way is when logging into KDE the first time. It takes a long while and that drive is just a getting it. The light just stays on while loading everything up. do you use prelink? I did use it at one time. It didn't seem to speed anything up. I can't recall when I stopped using it tho. It's been a while. Think it would help now? It is usually when I first login that it takes so long. After the first time, it only takes about 5 or 6 seconds. Not real bad but I was thinking about my poor five year old drive. o_O Thanks. Dale :-) :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thursday 14 February 2008, Thomas Kahle wrote: Hi, my 2 cents: | So if for example I copied everything over to a different hard | drive and then copied everything back, it would be defragmented | then? I think so yes, but still I would not do it as I think you will hardly notice the difference, but there is a good chance to screw things up. Yes, everything will be defragmented. In addition, it will leave gaps between files. So if a file lateron grows it will not immediately fragment. | I would think of something like this: | | Boot some live CD. | Mount old and backup drives. | Copy old drive to a backup drive using cp -av yada yada. Its very important to do this as root and preserve all the file permissions and symbolic links exactly as they are on the drive. In particular the backup file system must support all this. (You cannot backup to a FAT file system, etc.) the cp option -b could help, but surely you should read man cp and man mount The easiest way to preserve all permissions and symlinks is to use tar instead of cp. If you do so, read man tar of course. | The biggest slow down by the way is when logging into KDE the | first time. It takes a long while and that drive is just a | getting it. The light just stays on while loading everything up. I personally think this is not due to fragmentation. On loading KDE just preloads some big libraries (it is a big program :) and this takes some time. Furthermore the libraries are loaded with LD_BIND_NOW=true, which makes the linker resolve all the symbols when KDE starts. (KDE takes longer to load, but later the programs are loaded faster). You can google for that to learn what it means. There are two ways to speed up KDE load time. First, prelink everything (something like prelink -avmR). Second, you can configure kdm to preload as much of KDE as possible. So while you are still staring at your login screen or typing your user name and password, it loads as much as it can. BTW, KDE 4 starts significantly faster than 3.5. Uwe -- Informal Linux Group Namibia: http://www.linux.org.na/ SysEx (Pty) Ltd.: http://www.SysEx.com.na/ -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thu, Feb 14, 2008 at 07:53:57AM -0600, Penguin Lover Dale squawked: I did a little test. Something fishy here. I did a test with the /data partition. I store pictures and documents there and it was fragmented. I cp -av to another reiserfs formatted partition then remade the file system and copied it back using basically the same command just in reverse. This is what I got now: [EMAIL PROTECTED] / # /root/fragck.pl /data/ 3.88457269700333% non contiguous files, 1.04344379261138 average fragments. [EMAIL PROTECTED] / # That is not a lot better than it was before. It was 4.6% before. How is that? I copied it over then ran the command right after without even touching the files. Is the /data partition on reiser? Did you enable tailpacking? As I don't know what the script you ran actually does, I don't know how it handles block suballocation... Tail packing is something that can conceivably introduce data fragmentation in place of internal fragmentation. W -- Pintsize: Curses! HOURS in there and I STILL don't have mutant ice powers! Pintsize: Sorry waffles, you can't be my sidekick until I have some superhero powers to fight crime with... What? Waffle powers? Somehow I don't see soaking up syrup or browning in a toaster getting us a lot of hot supervillain ladies. Sortir en Pantoufles: up 433 days, 12:47 -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, just one more idea that came to my mind, reiserfs uses a technique to save small files in the filesystem tree which uses less disk space then. In ext3 a 1 byte file will take up 4k, while this is not the case in reiserfs. This yields a performace hit of about 5% (people say, not that i have measured anything) If you have enough free space you can disable this, make the small files consume more space again and gains some speed improvement. Another thing you could do is disable the writing of accesstimes. Read man mount how to do this. The mount options are noatime,notail. Concerning your observation I would start looking at how the fragmentation is measured. Maybe this also depends on the filesystem implementation in the kernel. Anyway: you can not get much better 1.043.. parts per file. This means that almost every file(96,12 % in your case) is contiguous. have fun | | | I did a little test. Something fishy here. I did a test with the /data | partition. I store pictures and documents there and it was fragmented. | I cp -av to another reiserfs formatted partition then remade the file | system and copied it back using basically the same command just in | reverse. This is what I got now: | | [EMAIL PROTECTED] / # /root/fragck.pl /data/ | 3.88457269700333% non contiguous files, 1.04344379261138 average fragments. | [EMAIL PROTECTED] / # | | That is not a lot better than it was before. It was 4.6% before. How | is that? I copied it over then ran the command right after without even | touching the files. | Any ideas? Is there a limit to the fragmenting smallness? | | Dale | | :-) :-) -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHtE6PrpEWPKIUt7MRAoAhAJ4wJ1Ygs7A75ayFCIAs+uXjW+uUbwCfdeaB rDCDg4kPoAfrKbMUZdJ/EdU= =WkET -END PGP SIGNATURE- -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thursday 14 February 2008, Dale wrote: I did a little test. Something fishy here. I did a test with the /data partition. I store pictures and documents there and it was fragmented. I cp -av to another reiserfs formatted partition then remade the file system and copied it back using basically the same command just in reverse. This is what I got now: [EMAIL PROTECTED] / # /root/fragck.pl /data/ 3.88457269700333% non contiguous files, 1.04344379261138 average fragments. [EMAIL PROTECTED] / # That is not a lot better than it was before. It was 4.6% before. How is that? I copied it over then ran the command right after without even touching the files. Before you copy back, you have to clean the old partition - either by deleting everything or by partioning it. Uwe -- Informal Linux Group Namibia: http://www.linux.org.na/ SysEx (Pty) Ltd.: http://www.SysEx.com.na/ -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
Uwe Thiem wrote: On Thursday 14 February 2008, Dale wrote: I did a little test. Something fishy here. I did a test with the /data partition. I store pictures and documents there and it was fragmented. I cp -av to another reiserfs formatted partition then remade the file system and copied it back using basically the same command just in reverse. This is what I got now: [EMAIL PROTECTED] / # /root/fragck.pl /data/ 3.88457269700333% non contiguous files, 1.04344379261138 average fragments. [EMAIL PROTECTED] / # That is not a lot better than it was before. It was 4.6% before. How is that? I copied it over then ran the command right after without even touching the files. Before you copy back, you have to clean the old partition - either by deleting everything or by partioning it. Uwe I just did a mkreiserfs /dev/hdb1. That should work right? Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
Daniel Iliev wrote: On Thu, 14 Feb 2008 06:01:16 -0600 Dale [EMAIL PROTECTED] wrote: Michal 'vorner' Vaner wrote: Hello On Thu, Feb 14, 2008 at 05:06:43AM -0600, Dale wrote: My questions; is this badly fragmented? How can I unfragment all the files and not bork something up badly? My opinion on this tho, considering this install is about 4 years old, not to bad. I've seen worse on a windoze rig shortly after a install. ;-) I would guess the fragmented files are the big ones. And, with average of 2 fragments per file, it is not too much. If you have a movie with 30MB fragments, then it is no problem. Unless you hear lot of rattling noise from the HDD, you could leave it as is. And the surest way to defragment a filesystem is take everything out and put it back again. It will write the files one after another and will have no reason to split them. So if for example I copied everything over to a different hard drive and then copied everything back, it would be defragmented then? I would think of something like this: Boot some live CD. Mount old and backup drives. Copy old drive to a backup drive using cp -av yada yada. Make a new file system on the old drive to make sure all is clean. Copy everything back over from the backup to the old drive using cp -av yada yada. I would also take the opportunity to redo a few partitions while I was able to. The biggest slow down by the way is when logging into KDE the first time. It takes a long while and that drive is just a getting it. The light just stays on while loading everything up. Your thoughts and others if needed. Dale :-) :-) :-) If you haven't already done this, you could try [1] for faster KDE boot. I believe it'll bring you much bigger application start-up boost than defragmenting your FS. Please, notice that I'm not saying that defragmentation is pointless. Just the opposite: I believe fragmentation leads to a perceivable (and actually measurable) performance hit. [1] http://www.gentoo.org/doc/en/prelink-howto.xml Now I remember why I stopped using prelink: The only maintenance required is re-running prelink every time a library is upgraded for a pre-linked executable. I knew there was a reason I stopped. I never could remember to run it after I finished emerging stuff. It was a thought tho. Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thursday 14 February 2008, Dale wrote: Uwe Thiem wrote: On Thursday 14 February 2008, Dale wrote: I did a little test. Something fishy here. I did a test with the /data partition. I store pictures and documents there and it was fragmented. I cp -av to another reiserfs formatted partition then remade the file system and copied it back using basically the same command just in reverse. This is what I got now: [EMAIL PROTECTED] / # /root/fragck.pl /data/ 3.88457269700333% non contiguous files, 1.04344379261138 average fragments. [EMAIL PROTECTED] / # That is not a lot better than it was before. It was 4.6% before. How is that? I copied it over then ran the command right after without even touching the files. Before you copy back, you have to clean the old partition - either by deleting everything or by partioning it. Uwe I just did a mkreiserfs /dev/hdb1. That should work right? Actually, I meant by formatting it instead of partioning. So yes, that should work. Maybe fragch.pl is simply buggy. Uwe -- Informal Linux Group Namibia: http://www.linux.org.na/ SysEx (Pty) Ltd.: http://www.SysEx.com.na/ -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thursday 14 February 2008, Dale wrote: Now I remember why I stopped using prelink: The only maintenance required is re-running prelink every time a library is upgraded for a pre-linked executable. I only prelink after major updates. Never had any problems in between. I knew there was a reason I stopped. I never could remember to run it after I finished emerging stuff. It was a thought tho. Leave the a option ou, and it prelinks only stuff that needs prelinking. You can force automatic prelinking in /etc/conf.d/prelink. Uwe Informal Linux Group Namibia: http://www.linux.org.na/ SysEx (Pty) Ltd.: http://www.SysEx.com.na/ -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thursday 14 February 2008, Dale wrote: I did a little test. Something fishy here. I did a test with the /data partition. I store pictures and documents there and it was fragmented. I cp -av to another reiserfs formatted partition then remade the file system and copied it back using basically the same command just in reverse. This is what I got now: [EMAIL PROTECTED] / # /root/fragck.pl /data/ 3.88457269700333% non contiguous files, 1.04344379261138 average fragments. [EMAIL PROTECTED] / # That is not a lot better than it was before. It was 4.6% before. How is that? I copied it over then ran the command right after without even touching the files. Any ideas? Is there a limit to the fragmenting smallness? Don't worry about fragmentation on reiserfs. This is not a valid concept for reiser or for ext2/3. Fragmentation is problematic on Windows machines because that code is brain dead. Some people seem to assume that it must therefore be problematic on all file systems. Reiserfs is not brain dead, it is intelligent and will balance itself out over time. It also has tail packing which can make fragmentation stats look odd if enabled. Short answer: Don't worry about it. Let reiserfs do what it wants to do when it wants to do it - it is much much much better at these decisions than you will ever be ;-) -- Alan McKinnon alan dot mckinnon at gmail dot com -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Donnerstag, 14. Februar 2008, Dale wrote: Now I remember why I stopped using prelink: The only maintenance required is re-running prelink every time a library is upgraded for a pre-linked executable. I knew there was a reason I stopped. I never could remember to run it after I finished emerging stuff. prelink runs by cron - not after every emerge. That is reallye enough. -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thursday 14 February 2008, Uwe Thiem wrote: Yes, everything will be defragmented. In addition, it will leave gaps between files. So if a file lateron grows it will not immediately fragment. Which will cause a stupid script to report fragmentation if the author does not understand file system structure... -- Alan McKinnon alan dot mckinnon at gmail dot com -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
Volker Armin Hemmann wrote: On Donnerstag, 14. Februar 2008, Dale wrote: Now I remember why I stopped using prelink: The only maintenance required is re-running prelink every time a library is upgraded for a pre-linked executable. I knew there was a reason I stopped. I never could remember to run it after I finished emerging stuff. prelink runs by cron - not after every emerge. That is reallye enough. May have to check this out again then, now that a cron can remember to prelink again. LOL Dang I'm getting old. :-( Dale :-) :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
Alan McKinnon wrote:
On Thursday 14 February 2008, Uwe Thiem wrote:
Yes, everything will be defragmented. In addition, it will leave gaps
between files. So if a file lateron grows it will not immediately
fragment.
Which will cause a stupid script to report fragmentation if the author
does not understand file system structure...
Yea, I have always read that Linux file systems are a lot better at
taking care of fragmentation. Considering how old this install is and
how much has been installed/removed/upgraded over the past several
years, I think it is not to bad really. Even 10% with the number of
files I have is better than fat or NTFS. My bro has XP with NTFS and it
gets downright awful.
For the record, I have over 502,000 files and over 49,000 directories on
this system. That's less than 20,000 files that are fragmented. It's
not just the OS but documents, little movies and a LOT of pictures.
Maybe I just need a bigger hard drive. O_O I have two 80GB drives and
a single 40GB drive. Waiting on DSL. he he he he he
I also attached a copy of the program I used. I think I got it off the
forums. Maybe some guru can improve it a little. ;-)
Dale
:-) :-)
#!/usr/bin/perl -w
#this script search for frag on a fs
use strict;
#number of files
my $files = 0;
#number of fragment
my $fragments = 0;
#number of fragmented files
my $fragfiles = 0;
#search fs for all file
open (FILES, find . $ARGV[0] . -xdev -type f |);
while (defined (my $file = FILES)) {
#quote some chars in filename
$file =~ s/!/\\!/g;
$file =~ s/#/\\#/g;
$file =~ s//\\/g;
$file =~ s//\\/g;
$file =~ s//\\/g;
$file =~ s/\$/\\\$/g;
$file =~ s/\(/\\\(/g;
$file =~ s/\)/\\\)/g;
$file =~ s/\|/\\\|/g;
$file =~ s/'/\\'/g;
$file =~ s/ /\\ /g;
#nb of fragment for the file
open (FRAG, filefrag $file |);
my $res = FRAG;
if ($res =~ m/.*:\s+(\d+) extents? found/) {
my $fragment = $1;
$fragments+=$fragment;
if ($fragment 1) {
$fragfiles++;
}
$files++;
} else {
print ($res : not understand for $file.\n);
}
close (FRAG);
}
close (FILES);
print ( $fragfiles / $files * 100 . % non contiguous files, . $fragments / $files . average fragments.\n);
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thursday 14 February 2008, Alan McKinnon wrote: On Thursday 14 February 2008, Uwe Thiem wrote: Yes, everything will be defragmented. In addition, it will leave gaps between files. So if a file lateron grows it will not immediately fragment. Which will cause a stupid script to report fragmentation if the author does not understand file system structure... One can assume any level stupidness of writer of little perl scripts. ;-) That aside, how would gaps *between* files ever translate into fragmentation unless the author of that particular piece of software managed to kill his very last brain cell? Uwe -- Informal Linux Group Namibia: http://www.linux.org.na/ SysEx (Pty) Ltd.: http://www.SysEx.com.na/ -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thursday 14 February 2008, Dale wrote: I also attached a copy of the program I used. I think I got it off the forums. Maybe some guru can improve it a little. ;-) Not me. Perl has been invented to generate reports from log files or such. It is not a general purpose language, though many sysadmins abuse it for such. I can't read my own perl scripts I have written three weeks ago. It all looks like spider legs or chicken cratches to me. Uwe -- Informal Linux Group Namibia: http://www.linux.org.na/ SysEx (Pty) Ltd.: http://www.SysEx.com.na/ -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thursday 14 February 2008, Uwe Thiem wrote: On Thursday 14 February 2008, Dale wrote: I also attached a copy of the program I used. I think I got it off the forums. Maybe some guru can improve it a little. ;-) Not me. Perl has been invented to generate reports from log files or such. It is not a general purpose language, though many sysadmins abuse it for such. I can't read my own perl scripts I have written three weeks ago. It all looks like spider legs or chicken cratches to me. Three weeks! You remember what you coded for up to three weeks but not usually longer Wow. You are one lucky SOB. I barely manage three DAYS lately grin -- Alan McKinnon alan dot mckinnon at gmail dot com -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Thursday 14 February 2008, Uwe Thiem wrote: That aside, how would gaps *between* files ever translate into fragmentation unless the author of that particular piece of software managed to kill his very last brain cell? Oops. I had a brain fart there. -- Alan McKinnon alan dot mckinnon at gmail dot com -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
Alan McKinnon wrote: On Thursday 14 February 2008, Uwe Thiem wrote: That aside, how would gaps *between* files ever translate into fragmentation unless the author of that particular piece of software managed to kill his very last brain cell? Oops. I had a brain fart there. You two are so funny. I found this too: http://www.oo-software.com/home/en/products/oodefrag/ Seems someone is trying to make money. I have also read that most Linux file systems do this automatically somehow. After doing my test, I tend to agree. So why have a commercial product for this? Is it just money? Your thoughts, humor is OK too. o_O Dale :-) :-) -- gentoo-user@lists.gentoo.org mailing list
Re: [gentoo-user] Linux, reiserfs and file fragmentation
On Friday 15 February 2008, Dale wrote: Alan McKinnon wrote: On Thursday 14 February 2008, Uwe Thiem wrote: That aside, how would gaps *between* files ever translate into fragmentation unless the author of that particular piece of software managed to kill his very last brain cell? Oops. I had a brain fart there. You two are so funny. Thank you. We try to please :-) I found this too: http://www.oo-software.com/home/en/products/oodefrag/ Seems someone is trying to make money. I have also read that most Linux file systems do this automatically somehow. After doing my test, I tend to agree. So why have a commercial product for this? Is it just money? Yeah, pretty much just money. Microsoft's business model is to trap the market, never perform at any level higher than mediocrity, and create an ecosystem that needs thousands of support apps just to keep the OS limping along. Then shaft all of them with vendor-lockin Coping with file fragmentation has to be one of the easiest algorithms around, it isn't even hard. Write a file, and look to see how the blocks are distributed. If it can be improved, then do so. Otherwise leave it as is But then again, if you have written a file system so that everything is just mushed onto the same device, all higeldypigeldy with no sane structure at all ... then I suppose you would need stuff like defrag to come along once a week and save your ass :-) -- Alan McKinnon alan dot mckinnon at gmail dot com -- gentoo-user@lists.gentoo.org mailing list
