On Mon, 17 Jul 2006, Antti Harri wrote:
> On Sun, 16 Jul 2006, Nick Holland wrote:
>
> > nope, you can still likely use multiple partitions. Break your backup job
> > into smaller chunks, put each chunk on its own partition. Or put each
> > machine on its own partition. Or ...
>
> Interesting ideas. I didn't think that having the same amount of files
> in many partitions will reduce the total time to fsck, does it really work
> that way although it goes through the same amount of files?
>
> > BTW: Yes, the dmesg could very well have helped. If your disks were not
> > being handled properly or you had insufficient RAM, you can have HORRIBLE
> > problems with fsck performance, adding to your fsck time by a non-trivial
> > multiple. Your times sound excessive to me, but then, I don't think I have
> > that many files on a single partition.
>
> Unfortunately the computer isn't at hand right now. I'll check the amount of
> RAM and add some more if there isn't much. Would changing BUFCACHEPCT
> help too? Because the computer is dedicated backup server so it can
> take up all the memory as far as I'm concerned.
>
> > One idea which has been suggested is to use softupdates, and simply "force"
> > mounting of the volume at boot, and periodically, fsck the thing on your
> > schedule, to reclaim lost disk space. Yes, when you do run the fsck, you
> > will spend a lot of time waiting for it, but you will be able to schedule
> > it.
>
> Hmm, actually I am using softupdates. Doesn't it *ever* get corrupted with
> softupdates even though there is a crash?
>
> > Keep in mind, partitions need not all be mounted in /etc/fstab, they can be
> > manually mounted "later" in rc.local. Why does your backup machine have to
> > boot "fast"? (I got one with way too little RAM, it needs to use swap to
> > fsck, but that's ok...I'm not in a hurry for this machine to come up). Doing
> > something else with it? Ok, just put the backup partition as noauto in
> > /etc/fstab, and fsck and mount (or just force-mount) the partition in
> > /etc/rc.local. Now, whatever it was that was bothering you about booting so
> > slowly is up quickly, and the backup partition will get mounted in due time.
>
> Well, I have it set up so that it comes up once a day and after it finishes
> doing backups it shuts down itself. So if it crashes and starting
> up takes too much time the backup job won't fit the window it's supposed
> to. I'm still working on the server and trying to find the best
> solution for my needs. Luckily there hasn't been much use for the
> backups since there hasn't been any real accidents or failures either ;-)
>
> PS. Thanks to Nick and others for the advices and ideas.
Another thing is to move to larger block and fragment sizes. Depending
on the size distribution of your files, this will waste some space,
though.
I tested 1TB filesystems with varying block and fragment sizes, and it
is really nice to see the speedup of newfs anf fsck.
-Otto
