Amanda doesn't know or worry about the 2 GB file size limit. Some
systems have a 2 GB limit and some don't. It's entirely up to the
filesystem you have installed on your holding disk(s). Amanda will
always try to stage the dump to the holding disk, unless the dump
size estimate is larger than all holding disk(s) available. Dumping
straight to tape only happens when the holding disk can't be used
due to total available size (i.e. the dump is bigger than the entire
holding disk(s)). Any 2 GB file size limit you might or might not
have doesn't come into consideration at this stage.
The way chunksize works is the dump will go to a file on the holding
disk until chunksize is reached. At that point the first file
(chunk) is closed and a 2nd is started. This process repeats itself
until the dump is complete. Then the dump is scheduled for taping,
at which point it is reassembled on the tape with no vestiges
remaining of the chunking process. (I.e. it's all back in one tape
file, identical to what it would have looked like had it been dumped
straight to tape.)
Knowing all this, would you now agree that this is a reasonable
solution?
-Mitch
On Thu, 14 Dec 2000, Clifford Ekholm wrote:
>
> Hi again Mitch, David
>
> and thanks for your answers. I will follow your suggestions and start
> using 1 Gb as chunksize.
>
> Why am I writing this then? Well, since my dumps did not work my guess
> is that this is what happened:
>
> Amanda made an estimate what size the file would be. Since most of the new
> stuff on the disk were tiff images (from an electron microscope) and mp3
> files I had just encoded, amandas estimate was wrong. So amanda figured it
> would fit within 2 Gbytes but did not check that this was not the case
> while it wrote it to the disk. If this is the case then perhaps there
> should be a check of how much is really written to disk and to store it in
> chunks or write it to tape instead, when it hits the 2 Gb limit.
>
> If I set chunksize to 1 Gb I guess everything will work fine, but it does
> not seem to me to be the right way of getting around the problem.
>
> Some of the filesystems I dump (from Windows NT machines) have a final
> size perhaps only 30 % of the uncompressed size; so a 2Gb filesystem is
> compressed to about 600 Mb. What happens if I have chunksize set to
> 1.5 Gb and they suddenly add lets say another 2 Gb of data thats does
> not compress at all? Will amanda think that it will fit without chunking
> or will it fail when it hits the 2Gb limit?
>
> Sorry Im not good enough hacker to be able to check the code myself or I
> would not waste your time. You dont have to answer me on this one, I just
> wanted you to consider this in case its relevant.
>
>
> Merry Christmas both of you.
>
> Clifford
>
>
> P.S Mitch, David also suggested a 1 Gb chunksize (answer not included
> here)
>
>
> On Wed, 13 Dec 2000, Mitch Collinsworth wrote:
>
> > The 2 GB limit you're seeing is a file size limit on your holding
> > disk. You're not encountering it with the level 0 because it goes
> > direct to tape. Direct to tape happens when the dump won't fit in
> > the holding disk at all, it has nothing to do with chunksize.
> >
> > Using chunksize -1 means don't do file chunking on the holding disk,
> > which is where your problem is. Holding disk chunking is a feature
> > that works around the 2 GB file size limit. If you turn chunking
> > on, say by setting chunksize 1000 Mb, that should fix things up
> > nicely.
> >
> > -Mitch
> >
> >
> > On Wed, 13 Dec 2000, Clifford Ekholm wrote:
> >
> > > Hi
> > >
> > > Have tried to look in various FAQs, manuals etc but not found a good
> > > answer.
> > >
> > > I am dumping several filesystem to a 12 Gbyte tape and have a holdingdisk
> > > that is 8Gbyte in size. Everything has worked fine untill I had an
> > > incremental (level 3) dump larger than about 2Gbytes.
> > >
> > > I get the errormessage saying
> > >
> > > "sdc2 lev 3 FAILED ["data write: File too large"]
> > >
> > > The whole partition is about 10 Gbyte and level 0 dumps work
> > > fine; directly to tape.
> > >
> > > In my amanda.conf I have the following settings for the holdingdisk:
> > >
> > > holdingdisk hold1 {
> > > comment "main holding disk"
> > > directory "/scratch/Amanda"
> > > use 8000 Mb
> > > chunksize -1
> > > }
> > >
> > > As far as i know "chunksize -1" should mean that filesystems larger than
> > > almost 2Gb should be dumped directly to tape. In level 0 dumps they are,
> > > but I seem to run into problems with incremental dumps larger than 2 Gb.
> > > The dump runs untill it has dumped about 2Gb to the holdingdisk and then
> > > fails (without dumping directly to tape after it failed with the
> > > disk); tape had lots of free space left.
> > >
> > > If this is covered in the FAQ Im sorry I bothered you, but would be glad
> > > if you could point me in how to config amanda for this.
> > > I also just noticed that amanda-2.4.2 is out; perhaps its "fixed" there.
> > >
> > > Thanks a lot for a great utility!
> > >
> > > and
> > >
> > > have yourself a nice day.
> > >
> > >
> > > Clifford
> > >
> > > P.S. If replying use [EMAIL PROTECTED] - machine is behind firewall.
> > >
> > > /********************** [EMAIL PROTECTED] *********************************
> > > Clifford Ekholm <- WWW: http://noon.abo.fi/~cliffa/
> > > Abo Akademi University <- WWW: http://www.abo.fi/
> > > Dept. of Chemical Engineering *
> > > Laboratory for Inorganic Chem. <- WWW: http://noon.abo.fi/~Ook/
> > > Biskopsgatan 8 * Voice: (+358)-2-2154822 Noon - Post Midnight
> > > FIN-20 500 Abo * Fax: -2154962
> > > FINLAND *
> > > ******************************************************************************/
> > >
> >
>
> /********************** [EMAIL PROTECTED] *********************************
> Clifford Ekholm <- WWW: http://noon.abo.fi/~cliffa/
> Abo Akademi University <- WWW: http://www.abo.fi/
> Dept. of Chemical Engineering *
> Laboratory for Inorganic Chem. <- WWW: http://noon.abo.fi/~Ook/
> Biskopsgatan 8 * Voice: (+358)-2-2154822 Noon - Post Midnight
> FIN-20 500 Abo * Fax: -2154962
> FINLAND *
> ******************************************************************************/
>
>
