Re: amanda tape algorithm

2018-01-05 Thread Jean-Louis Martineau 
On 04/01/18 12:59 PM, Ned Danieley wrote:
>
> I'm using
>
> taperalgo largestfit
>
> which I assume means that amanda will write to tape the largest DLE
> available. I have 'runtapes' set to 2, and occasionally I'll see amanda
> move on to the second tape when the first tape is reporting around 80% 
> full.
> I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs
> that would fit in the remaining 20%. is there any way to find out (after
> the fact) what DLEs were in the queue? I've looked at the taper debug file
> but it just seems to have a list of each DLE as it is written.
Not easily, you can parse the amdump. file

Make a copy of amdump.
Remove all lines after it sent the command to the taper
Run amstatus on that modified file

What was the size of the dle that failed to be written to the first tape?
>
> I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know
> about the ability to split DLEs, but I'm not ready to take that step.
You should try.

Jean-Louis
This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.
If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone.  Instead, destroy it and notify me by reply 
e-mail

Re: amanda tape algorithm

2018-01-04 Thread Jon LaBadie 
On Thu, Jan 04, 2018 at 08:59:32PM -0500, Ned Danieley wrote:
> On Thu, Jan 04, 2018 at 05:16:59PM -0700, Steven Backus wrote:
> > Ned Danieley  wrote:
> > 
> > > ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that
> > > the rated capacity would be accurate. I'll give it a try; in the meantime,
> > > has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive.
> > 
> > I did and got:
> > 
> > define tapetype LTO6 {
> > comment "Created by amtapetype; compression disabled"
> > length 2442954880 kbytes
> > filemark 7456397 kbytes
> > speed 154519 kps
> > blocksize 32 kbytes
> > }
> 
> thanks. that fairly well matches what I was using
> 
> define tapetype LTO6comp {
>   length 244352 kbytes
>   filemark 868 kbytes
>   speed 157129 kps
>   blocksize 2048 kbytes
>   }
> 
> except for filemark; can anyone comment on that? looking at the tapetype
> definitions on the zmanda wiki, it seems that most of the LTO entries have
> zero kbytes for filemark...
> 
It is the space (if any) left between files written to the tape.

Suppose you write a continuous stream of data to a tape
and you can write exactly 100GB.  If no space was left
between files you should be able to write 100 x 1GB files.
When you try it (as amtapetype does) and you find you can
only write 98 x 1GB files, 2GB was used by "filemarks".
Divide 2GB/98 files and you have your filemark.

I believe Amanda does consider filemarks in its determination
of what will fit on a tape.

jl
-- 
Jon H. LaBadie j...@jgcomp.com
 11226 South Shore Rd.  (703) 787-0688 (H)
 Reston, VA  20190  (703) 935-6720 (C)

Re: amanda tape algorithm

2018-01-04 Thread Ned Danieley 
On Thu, Jan 04, 2018 at 05:16:59PM -0700, Steven Backus wrote:
> Ned Danieley  wrote:
> 
> > ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that
> > the rated capacity would be accurate. I'll give it a try; in the meantime,
> > has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive.
> 
> I did and got:
> 
> define tapetype LTO6 {
> comment "Created by amtapetype; compression disabled"
> length 2442954880 kbytes
> filemark 7456397 kbytes
> speed 154519 kps
> blocksize 32 kbytes
> }

thanks. that fairly well matches what I was using

define tapetype LTO6comp {
length 244352 kbytes
filemark 868 kbytes
speed 157129 kps
blocksize 2048 kbytes
}

except for filemark; can anyone comment on that? looking at the tapetype
definitions on the zmanda wiki, it seems that most of the LTO entries have
zero kbytes for filemark...

-- 
Ned Danieley (ned.danie...@duke.edu)
Department of Biomedical Engineering
Box 90281, Duke University
Durham, NC  27708   (919) 660-5111

http://dilbert.com/strips/comic/2012-02-11/

Re: amanda tape algorithm

2018-01-04 Thread Steven Backus 
Ned Danieley  wrote:

> ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that
> the rated capacity would be accurate. I'll give it a try; in the meantime,
> has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive.

I did and got:

define tapetype LTO6 {
comment "Created by amtapetype; compression disabled"
length 2442954880 kbytes
filemark 7456397 kbytes
speed 154519 kps
blocksize 32 kbytes
}

Steve
-- 
Steven J. BackusComputer Systems Manager
University of Utah  E-Mail:  steven.bac...@utah.edu
Genetic EpidemiologyAlternate:  bac...@math.utah.edu
391 Chipeta Way -- Suite D  Office:  801.587.9308
Salt Lake City, UT 84108-1266   http://www.math.utah.edu/~backus

Re: amanda tape algorithm

2018-01-04 Thread Jon LaBadie 
On Thu, Jan 04, 2018 at 04:58:13PM -0500, Ned Danieley wrote:
> On Thu, Jan 04, 2018 at 09:43:35PM +, Debra S Baddorf wrote:
> > 
> > > On Jan 4, 2018, at 11:59 AM, Ned Danieley  wrote:
> > > 
> > > 
> > > I'm using
> > > 
> > > taperalgo largestfit
> > > 
> > > which I assume means that amanda will write to tape the largest DLE
> > > available. I have 'runtapes' set to 2, and occasionally I'll see amanda
> > > move on to the second tape when the first tape is reporting around 80% 
> > > full.
> > > I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs
> > > that would fit in the remaining 20%. is there any way to find out (after
> > > the fact) what DLEs were in the queue? I've looked at the taper debug file
> > > but it just seems to have a list of each DLE as it is written.
> > > 
> > > I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know
> > > about the ability to split DLEs, but I'm not ready to take that step.
> > > 
> > 
> > Did you actually test the size of your own tapes? 
> > 
> 
> ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that
> the rated capacity would be accurate. I'll give it a try; in the meantime,
> has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive.

That 2.5TB is likely marketing-speak using decimal number.
Switching to binary TB would immediately reduce the size
by nearly 250GB (about 9% of the listed capacity).

jl
-- 
Jon H. LaBadie j...@jgcomp.com
 11226 South Shore Rd.  (703) 787-0688 (H)
 Reston, VA  20190  (703) 935-6720 (C)

Re: amanda tape algorithm

2018-01-04 Thread Ned Danieley 
On Thu, Jan 04, 2018 at 09:43:35PM +, Debra S Baddorf wrote:
> 
> > On Jan 4, 2018, at 11:59 AM, Ned Danieley  wrote:
> > 
> > 
> > I'm using
> > 
> > taperalgo largestfit
> > 
> > which I assume means that amanda will write to tape the largest DLE
> > available. I have 'runtapes' set to 2, and occasionally I'll see amanda
> > move on to the second tape when the first tape is reporting around 80% full.
> > I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs
> > that would fit in the remaining 20%. is there any way to find out (after
> > the fact) what DLEs were in the queue? I've looked at the taper debug file
> > but it just seems to have a list of each DLE as it is written.
> > 
> > I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know
> > about the ability to split DLEs, but I'm not ready to take that step.
> > 
> > -- 
> > Ned Danieley (ned.danie...@duke.edu)
> > Department of Biomedical Engineering
> > Box 90281, Duke University
> > Durham, NC  27708   (919) 660-5111
> > 
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__dilbert.com_strips_comic_2012-2D02-2D11_=DwIGaQ=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc=seIgoLdvEn-8badm2JmCmh78EJ9F9-Hw-LulwGbFPZI=DGzKNKCqh_8C-Y1Y3Smdu29gOMu_5Y82AnrH-oAP-vg=dB2_g2FDMHahFgcGA1CJLdbMwaXZJjrvHGFYP_pmwGM=
> >  
> 
> 
> Did you actually test the size of your own tapes?   (There’s a proceedure 
> that does this …. help me
> out with the name, guys?) 
> 
> Amanda, I believe, uses your number to decide how much space is left on the 
> tape.  It’ll
> pick the biggest DLE that ought to fit in that spot   (“largestfit”).   If 
> you’ve over-estimated
> the tape size,  then this DLE  will NOT fit.  At that point,  amanda can’t 
> back up and try
> a smaller DLE.  It merely stops trying with that DLE  (saves it for later) 
> and moves on to 
> the next tape.  
> At this point, the “largestfit”  is probably a bigger DLE, since the 
> whole tape is available
> for the second tape.
> 
> Rinse and repeat.
> 
> So — maybe tell amanda a smaller size for the tape?   Then it’ll try a 
> smaller  “last”  DLE,
> and it will actually fit.
> 
> At least,  this is my understanding of how she works.

ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that
the rated capacity would be accurate. I'll give it a try; in the meantime,
has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive.

-- 
Ned Danieley (ned.danie...@duke.edu)
Department of Biomedical Engineering
Box 90281, Duke University
Durham, NC  27708   (919) 660-5111

http://dilbert.com/strips/comic/2012-02-11/

Re: amanda tape algorithm

2018-01-04 Thread Debra S Baddorf 

> On Jan 4, 2018, at 11:59 AM, Ned Danieley  wrote:
> 
> 
> I'm using
> 
> taperalgo largestfit
> 
> which I assume means that amanda will write to tape the largest DLE
> available. I have 'runtapes' set to 2, and occasionally I'll see amanda
> move on to the second tape when the first tape is reporting around 80% full.
> I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs
> that would fit in the remaining 20%. is there any way to find out (after
> the fact) what DLEs were in the queue? I've looked at the taper debug file
> but it just seems to have a list of each DLE as it is written.
> 
> I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know
> about the ability to split DLEs, but I'm not ready to take that step.
> 
> -- 
> Ned Danieley (ned.danie...@duke.edu)
> Department of Biomedical Engineering
> Box 90281, Duke University
> Durham, NC  27708   (919) 660-5111
> 
> http://dilbert.com/strips/comic/2012-02-11/


Did you actually test the size of your own tapes?   (There’s a proceedure that 
does this …. help me
out with the name, guys?) 

Amanda, I believe, uses your number to decide how much space is left on the 
tape.  It’ll
pick the biggest DLE that ought to fit in that spot   (“largestfit”).   If 
you’ve over-estimated
the tape size,  then this DLE  will NOT fit.  At that point,  amanda can’t back 
up and try
a smaller DLE.  It merely stops trying with that DLE  (saves it for later) and 
moves on to 
the next tape.  
At this point, the “largestfit”  is probably a bigger DLE, since the whole 
tape is available
for the second tape.

Rinse and repeat.

So — maybe tell amanda a smaller size for the tape?   Then it’ll try a smaller  
“last”  DLE,
and it will actually fit.

At least,  this is my understanding of how she works.

Deb Baddorf
Fermilab

amanda tape algorithm

2018-01-04 Thread Ned Danieley 

I'm using

taperalgo largestfit

which I assume means that amanda will write to tape the largest DLE
available. I have 'runtapes' set to 2, and occasionally I'll see amanda
move on to the second tape when the first tape is reporting around 80% full.
I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs
that would fit in the remaining 20%. is there any way to find out (after
the fact) what DLEs were in the queue? I've looked at the taper debug file
but it just seems to have a list of each DLE as it is written.

I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know
about the ability to split DLEs, but I'm not ready to take that step.

-- 
Ned Danieley (ned.danie...@duke.edu)
Department of Biomedical Engineering
Box 90281, Duke University
Durham, NC  27708   (919) 660-5111

http://dilbert.com/strips/comic/2012-02-11/