Re: tape size question
I only do HW compression. I tried the software compression once before and it was excruciating slow. I have a newer tape server with many more cores so maybe I should give SW compression another try.The big question is how much better is the SW compression than the HW compression? What do I gain. Well any way the range that Toomas gave is about the range that I have been seeing so maybe things are working OK. Thanks all for your comments. Robert _ Robert P. McGraw, Jr. Manager, Computer SystemEMAIL: rmcg...@purdue.edu Purdue UniversityROOM: MATH-807 Department of Mathematics PHONE: (765) 494-6055 150 N. University Street West Lafayette, IN 47907-2067 On 5/24/13 12:24 AM, Toomas Aas toomas@raad.tartu.ee wrote: Hello! On Thu, 23 May 2013 McGraw, Robert P wrote: I use hardware compression to max the size of the tape. Uncompressed the tape is 800GB compressed the theoretical max size is 1.6TB. I know that I will not get the 1.6TB. In the amanda.conf file I tell amanda that my tape is length 1500 gbytes and that my tape drive uses LEOM. For a real life data point, on my LTO4 tape drive with HW compression enabled I get 900 GB to 1.1 TB. -- Toomas Aas
Re: tape size question
Robert,
Do amanda report a tape error?
What it doesn't put on tape? dump from the current run or from previous run?
You give us nothing to look at it, can you post the amdump.? log file?.
Jean-Louis
On 05/23/2013 09:59 AM, McGraw, Robert P wrote:
Amanda Server Info:
Thu May 23 05:36:00 2013: amandad: build: VERSION=Amanda-3.2.3
Thu May 23 05:36:00 2013: amandad:BUILT_DATE=Tue Feb 5
17:10:59 EST 2013 BUILT_MACH=
Thu May 23 05:36:00 2013: amandad:BUILT_REV=3994
BUILT_BRANCH=3_2_3
Thu May 23 05:36:00 2013: amandad:
CC=/opt/SunStudio/sunstudio12.1/bin/cc
Amanda.conf snippet:
define changer sl24 {
tpchanger chg-robot:/dev/changer/0
changerfile chg-zd-mtx-state
property tape-device 0=tape:/dev/rmt/0cbn
property eject-before-unload true
property eject-delay 10
property unload-delay 10
property use-slots 1-20
property load-poll 0s poll 5s until 120s
device-property BLOCK_SIZE 2 mbytes
device-property LEOM true
define tapetype LTO4-HWC {
comment LTO4-Hardware Compression on.
length 1500 gbytes
filemark 0 kbytes
speed 102216 kps
part_size 40GB
}
holdingdisk hd1 {
comment holding disk
directory /zexport/planck/holdingdisk/daily
use 3000GB
chunksize 10GB
}
I use hardware compression to max the size of the tape. Uncompressed
the tape is 800GB compressed the theoretical max size is 1.6TB. I know
that I will not get the 1.6TB.
In the amanda.conf file I tell amanda that my tape is length 1500
gbytes and that my tape drive uses LEOM.
Why does amanda stop at %52 when I still have 1.5TB of data in the
holding disk to write to the tape? It is hard to believe that the LTO4
compression is so bad that I am not getting any compression at all.
Does anyone else have a similar setup if so do my parameters look OK.
Thanks
USAGE BY TAPE:
Label Time Size % DLEs Parts
D02006 2:24 806117M 52.57793
_
Robert P. McGraw, Jr.
Manager, Computer SystemEMAIL: rmcg...@purdue.edu
mailto:rmcg...@purdue.edu
Purdue UniversityROOM: MATH-807
Department of Mathematics PHONE: (765) 494-6055
150 N. University Street
West Lafayette, IN 47907-2067
_
Robert P. McGraw, Jr.
Manager, Computer SystemEMAIL: rmcg...@purdue.edu
mailto:rmcg...@purdue.edu
Purdue UniversityROOM: MATH-807
Department of Mathematics PHONE: (765) 494-6055
150 N. University Street
West Lafayette, IN 47907-2067
Re: tape size question
On Thu, May 23, 2013 at 01:59:33PM +, McGraw, Robert P wrote:
Amanda Server Info:
Thu May 23 05:36:00 2013: amandad: build: VERSION=Amanda-3.2.3
Thu May 23 05:36:00 2013: amandad:BUILT_DATE=Tue Feb 5 17:10:59
EST 2013 BUILT_MACH=
Thu May 23 05:36:00 2013: amandad:BUILT_REV=3994
BUILT_BRANCH=3_2_3
Thu May 23 05:36:00 2013: amandad:
CC=/opt/SunStudio/sunstudio12.1/bin/cc
Amanda.conf snippet:
define changer sl24 {
tpchanger chg-robot:/dev/changer/0
changerfile chg-zd-mtx-state
property tape-device 0=tape:/dev/rmt/0cbn
property eject-before-unload true
property eject-delay 10
property unload-delay 10
property use-slots 1-20
property load-poll 0s poll 5s until 120s
device-property BLOCK_SIZE 2 mbytes
device-property LEOM true
define tapetype LTO4-HWC {
comment LTO4-Hardware Compression on.
length 1500 gbytes
filemark 0 kbytes
speed 102216 kps
part_size 40GB
}
holdingdisk hd1 {
comment holding disk
directory /zexport/planck/holdingdisk/daily
use 3000GB
chunksize 10GB
}
I use hardware compression to max the size of the tape. Uncompressed the tape
is 800GB compressed the theoretical max size is 1.6TB. I know that I will not
get the 1.6TB.
In the amanda.conf file I tell amanda that my tape is length 1500 gbytes
and that my tape drive uses LEOM.
Why does amanda stop at %52 when I still have 1.5TB of data in the holding
disk to write to the tape? It is hard to believe that the LTO4 compression is
so bad that I am not getting any compression at all.
Perhaps this is a case of misunderstanding the tape properties.
Your tapes will only hold 800GB of data! Period.
However, if your data is compressible (lots of data is not)
then the hardware compressor might fit more data into those
800GB of space on the tape. If your data is not compressible
then the hardware compressor can't work magic and squeeze
more onto the tape.
Lots of data is not compressible, or can be compressed only
a small amount. For example, already compressed data, audio,
photo, and video, compiled programs, encrypted data, and
(I think) database files compress poorly.
Jon
--
Jon H. LaBadie j...@jgcomp.com
11226 South Shore Rd. (703) 787-0688 (H)
Reston, VA 20190 (609) 477-8330 (C)
Re: tape size question
On Thu, 23 May 2013 13:59:33 + McGraw, Robert P rmcg...@purdue.edu wrote: Why does amanda stop at %52 when I still have 1.5TB of data in the holding disk to write to the tape? It is hard to believe that the LTO4 compression is so bad that I am not getting any compression at all. It would be if you have both tape drive compression turned on (as you do) and compression specified in the disk list entries (DLEs). Conventional wisdom on this list has been that DLE compression (using gzip, say), is more efficient than tape drive compression. -- Charles Curley /\ASCII Ribbon Campaign Looking for fine software \ /Respect for open standards and/or writing? X No HTML/RTF in email http://www.charlescurley.com/ \No M$ Word docs in email Key fingerprint = CE5C 6645 A45A 64E4 94C0 809C FFF6 4C48 4ECD DFDB
Re: tape size question
Robert, On Thu, May 23, 2013 at 10:04:29AM -0600, Charles Curley wrote: On Thu, 23 May 2013 13:59:33 + McGraw, Robert P rmcg...@purdue.edu wrote: Why does amanda stop at %52 when I still have 1.5TB of data in the holding disk to write to the tape? It is hard to believe that the LTO4 compression is so bad that I am not getting any compression at all. It would be if you have both tape drive compression turned on (as you do) and compression specified in the disk list entries (DLEs). Conventional wisdom on this list has been that DLE compression (using gzip, say), is more efficient than tape drive compression. Tape drive compression is nice, and the newer technologies do a block by block determination as to whether the data can be compressed or not [used to be that if you SW compressed and then HW compressed you might get inflation and not only waste CPU but end up occupying more tape than if you hand't compressed at all]. What is more difficult is estimation of tape usage. SW compression gives you a raw size number, a compressed size number and Amanda can sum the compressed size numbers up and compare them to the capacity of the tape. With HW compression its kid of out of your hands and you don't really know, except from experience, what is going to fit on tape any given night. If you run SW compression and see you get 1/3 data reduction, you can move the DLE to HW compression and will hopefully get 1/3 data compression, which you might allow for by lenthening the tape capacity by 1/3 the size of the particular DLE. At least, I've done this in the past. I don't currently run HW compression on any DLE or amanda config that I'm managing. YMMV. Database compression depends a lot on your database. I have a lot of sparse databases that compress very well. This is not always the case, you have to know your data. -- Charles Curley /\ASCII Ribbon Campaign Looking for fine software \ /Respect for open standards and/or writing? X No HTML/RTF in email http://www.charlescurley.com/ \No M$ Word docs in email Key fingerprint = CE5C 6645 A45A 64E4 94C0 809C FFF6 4C48 4ECD DFDB --- Brian R Cuttler brian.cutt...@wadsworth.org Computer Systems Support(v) 518 486-1697 Wadsworth Center(f) 518 473-6384 NYS Department of HealthHelp Desk 518 473-0773
Re: tape size question
Hello! On Thu, 23 May 2013 McGraw, Robert P wrote: I use hardware compression to max the size of the tape. Uncompressed the tape is 800GB compressed the theoretical max size is 1.6TB. I know that I will not get the 1.6TB. In the amanda.conf file I tell amanda that my tape is length 1500 gbytes and that my tape drive uses LEOM. For a real life data point, on my LTO4 tape drive with HW compression enabled I get 900 GB to 1.1 TB. -- Toomas Aas
Re: tape size question
On Fri, May 24, 2013 at 07:24:01AM +0300, Toomas Aas wrote: Hello! On Thu, 23 May 2013 McGraw, Robert P wrote: I use hardware compression to max the size of the tape. Uncompressed the tape is 800GB compressed the theoretical max size is 1.6TB. I know that I will not get the 1.6TB. In the amanda.conf file I tell amanda that my tape is length 1500 gbytes and that my tape drive uses LEOM. For a real life data point, on my LTO4 tape drive with HW compression enabled I get 900 GB to 1.1 TB. Toomas, Is that with HW compression ONLY? Or are you also doing SW compression? Jon -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (609) 477-8330 (C)
