<lst_hoe02 <at> kwsoft.de> writes: > > > Zitat von Conrad Lawes <pxeboot <at> gmail.com>: > > > I have been using Bareos for a couple years now and Bacula > > (Community Edition) a few years before that. > > > > One thing that I have never gotten to work is hardware compression. > > I used to use LTO3 tapes that natively storage 400GB and 800GB with > > compression. Never got beyond 400GB. > > > > I've upgraded by tape library to a Dell TL2000 tape library with > > LTO6 tape drive. > > LTO6 natively stores 2.5TB and 6.25TB with compression. Hardware > > compression is turned on. However, I never get above 2.5TB of data > > on each tape. > > > > I read somewhere that you must spool your data before writing to > > tape because Bacula/Bareos requires a steady stream for compression > > to work. Tried that, still no dice. And yes, I have disabled soft > > compression for my backup jobs. All the documentation I found > > online are limited and/or out-of-date. > > > > I'm presently using 2 LTO6 tapes to complete a full backup (~3.5TB > > of data). Obviously, my 2nd tape is under utilized and should not > > be required if 6.25TB capacity is achievable. > > > > Is there any H/W compression expert that can shed some light on what > > is required by Bareos to get compression working as advertised? > > > > > > Thanks. > > To my knowledge the LTO device automatically try compression on the > delivered data. If the compression ratio is too small or negative the > compression is turned off. What is needed to get a good ration is > always the same: > > - data which are not already compressed, so no jpeg,mpeg,zip and the like > > - no compression done in software before transfering the data to tape > > - no encryption because this hides the redundancy the compression is > looking for > > spooling data is about speed, not compression. So simply do fill a > tape with well compressable data and compare the ratio with for > example a zip archive.
Correct spooling is only needed to not get shoe-shining e.g. keep the drives streaming (for LTO6 you need quite some speed to keep things running e.g. see https://en.wikipedia.org/wiki/Linear_Tape-Open e.g. 160 Mb/s uncompressed which is probably more then any single disk can give. The whole compression also kind of depends on your operating system and even on the kernel version used. The Amanda project has some notes on this: https://wiki.zmanda.com/index.php/Hardware_compression (b.t.w. searching on google on linux st driver compression has quite some hits. ST is the kernel driver that abstracts the SCSI Tape over either SAS or FC.) You also might want to look into the tapeinfo of your tape drive, for my drive in my T24 tandberg library with LTO4 SAS drive on Solaris this gives: 15:14 [root:corona][319] ~ > tapeinfo -f /dev/rmt/0ubn Product Type: Tape Drive Vendor ID: 'HP ' Product ID: 'Ultrium 4-SCSI ' Revision: 'U52U' Attached Changer API: No SerialNumber: 'HU1145KF03' MinBlock: 1 MaxBlock: 16777215 Ready: yes BufferedMode: yes Medium Type: Not Loaded Density Code: 0x44 BlockSize: 0 DataCompEnabled: yes DataCompCapable: yes DataDeCompEnabled: yes CompType: 0x1 DeCompType: 0x1 Block Position: 8126794 Partition 0 Remaining Kbytes: 400308 Partition 0 Size in Kbytes: 400308 ActivePartition: 0 EarlyWarningSize: 0 NumPartitions: 0 MaxPartitions: 0 The following things are mostly interesting: DataCompEnabled: yes DataCompCapable: yes DataDeCompEnabled: yes CompType: 0x1 DeCompType: 0x1 And things have also worked quite well for LTO3 I always got more then 400 Gb and now with LTO4 tapes also get more then 800 Gb. I had some problems before with bad LTO tapes but now with HP certified LTO3 (Yellow) and LTO4 (Green) things look quite ok. Also keep in mind its the famous disk vendor GB and NOT GiB Some examples: For incrementals/differentials: | 29 | MHE345L3 | Full | 1 | 561,414,389,760 | 1,326 | 157,680,000 | 1 | 0 | 0 | LTO | 2011-08-10 08:06:56 | | 33 | MHE349L3 | Full | 1 | 241,996,640,256 | 169 | 157,680,000 | 1 | 0 | 0 | LTO | 2011-11-05 08:05:40 | | 36 | MHE352L3 | Full | 1 | 403,016,076,288 | 484 | 157,680,000 | 1 | 0 | 0 | LTO | 2011-10-16 08:13:47 | | 55 | MHE361L3 | Full | 1 | 418,060,339,200 | 737 | 157,680,000 | 1 | 0 | 0 | LTO | 2011-02-03 08:08:48 | | 82 | PLN003L3 | Full | 1 | 828,656,640,000 | 1,077 | 157,680,000 | 1 | 0 | 0 | LTO | 2012-07-14 10:05:40 | | 86 | PLN007L3 | Full | 1 | 614,782,393,344 | 1,552 | 157,680,000 | 1 | 7 | 1 | LTO | 2013-03-31 10:06:20 | | 95 | PLN016L3 | Full | 1 | 481,131,463,680 | 1,259 | 157,680,000 | 1 | 16 | 1 | LTO | 2013-10-25 09:05:31 | | 104 | PLN102L4 | Full | 1 | 885,321,116,672 | 1,374 | 157,680,000 | 1 | 23 | 1 | LTO | 2014-06-05 10:08:36 | | 115 | PLN107L4 | Append | 1 | 979,187,072,000 | 2,357 | 157,680,000 | 1 | 5 | 1 | LTO | 2015-07-10 09:01:05 | For Fulls: | 101 | PLN020L3 | Full | 1 | 508,205,988,864 | 138 | 157,680,000 | 1 | 20 | 1 | LTO | 2013-09-01 09:19:29 | | 102 | PLN021L3 | Full | 1 | 499,406,358,528 | 139 | 157,680,000 | 1 | 21 | 1 | LTO | 2013-10-06 09:09:07 | | 103 | PLN101L4 | Full | 1 | 1,010,252,371,968 | 269 | 157,680,000 | 1 | 22 | 1 | LTO | 2013-12-01 09:40:53 | | 105 | PLN103L4 | Full | 1 | 992,451,511,296 | 263 | 157,680,000 | 1 | 1 | 1 | LTO | 2014-02-02 09:51:31 | | 106 | PLN104L4 | Full | 1 | 1,012,483,519,488 | 265 | 157,680,000 | 1 | 2 | 1 | LTO | 2014-04-06 09:50:07 | | 107 | PLN105L4 | Full | 1 | 1,005,624,935,424 | 258 | 157,680,000 | 1 | 3 | 1 | LTO | 2014-06-01 09:43:43 | | 112 | PLN106L4 | Full | 1 | 1,001,966,861,312 | 249 | 157,680,000 | 1 | 4 | 1 | LTO | 2014-07-20 09:14:42 | That is all on Solaris with a tape device named /dev/rmt/0ubn which means use ultra compression (u) BSD type (b) non-rewinding (n) -- Marco van Wieringen [email protected] Bareos GmbH & Co. KG Phone: +49-221-63069389 http://www.bareos.com Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646 Komplementär: Bareos Verwaltungs-GmbH Geschäftsführer: Stephan Dühr, M. Außendorf, J. Steffens, P. Storz, M. v. Wieringen -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. For more options, visit https://groups.google.com/d/optout.
