Hi all, I’m setting up bareos as a backup solution for my large (60TB) RAID. I’m backing up to another RAID on a different machine, and the File Daemon and Storage Daemon are connected by GbE.
The setup works well. With no software compression, I’m maxing out the GbE link, sustaining 117 MB/s (10^6 *bytes/sec). I’m looking into compression because 7.4Tb takes almost 18 hours to backup, and there will be more. With compression, I have a question: When Bareos reports “[FD|SD] Bytes Written”, is this is a post-compression value? (e.g., the number of bytes read by the FD is larger by the “Software Compression” percentage, correct?) The short version is: assuming the above, it looks like my “effective backup rates” (Bytes read by the FD/sec) is: No compression: 117Mb/sec (100%) GZIP: 65.82 Mb/sec (56%) LZOFAST: 236.44 Mb/sec (202%) Pigz [1] projection: 532.76 Mb/sec (454%) That last bit is a projection based on comparing the Parallel Implementation of GZip (PIGZ)[1] to the standard gzip on my system. I started messing with parallel compression because I noted that during a GZIP run, my 24-thread system was ~95% idle. It is interesting to note that PIGZ compiles against versions of libz found on RHEL/CentOS/SL 6 machines. I would like to submit a feature request for PIGZ support on Linux-based FDs, but I wanted to run it by the group first. It produces GZIP-compatible output files, it just does the compression in parallel if you have an SMP machine. (I checked by compressing with PIGZ and uncompressing with gunzip.) Thanks, Bryce [1] http://zlib.net/pigz/ -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
,,,,Backup rate (1e6 Bytes/sec),, Compression Method,Compression Ratio,SD Bytes Written,Time Elapsed (s),Compressed,Raw,Rate % non compressed GZIP,91.0%,"20,690,583,231",3493,5.92,65.82,56% LZOFAST,89.8%,"60,846,309,819",2523,24.12,236.44,202% None,0%,"7,460,641,452,593",63599,117.31,117.31,100% pigz projection,91.0%,"20,690,583,231",432,47.95,532.76,454%
