I am using Bacula Community 9.4.4 for backup of a server. It has two drives with about 2.8 TB of information on each drive.
My backup system has ~28 TB of disk storage in a single partition using ZFS 0.8.5 with compression and deduplication turned on. I get about 1.3 - 1.4x compression so it is equivalent to 36 - 39 TB of storage. Bacula deduplication is not turned on. I have backup jobs for each drive with daily incremental, weekly differential, and monthly full backups. 1. The full backups are about 2.8 TB with 12,000,000 - 20,000,000 files and each run for 2 - 3 days. This is a problem since it means that we can lose up to three days of files if someone has a problem. I am looking for suggestions about how to configure my backup schedule to avoid this. I can only get 3 - 4 months of backups out of this system. I set things so pruning keeps enough volumes empty for the backups to finish. Each volume has > 1 TB of files that do not change much such as cad tools. These are not as critical since they can be downloaded and installed again. Each volume also has about 1 TB of user files which are more critical although most of them are not changing. I am splitting the backups into separate jobs for the CAD tools and the user data. I am looking for ways to optimize this storage. 2. Any suggestions for backup strategies? 3. I would try Virtual backups. Can I just run virtual full backups periodically after the first full backup? I already have full backups so I could just schedule virtual full backups instead of full ones. Do I need separate media types or devices for Virtual Full backups? My ZFS partition is already at ~ 80 % so I do not have much room. Are Virtual full backups sort of like pruning so that my oldest backup data is from when the last VirtualFull took place? I looked at the following documents, alhtough they may be more for the Enterprise version than the community one. https://www.baculasystems.com/corporate-data-backup-software-solutions/bacula-enterprise-data-backup-software/features/synthetic-backup/ https://www.baculasystems.com/ml/pvf3.svg#PVF-title 4. I would be interested to try turning on deduplication. I looked at: https://blog.bacula.org/whitepapers/DedupVolumes.pdf I am worried about the statement below in the guide: You must take particular attention to define a unique Media Type for devices that are Aligned as well as for each Virtual Autochanger that uses a different Archive Device directory. If you use the same Media Type for an Aligned device type that you use for a normal disk Volume, you run the risk that you will have data corruption on disk Volumes that are used on Aligned and non-Aligned devices. In my bacula-sd.conf file I had already specified most of the settings for deduplication. Note that even though the device is called Aligned-Disk it was not specified as aligned disk. The problem again is my ZFS partition is already mostly filled. If I make a separate media type then can I point it at the same disk partition or do I have to use a separate partition so that new disk volumes will be created just for the aligned storage. My concern with deduplication is that it uses a lot of ram so I really need to test it with my largest backups to see how it goes. I have seen numbers like 5.5 GB of Ram/TB of storage. For my system that would be > 150 GB ram and I only have 48 GB. It is probably cheaper to get a new backup system with more drives than to start adding tons of RAM. Device { Name = Aligned-Disk # Plugin Directory = /opt/bacula/plugins # Device Type = Aligned # Must be aligned Media Type = File1 Archive Device = /mpool1/bacula # Or /mnt/ddumbfs.mnt if ddumfs mounting point. LabelMedia = yes; Random Access = Yes; AutomaticMount = yes; RemovableMedia = no; AlwaysOpen = no; Maximum Concurrent Jobs = 1 # Always 1 for Aligned Minimum Block Size=0K Maximum Block Size=128K File Alignment=128K Padding Size=512 Minimum Aligned Size=4096 } Device { Name = FileChgr1-Dev2 Media Type = File1 Archive Device = /mpool1/bacula LabelMedia = yes; # lets Bacula label unlabeled media Random Access = Yes; AutomaticMount = yes; # when device opened, read it RemovableMedia = no; AlwaysOpen = no; Maximum Concurrent Jobs = 5 } Thanks
_______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
