Andrea, Sorry for the delay in reply. I'm going to have to post this in multiple pieces due to message length limits.
One of my bacula installations runs on a synology appliance and uploads to the Backblaze B2, an S3 compatible cloud service. You should be able to use any S3 compatible cloud service. I separate out different job levels into their own pool for the local storage. This structure is mirrored in the cloud. I have a bucket for each cloud pool. Each cloud pool must have its own autochanger and device resources in bacula-sd.conf, and its own storage resource in bacula-dir.conf. Each local pool has an equivalent cloud pool. The local pools reference the cloud pools using 'NextPool='.I use selection method PoolUncopiedJobs to choose which jobs are uploaded. I am using bacula 15.x, and the amazon cloud driver and volume encryption introduced in that version of bacula. You do not need to use volume encryption if you are not concerned about this, but if you do you should know that each volume is encrypted with a unique key. If these keys are not backed up separately, recovery from your cloud volumes alone will be impossible. I delete all local copies of the copy jobs after upload. I do not upload at the time when the job is ran. I upload afterward with a copy volume part sweeper script. I previously uploaded at time of copy job execution, but I found that bacula does not retry failed part uploads during that process, and bacula also does not throw an error when a part fails to upload. Without a part sweeper script to instruct bconsole to upload any cloud volume parts not previously uploaded, any such part would simply be left unuploaded forever. My part sweeper script also outputs to a log file, then reads its own log to check for errors. The part sweeper script exits 1 once it's done all its work if it finds any failure to upload a cloud volume part. My basic psuedocode config structure (real examples further down): bacula-sd.conf: Cloud resource for bucket Cloud-Full (TruncateCache = "AfterUpload"; Upload = "no") Cloud resource for bucket Cloud-Diff (TruncateCache = "AfterUpload"; Upload = "no") Cloud resource for bucket Cloud-Inc (TruncateCache = "AfterUpload"; Upload = "no") Autochanger resource for Cloud resource Cloud-Full Autochanger resource for Cloud resource Cloud-Diff Autochanger resource for Cloud resource Cloud-Inc Device resources for Cloud resource Cloud-Full Device resources for Cloud resource Cloud-Diff Device resources for Cloud resource Cloud-Inc Autochanger resource for local file based storage Device resources for local file based storage bacula-dir.conf: Storage for Cloud-Full Storage for Cloud-Diff Storage for Cloud-Inc Storage for local file based storage Pool Local-Full (MaximumVolumeJobs = 1; NextPool references the equivalent cloud pool) Pool Local-Diff (MaximumVolumeJobs = 1; NextPool references the equivalent cloud pool) Pool Local-Inc (MaximumVolumeJobs = 1; NextPool references the equivalent cloud pool) Pool Cloud-Full (MaximumVolumeJobs = 1) Pool Cloud-Diff (MaximumVolumeJobs = 1) Pool Cloud-Inc (MaximumVolumeJobs = 1) Schedule: Daily at 23:05. Used with backup jobs. Jobdefs selects the correct level. Schedule: Copy-End-Of-Day at 23:50. Copy jobs. Doesn't actually need to be separate at this point. Priority ensures that this will run at the correct point in the process. Schedule: NightlyAfterCopy at 23:54. Used to launch the part sweeper after copy jobs are finished. Not actually needed - priority ensures this will run at the correct point in the process. Jobdefs: Specifies local full, diff, inc pools; maximum interval between levels full and diff; no duplicate jobs allowed; ReRunFailedLevels = yes admin-prune-allpools-job: Priority 1. Script with bconsole command to prune all eligible jobs. admin-truncate-volumes-job: Priority 2. Script with bconsole commands to truncate all pools on each storage. Minimizes cloud storage usage. Job 1 Job 2 admin-copy-control-launcher-job (lower priority than main backup jobs; launches the copy jobs after main jobs have finished. Otherwise the copy jobs will only queue up yesterdays jobs for copying, because todays backup jobs have not finished at the time when the copy jobs were launched) Copy-Control-Full-job (one copy control job per pool. Manual schedule. Targets local pool for each level. Copy control job for each level prevents cancellation of copy control jobs as duplicate) Copy-Control-Diff-job (one copy control job per pool. Manual schedule. Targets local pool for each level. Copy control job for each level prevents cancellation of copy control jobs as duplicate) Copy-Control-Inc-job (one copy control job per pool. Manual schedule. Targets local pool for each level. Copy control job for each level prevents cancellation of copy control jobs as duplicate) admin-cloud-volume-part-sweeper-job (Script with bconsole commands to verify all cloud volume parts have been uploaded. Also responsible to perform initial upload of copy job volumes (SD cloud resource specifies "Upload = no")) Message continues. Regards, Robert Gerber 402-237-8692 [email protected]
_______________________________________________ Bacula-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-users
