Hi Dobri,
I have some progress, the backups get written in google drive,
however, the latest backup is not written, i.e. it stays locally in
the file. It only gets written when a new job starts.
Have you checked the status of the volume in Bareos? It might be that
it isn't yet safe to send the volume to the cloud.
We have a setup rclone (to Google Drive) and bareos. It is completely
different than the rclone changer you found.
If 'rclone mount' works flawlessly these days, the only thing you need
to do is write file volumes into the 'rclone mount'ed google drive
directory.
However, since this didn't work a few years ago (and maybe still does
not) we had to work around the 'rclone mount' shortcomings.
What we did is use mergerfs to create a kind of overlay filesystem
where bareos reads and writes file volumes. Behind the scenes 'rclone
mount' AND 'rclone move' puts the volumes on google drive and makes the
available for reading.
First Bareos is configured to write to file volumes. The volumes are
limited to 10G in size. The file volumes are written to a directory,
say '/bareos/volumes'
In other directory '/bareos/remote' google drive is mounted with
rclone. This is where files are read from google drive.
Finally there is a directory '/bareos/local' which is a local cache
for mergerfs.
mergerfs is mounted like this:
/bareos/remote=RO:/bareos/local=RW /bareos/volumes
/bareos/volumes is the merged mount point.
/bareos/remote=RO is rclone mounted google drive. RO means mergerfs
only reads from it.
The RO and RW options to mergerfs mean that files can only be read from
/bareos/local=RW means that new files written to /bareos/volumes are
created in /bareos/local .
Finally, there is a script which queries bareos volumes to see what
their status is. If they are 'Full' that means they won't be written to
again. This triggers the script to 'rclone move' the volume from
/bareos/volumes to google drive. (Directly to google drive, not by
copying to /bareos/remote).
The script also looks for 'Purged' volumes, which it removes from google
drive. The bareos Pool has the option 'Recycle = no' so that bareos
doesn't attempt to reuse file volumes (which are uploaded to google drive).
So bareos writes and appends to volumes to /bareos/volumes, which
mergerfs places in /bareos/local. When bareos is done writing it
changes the status of the volume to 'Full'. The script notices this and
'rclone move's the volume to google drive. (In the background the
volume switches from being in /bareos/local to /bareos/remote) If
bareos needs to read the volume, it does so through mergerfs, which find
the file in /bareos/remote and causes 'rclone mount' to download the
volume for bareos to read. Restores are so quick that I suspect 'rclone
mount' has the ability to skip around in the volume as bareos requests
(instead of downloading the whole file).
Good luck!
C.
--
You received this message because you are subscribed to the Google Groups
"bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/bareos-users/0230b11b-840d-51bd-8dd5-0f8c2068bca8%40physics.wisc.edu.
