I've got a mail server full of millions of tiny little files (messages)
that I back up to my Bareos server at another location. It works, but it
takes several days to complete, and I've had to use LVM snapshots to get
steady, consistent backups, as the data changes fairly rapidly from hour
to hour.
In total I've got around 1.2 terabytes of data spread across 18-20
million files that needs to be backed up to eliminate any possible
conflicts (uplink bandwidth on the switch during the day, for example).
I'm trying to see if there is a way to configure Bareos so that there's
more efficient use of the bandwidth, or just find ways to speed the
whole process up. I've fiddled with setting the network buffer at the
bareos-sd level, enabling compression, enabling/disabling spooling, etc.
I'm getting10-15 MB/s at best, but looking uniquely at bandwidth alone,
I should see even better transfer speeds (network bandwidth is at least
1 Gbps along the entire path).
Bandwidth between locations isn't an issue. From what I can see, the
excess overhead of transferring so many small files is what is causing
the majority of the slowdowns. Read speed from the disk (snapshot I'm
coping is riding off of an Areca RAID card) is 10-12 Mbps.
The Bareos server is CentOS. The target system is a Ubuntu 12.04 LTS rig
(yes, I know -- planning to upgrade the OS soon).
What do you suggest I try to speed things up?
Best
Jason
--
You received this message because you are subscribed to the Google Groups
"bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
For more options, visit https://groups.google.com/d/optout.
