Hi all,
I'm currently deploying our brand-new BareOS infrastructure involving
both an "on-disk" storage (a ZFS box, with 96TB raw capacity, acting as
both FD ad SD) and a "tape-based" system (a dedicated linux host with 10TB
raw disks and a SAS autoloader with 2xLTO6 + 1xLTO5 drives).
After some preliminary checks to ensure that everything is properly
working, now we're trying to squeeze performances, so to shrink as much as
possible our backup timings.
Unfortunately we have one _BIG_ problem: our most critical dataset to
protect, is made up by our mail servers, storing e-mails in Maildir format.
In other words: we have 6.5TB of data made up by 25M files. So we have a
huge number of really small files.
We have already splitted the above backup in 5 sub-jobs, so to deal with
+/-1.5TB of data, each.
We have also ensured that on the network side, we have no bottleneck
(we've been able to spool a "single" huge-file, at 2.5Gbps substained over
a one-hour timeframe).
We're also working on improving "read-speed" on the ZFS side
(bareos-FD), when reading _LOTS_ of small files. We think we still have
some "margin" to cut, here.... and we're working on it.
Anyway, when writing (to tape) small-local-files (stored locally with
respect to the SD) _OR_ when despooling a backup-job made up by
small-local-files _OR_ when despooling a job with huge-files inside....
performances are relatively poor. To give you some numbers:
- JobID 101
- 9 big files for a total of 384GB coming from ZFS/FD and *spooled*
on the SD at *273 MB/sec*.
- 17-mag 16:15 srv-R630-sd JobId 101: *Despooling* elapsed time =
01:36:23, Transfer rate = *66.51 M Bytes/second*
- JobID 102
- 6M small files for a total of 1.6TB coming from ZFS/FD and *spooled*
on the SD at *28 MB/sec*.
- 18-mag 12:43 srv-R630-sd JobId 102: Despooling elapsed time =
05:53:59, Transfer rate = *76.25 M Bytes/second*
So, in the end, my question is: are there some best-practice and/or
performance-tuning activities being able to increase the tape-writing
speed? How can I increase the 60/70 MB/sec that I'm currently experiencing?
Thanks in advance and... should you need further details, please, don't
hesitate to ask.
Cheers,
DV
--
You received this message because you are subscribed to the Google Groups
"bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
For more options, visit https://groups.google.com/d/optout.
