Am 18.09.24 um 19:31 schrieb Markus Dubois:
i'm trying to backup between a directly attached (no switch,. no router in between) a client via a 10G network[ ID] Interval Transfer Bitrate[ 5] 0.00-10.00 sec 4.02 GBytes 3.45 Gbits/sec receiver
Honestly, that looks pretty awful.I just measured two virtual machines on different hosts in different networks. So the path (that probably also has some switches in between) looks like this:
VM -> Host -> Router -> Router -> Host -> VM And the result I got was [ ID] Interval Transfer Bitrate Retr[ 5] 0.00-10.04 sec 10.7 GBytes 9.16 Gbits/sec receiver
So there seems to be something wrong with your network.
the backup task runs with lz4 compression over TLS
Do you have checksums enabled in the Fileset (i.e. "Signature = md5")? Depending on the checksum used, this can severely impact performance.
but in average i get this:Full Backup Job started: 18-Sep-24 15:16 Files=6,075,920 Bytes=1,715,453,454,098 *Bytes/sec=113,171,490* Errors=2
So when I calculate the average file size on that, I get1,715,453,454,098 Bytes / 6,075,920 Files = 282,336 Bytes/File or around 282 KB per file on average. Bareos (like virtually every other file-based backup system) does not perform at line-speed for very small files. This has a lot of reasons, but basically boils down to reading and writing a lot of metadata compared to the actual data being backed up.
this seems not the "best" transfer rate.
agreed.
I'm trying to find a way to optimize this. Any hints?
1. Make your network perform properly2. Use tar or cpio redirected to /dev/null to see how fast your client could actually discover and read the files - Bareos will never be able to outperform that value 3. Check the CPU-load of bareos-fd (and probably also bareos-sd) during the backup, if the processes sit at 100% most of the time, your performance is CPU-bound in which case you simply need to reduce that to improve performance (i.e. disable compression, disable checksums or maybe remove regex expressions from your fileset).
Hope that helps! Best Regards, Andreas -- Andreas Rogge [email protected] Bareos GmbH & Co. KG Phone: +49 221-630693-86 http://www.bareos.com Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646 Komplementär: Bareos Verwaltungs-GmbH Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/318e71d4-6497-461c-86e3-2313e5a2910e%40bareos.com.
OpenPGP_signature.asc
Description: OpenPGP digital signature
