Am 19.09.24 um 13:06 schrieb Markus Dubois:
the iperf looks not "at its best" because the bareos director and the storage daemon are installed on a NAS system with an Intel Atom 2,1 Ghz 4core CPU
Maybe that system simply cannot fully utilize 10 GBe.Did you verify that the NAS system can actually handle a sustained data-stream of 300+ MB/s?
On backup server CPU is on 30-50% overall
Overall as in 30-50% of all four cores?Because that could mean one core is busy with bareos-sd and it simply cannot work faster. Also when bareos-dir and the postgresql database are on the same system, you may run out of resources (CPU, memory or IOPS).
as mentioned above, i've launched a job without tls and lz4, the signature is configured as XXH128 I can see that the NAS is reporting 250 MB/s in average and 300 MB/s max during job runtime. But i have also, how to say this properly, speed crashes from 250 MB/s to 80 MB/s. The network rate is not constantly high.So basically your maximum throughput is in the ballpark of what iperf shows. That's not bad.
Depending on the structure of files you're backing up, the drop to 80 MB/s might be normal. If - for example - you have a set of large files, these will be backed up around 250 MB/s. But as soon as the FD gets to a directory with a lot of really small files, the rates will drop. Even if the FD could keep up with it, the SD will send the file metadata for every file to the director which will have to put it into the database. Maybe it is a good idea to run a backup with (almost) only large files (i.e. some video archive) and one with lots and lots of small files, you might see a pattern evolving. If it really is the database inserts, you can enable "Spool Attributes" on the job. This will spool the metadata and only insert it to the database when the job is finished.
the bareos FD is running on a 30 Core 192GB RAM server, i can see 75% CPU of one core usage. Overall server (to be backued up) is on 34 % CPUSo I guess we can rule out signatures and compression then, as that's happening on the FD.
so, what do you mean with "network perform properly"? As mentioned, i have some hardware limitations on backup server side, but i think even those limitations shouldn't lead to those numbers....I really don't know. The only thing I can say for sure is that when iperf reports only around 3 GBit/s, you definitely won't see a backup transmitting more than 300 MB/s.Should i fiddle around with systemctl settings or max nettwork buffers?
-- Andreas Rogge [email protected] Bareos GmbH & Co. KG Phone: +49 221-630693-86 http://www.bareos.com Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646 Komplementär: Bareos Verwaltungs-GmbH Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/1caeac60-662d-4136-8d52-5f02bdcd3ec8%40bareos.com.
OpenPGP_signature.asc
Description: OpenPGP digital signature
