Re: [Bacula-users] slow job with many files?

2026-01-31 Thread Phil Stracchino 

On 1/31/26 15:25, Arno Lehmann via Bacula-users wrote:

for c in md5 sha1 sha256 sha512; do openssl speed -bytes 65536 \
-seconds 10 -evp $c 2>/dev/null | grep -E '^'$c ; done


For clarity, higher numbers here are better.

For a datapoint as a matter of interest, on my AMD Ryzen 9, sha512 is 
slightly faster than md5; sha256 roughly 2.2 times faster than sha512; 
and sha1 only perhaps 7% faster than sha256.


So a newer and much stronger hash is *not necessarily* slower than 
old-school md5, and may even be faster.  The overhead of storing the 
larger hash is, frankly, trivial.  On my hardware, the best 
bang-for-the-buck of time vs. strength is probably sha256.  sha512 is 
stronger, but *for this purpose*, probably not in any truly meaningful way.



--
  Phil Stracchino
  Fenian House Publishing
  [email protected]
  [email protected]
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958


___
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] slow job with many files?

2026-01-31 Thread Arno Lehmann via Bacula-users 

Hi all,

Am 28.01.2026 um 14:06 schrieb Stefan G. Weichinger via Bacula-users:

Am 28.01.26 um 08:58 schrieb Stefan G. Weichinger via Bacula-users:

The defs are shown in the first posting. *no* signature set. Is that 
wrong/problematic?


added md5 sig for the most jobs on this fd for now


While md5 is IMO still good for bitrot detection (which we also do with 
some CRC, but not at the file level) I'd recommend going for longer and 
more collision resistant hashes.


The downside of them taking up more catalog space is rather minimal, 
given all the other things we keep track of already. The potential 
downside of them slowing down things is a very interesting topic for 
some research :-)


Essentially, Bacula uses the openSSL functions. OpenSSL has built-in 
speed test functions. OpenSSL is also, on many platforms, highly tunded. 
On others, particularly RISC CPUs without crypto acceleration, all 
software tuning will probably be pointless.


Anyway -- try something like

for c in md5 sha1 sha256 sha512; do openssl speed -bytes 65536 \
-seconds 10 -evp $c 2>/dev/null | grep -E '^'$c ; done

on your typival clients and see what you get.

(Also, I try to solve throughput bottlenecks that are due to CPU limits 
by parallelizing backups, but that can be a challenge in itself, and in 
many cases, it's not worth the effort anyway. Most of my backups do not 
depend on particular tight backup windows, and if a backup finishes 
after 15 minutes or 15 hours is not important.)



However, to your first question: I usually very strongly recommend to 
set some signature in all of your file set options blocks. Of course 
it's not wrong to not have any, but not having them does make it more 
likely that certain errors go undetected.


Cheers,

Arno



___
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users


--
Arno Lehmann

IT-Service Lehmann
Sandstr. 6, 49080 Osnabrück



___
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] slow job with many files?

2026-01-28 Thread Stefan G. Weichinger via Bacula-users 

Am 28.01.26 um 08:58 schrieb Stefan G. Weichinger via Bacula-users:

The defs are shown in the first posting. *no* signature set. Is that 
wrong/problematic?


added md5 sig for the most jobs on this fd for now


___
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] slow job with many files?

2026-01-28 Thread Stefan G. Weichinger via Bacula-users 

Am 27.01.26 um 19:05 schrieb Bill Arlofski via Bacula-users:

On 1/27/26 9:27 AM, Stefan G. Weichinger via Bacula-users wrote:
 >

So I assume the number of files somehow kills the overall performance by
doing the database/catalog inserts, right?


Hello Stefan,

No. You have `SpoolAttributes = yes` (the default setting) explicitly 
set in your JobDefs configuration.


This means that the SD writes the attribute data to its configured 
`WorkingDirectory` during a job, and when the job finishes, it sends 
this file to the Director which does an efficient batch insert of the 
attributes to the catalog.


If you disable attribute spooling, you will see a severe drop in 
performance because the SD will be sending each file's attributes to the 
Director as they are backed up and the Director will be inserting the 
attributes for each file into the catalog one at a time.


The hypothesis with the DB is wrong, I know. I saw it yesterday, the 
load on the server is low.



 > And I see the NVMe maxxed out somehow, but with writes, not reads.

If this is on the client, it makes no sense at all. :)

During a backup, the FD is simply stat'ing, opening, reading, closing 
files. It is not writing anything, unless you are using Bacula 
Enterprise with Global Endpoint Deduplication and have set `dedup = 
bothsides` in the Fileset. :)


definitely no enterprise stuff here

I wonder if it is related to btrfs on the client: there are 
btrfs-snapshots in that pool, etc / this might somehow lead to 
write-activities ... I can't explain it fully yet.


Now without compression it's not an issue anymore, as far as I saw 
yesterday.


 > I compared the Filesets, both enabled zstd ... I now removed that and 
now the job is way faster.


Interesting, zstd is supposed to be better, faster, smaller, less CPU 
intense. Maybe the data it is trying to compress is not very 
compressible? In that case, disabling it would not show much increase in 
backup data storage use.


Maybe try with gzip and see if there is a difference?

You may also want to disable Communication Line Compression 
(CommCompression = no) in FD, SD, and Director to save some wasted CPU 
cycles.


might try, thanks

And finally, you didn't mention what `signature` setting you were using
in the Fileset(s). Take a look at that and maybe using a different one 
(less CPU intensive) may also help. This is not a security feature, so 
even md5 is OK to use. I typically use sha1, but sha256 and sha512 are 
also supported.


The defs are shown in the first posting. *no* signature set. Is that 
wrong/problematic?


thanks, Stefan



___
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] slow job with many files?

2026-01-27 Thread Bill Arlofski via Bacula-users 

On 1/27/26 9:27 AM, Stefan G. Weichinger via Bacula-users wrote:
>

So I assume the number of files somehow kills the overall performance by
doing the database/catalog inserts, right?


Hello Stefan,

No. You have `SpoolAttributes = yes` (the default setting) explicitly set in 
your JobDefs configuration.

This means that the SD writes the attribute data to its configured `WorkingDirectory` during a job, and when the job 
finishes, it sends this file to the Director which does an efficient batch insert of the attributes to the catalog.


If you disable attribute spooling, you will see a severe drop in performance because the SD will be sending each file's 
attributes to the Director as they are backed up and the Director will be inserting the attributes for each file into the 
catalog one at a time.



> I might start the slower job now and watch "top" etc on the server ... could 
have done that already ;-)

Might want to run htop, or iotop on both client and server 
during this job.



> And I see the NVMe maxxed out somehow, but with writes, not reads.

If this is on the client, it makes no sense at all. :)

During a backup, the FD is simply stat'ing, opening, reading, closing files. It is not writing anything, unless you are using 
Bacula Enterprise with Global Endpoint Deduplication and have set `dedup = bothsides` in the Fileset. :)



> I compared the Filesets, both enabled zstd ... I now removed that and now the 
job is way faster.

Interesting, zstd is supposed to be better, faster, smaller, less CPU intense. Maybe the data it is trying to compress is not 
very compressible? In that case, disabling it would not show much increase in backup data storage use.


Maybe try with gzip and see if there is a difference?

You may also want to disable Communication Line Compression (CommCompression = no) in FD, SD, and Director to save some 
wasted CPU cycles.


And finally, you didn't mention what `signature` setting you were using
in the Fileset(s). Take a look at that and maybe 
using a different one (less CPU intensive) may also help. This is not a security feature, so even md5 is OK to use. I 
typically use sha1, but sha256 and sha512 are also supported.



Best regards,
Bill

--
Bill Arlofski
[email protected]


signature.asc
Description: OpenPGP digital signature
___
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] slow job with many files?

2026-01-27 Thread Stefan G. Weichinger via Bacula-users 

Am 27.01.26 um 17:50 schrieb Jan Sielemann via Bacula-users:

Hello Stefan,

when you say it's writing: Did you use data spooling?


I posted the jobdefs:

SpoolData = no


___
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] slow job with many files?

2026-01-27 Thread Jan Sielemann via Bacula-users 

Hello Stefan,

when you say it's writing: Did you use data spooling?

Regards,

Jan


Logo JSIIT Consulting   JSIIT ConsultingPhone:  +49 40 22865465 
Jan Sielemann   Telefax:+49 40 22865468 
Riekbornweg 22  Mobile: +49 1578 6769143
22547 Hamburg 	GPG-Public-Key: 	809774813E4461D7 
 
	


:   



On 1/27/26 17:41, Stefan G. Weichinger via Bacula-users wrote:

Am 27.01.26 um 17:27 schrieb Stefan G. Weichinger via Bacula-users:

I might start the slower job now and watch "top" etc on the server 
... could have done that already ;-)


very strange.

The server seems bored.

The client is stressed! It's a rather beefy thinkpad with lots of RAM 
and CPU cores.


And I see the NVMe maxxed out somehow, but with writes, not reads.

I compared the Filesets, both enabled zstd ... I now removed that and 
now the job is way faster.


I will see if I can live with the difference in size.

thanks ...


___
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] slow job with many files?

2026-01-27 Thread Stefan G. Weichinger via Bacula-users 

Am 27.01.26 um 17:27 schrieb Stefan G. Weichinger via Bacula-users:

I might start the slower job now and watch "top" etc on the server ... 
could have done that already ;-)


very strange.

The server seems bored.

The client is stressed! It's a rather beefy thinkpad with lots of RAM 
and CPU cores.


And I see the NVMe maxxed out somehow, but with writes, not reads.

I compared the Filesets, both enabled zstd ... I now removed that and 
now the job is way faster.


I will see if I can live with the difference in size.

thanks ...


___
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users