On 1/24/25 14:50, Phil Stracchino wrote:
On 1/24/25 14:43, Josh Fisher via Bacula-users wrote:
On 1/24/25 13:22, Bill Arlofski via Bacula-users wrote:
On 1/24/25 10:17 AM, Phil Stracchino wrote:
this huge atomic write *also* makes it incompatible with Galera 3
clusters.)
Are you sure about that? The only thing attribute spooling is doing is
delaying execution of the SQL statements generated by the job. Why would
Galera care when the SQL was executed?
Josh,
The problem is that except on small jobs, the current implementation
of attribute spooling in Bacula's MySQL driver results in a batch
write that exceeds Galera's hard limit of 128K rows in a single
transaction.
I see, although recent versions do not have that hard row limit. The
global variable wsrep_max_ws_rows is used to set the write set row limit
in newer versions and defaults to unlimited. This makes sense, because
synchronization is "virtually" row-level anyway, regardless of
transaction size. I assume that means that cluster replication is still
occurring row by row, even during a transaction. From MariaDB 10.1.17 it
should default to unlimited write set rows and a max 16 GB write set
size (wsrep_max_ws_size).
One *fix* for this would be to modify the current attribute spooling
code so that when despooling, it loops through its cache no more than,
say, 125K records at a time. That would avoid the problem.
I saw signs in the code when I last looked at it that *appeared* to be
an attempt to set such a maximum row batch limit, but that limit
doesn't appear to actually work.
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
