Summarizing what I've learned. 1. This problem ONLY affects Bacula Director 9.6.5. Running a 9.6.5 Director with every other daemon 9.6.3 is sufficient to trigger the bug. Running a 9.6.3 Director with everything else 9.6.5 is sufficient to prevent it.
2. With Director 9.6.5, the hung-job bug occurs regardless of whether Galera auto-increment control is in use, even if Bacula is connecting directly to a single node without using HAproxy, even if the cluster is brought down to a single node (i.e. Galera not active). With Director 9.6.3, the bug does NOT occur even when Bacula DB connections are load-balanced across the cluster using HAproxy, even if auto-increment control is off. 3. The bug occurs with 9.6.5, and not with 9.6.3, using the exact same *build* of MariaDB. To me, this strongly points to a bug introduced into the Director or the MySQL driver between 9.6.3 and 9.6.5. However, I have looked at the code changes from 9.6.3 to 9.6.5, but have not been able to spot a "smoking gun". I don't see a single change that looks to me as though it would have an impact. I ran for a couple of weeks with dird 9.6.5 with every possible measure to prevent DB conflicts and still got hung jobs. Then I rolled ONLY the Director back to 9.6.3 again and rolled back the DB connection config to what I'd been using for the past [mumble] years, and have not seen a single hung job in the last 12 days. I'm getting nowhere trying to narrow this down any further. If anyone who knows and understands the codebase better than I do would care to look at the code diffs from 9.6.3 to 9.6.5 in the context of the stack traces I've been able to provide, it would be much appreciated. I've done all I can to troubleshoot this and I am unable to point a finger to any particular piece of code, but the evidence that this is a regression in Director 9.6.5 looks pretty damning to me. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
