On 10/09/2018 06:34 AM, Florian Engelmann wrote:
Am 10/9/18 um 11:41 AM schrieb Jay Pipes:
On 10/09/2018 04:34 AM, Christian Berendt wrote:

On 8. Oct 2018, at 19:48, Jay Pipes <jaypi...@gmail.com> wrote:

Why not send all read and all write traffic to a single haproxy endpoint and just have haproxy spread all traffic across each Galera node?

Galera, after all, is multi-master synchronous replication... so it shouldn't matter which node in the Galera cluster you send traffic to.

Probably because of MySQL deadlocks in Galera:

Galera cluster has known limitations, one of them is that it uses cluster-wide optimistic locking. This may cause some transactions to rollback. With an increasing number of writeable masters, the transaction rollback rate may increase, especially if there is write contention on the same dataset. It is of course possible to retry the transaction and perhaps it will COMMIT in the retries, but this will add to the transaction latency. However, some designs are deadlock prone, e.g sequence tables.

Source: https://severalnines.com/resources/tutorials/mysql-load-balancing-haproxy-tutorial

Have you seen the above in production?

Yes of course. Just depends on the application and how high the workload gets.

Please read about deadloks and nova in the following report by Intel:


I have read the above. It's a synthetic workload analysis, which is why I asked if you'd seen this in production.

For the record, we addressed much of the contention/races mentioned in the above around scheduler resource consumption in the Ocata and Pike releases of Nova.

I'm aware that the report above identifies the quota handling code in Nova as the primary culprit of the deadlock issues but again, it's a synthetic workload that is designed to find breaking points. It doesn't represent a realistic production workload.

You can read about the deadlock issue in depth on my blog here:


That explains where the source of the problem comes from (it's the use of SELECT FOR UPDATE, which has been removed from Nova's quota-handling code in the Rocky release).

If just Nova is affected we could also create an additional HAProxy listener using all Galera nodes with round-robin for all other services?

I fail to see the point of using Galera with a single writer. At that point, why bother with Galera at all? Just use a single database node with a single slave for backup purposes.

Anyway - proxySQL would be a great extension.

I don't disagree that proxySQL is a good extension. However, it adds yet another services to the mesh that needs to be deployed, configured and maintained.


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Reply via email to