Hi Li, comments inline.

On 08/08/2014 12:03 AM, Li Ma wrote:
Getting a massive amount of information from data storage to be displayed is
where most of the activity happens in OpenStack. The two activities of reading
data and writing (creating, updating and deleting) data are fundamentally

The optimization for these two opposite database activities can be done by
physically separating the databases that service these two different
activities. All the writes go to database servers, which then replicates the
written data to the database server(s) dedicated to servicing the reads.

Currently, AFAIK, many OpenStack deployment in production try to take
advantage of MySQL (includes Percona or MariaDB) multi-master Galera cluster.
It is possible to design and implement a read/write separation schema
for such a DB cluster.

The above does not really make sense for MySQL Galera/PXC clusters *if only Galera nodes are used in the cluster*. Since Galera is synchronously replicated, there's no real point in segregating writers from readers, IMO. Better to just spread the write AND read load equally among all Galera cluster nodes.

However, if you have a Galera cluster that then slaves off to one or more standard MySQL slaves, then certainly doing writer/reader segregation could be useful, especially for directing readers of aggregate or report-type data to the read-only slaves.

Actually, OpenStack has a method for read scalability via defining
master_connection and slave_connection in configuration, but this method
lacks of flexibility due to deciding master or slave in the logical
context(code). It's not transparent for application developer.
As a result, it is not widely used in all the OpenStack projects.

So, I'd like to propose a transparent read/write separation method
for oslo.db that every project may happily takes advantage of it
without any code modification.

I've never seen a writer/reader segregation proxy or middleware piece that was properly able to send the "right" reads to the slaves. Unfortunately, determining what are the "right" reads to send to the slaves is highly application-dependent, since the application knows when it can tolerate slave lags.

Moreover, I'd like to put it in the mailing list in advance to
make sure it is acceptable for oslo.db.

I think oslo.db is not the right place for this. I believe the efforts that Mike Wilson has been doing in the "slavification" blueprints are the more appropriate place to add this slave-aware code.


I'd appreciate any comments.

Li Ma

OpenStack-dev mailing list

OpenStack-dev mailing list

Reply via email to