Hi Li, comments inline.
On 08/08/2014 12:03 AM, Li Ma wrote:
Getting a massive amount of information from data storage to be displayed is
where most of the activity happens in OpenStack. The two activities of reading
data and writing (creating, updating and deleting) data are fundamentally
different.
The optimization for these two opposite database activities can be done by
physically separating the databases that service these two different
activities. All the writes go to database servers, which then replicates the
written data to the database server(s) dedicated to servicing the reads.
Currently, AFAIK, many OpenStack deployment in production try to take
advantage of MySQL (includes Percona or MariaDB) multi-master Galera cluster.
It is possible to design and implement a read/write separation schema
for such a DB cluster.
The above does not really make sense for MySQL Galera/PXC clusters *if
only Galera nodes are used in the cluster*. Since Galera is
synchronously replicated, there's no real point in segregating writers
from readers, IMO. Better to just spread the write AND read load equally
among all Galera cluster nodes.
However, if you have a Galera cluster that then slaves off to one or
more standard MySQL slaves, then certainly doing writer/reader
segregation could be useful, especially for directing readers of
aggregate or report-type data to the read-only slaves.
Actually, OpenStack has a method for read scalability via defining
master_connection and slave_connection in configuration, but this method
lacks of flexibility due to deciding master or slave in the logical
context(code). It's not transparent for application developer.
As a result, it is not widely used in all the OpenStack projects.
So, I'd like to propose a transparent read/write separation method
for oslo.db that every project may happily takes advantage of it
without any code modification.
I've never seen a writer/reader segregation proxy or middleware piece
that was properly able to send the "right" reads to the slaves.
Unfortunately, determining what are the "right" reads to send to the
slaves is highly application-dependent, since the application knows when
it can tolerate slave lags.
Moreover, I'd like to put it in the mailing list in advance to
make sure it is acceptable for oslo.db.
I think oslo.db is not the right place for this. I believe the efforts
that Mike Wilson has been doing in the "slavification" blueprints are
the more appropriate place to add this slave-aware code.
Best,
-jay
I'd appreciate any comments.
br.
Li Ma
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
