Hi Li, How are you going to make this separation transparent? I mean, generally, in a function code, you can't know in advance if the transaction will be read-only or it will contain an INSERT/UPDATE/DELETE statement. On the other hand, as a developer, you could analyze the DB queries that can be possibly issued by this function and mark the function somehow, so that oslo.db would know for which database connection the transaction should be created, but this is essentially what slave_connection option is for and how it works now.
Secondly, as you said, the key thing here is to separate reads and writes. In order to make reads fast/reduce the load on your 'writable' database, you'd move reads to asynchronous replicas. But you can't do this transparently either, as there is a lot of places in our code, in which we assume we are using the latest state of data, while asynchronous replicas might actually be a little bit out of date. So, in case of slave_connection, we use it only when it's ok for the code to work with outdated rows, i.e. *explicitly* modify the existing functions to work with slave_connection. Thanks, Roman On Fri, Aug 8, 2014 at 7:03 AM, Li Ma <skywalker.n...@gmail.com> wrote: > Getting a massive amount of information from data storage to be displayed is > where most of the activity happens in OpenStack. The two activities of reading > data and writing (creating, updating and deleting) data are fundamentally > different. > > The optimization for these two opposite database activities can be done by > physically separating the databases that service these two different > activities. All the writes go to database servers, which then replicates the > written data to the database server(s) dedicated to servicing the reads. > > Currently, AFAIK, many OpenStack deployment in production try to take > advantage of MySQL (includes Percona or MariaDB) multi-master Galera cluster. > It is possible to design and implement a read/write separation schema > for such a DB cluster. > > Actually, OpenStack has a method for read scalability via defining > master_connection and slave_connection in configuration, but this method > lacks of flexibility due to deciding master or slave in the logical > context(code). It's not transparent for application developer. > As a result, it is not widely used in all the OpenStack projects. > > So, I'd like to propose a transparent read/write separation method > for oslo.db that every project may happily takes advantage of it > without any code modification. > > Moreover, I'd like to put it in the mailing list in advance to > make sure it is acceptable for oslo.db. > > I'd appreciate any comments. > > br. > Li Ma > > > _______________________________________________ > OpenStack-dev mailing list > OpenStackemail@example.com > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStackfirstname.lastname@example.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev