Re: [ZODB-Dev] RelStorage: Write on master, read from slave(s)?
On 26/10/2010 11:27, Shane Hathaway wrote: On 10/26/2010 04:04 AM, Anton Stonor wrote: In order to scale an application using RelStorage I was thinking about seperating reads and writes accross databases. Writes would go to a Mysql master and the app would read from one or more Mysql slaves. If you mean that you intend to set up some clients to write to a master database, while other clients only read from asynchronous replicas, then yes, you'll be in good shape. You should be able to get amazing scalability that way. Wouldn't this require zero replication lag? Even if the read replicas had only a second or two delay (or spike delays caused by large writes?), would there not potentially be quite serious problems? cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage: Write on master, read from slave(s)?
On Mon, Nov 29, 2010 at 5:17 PM, Chris Withers ch...@simplistix.co.uk wrote: On 26/10/2010 11:27, Shane Hathaway wrote: On 10/26/2010 04:04 AM, Anton Stonor wrote: In order to scale an application using RelStorage I was thinking about seperating reads and writes accross databases. Writes would go to a Mysql master and the app would read from one or more Mysql slaves. If you mean that you intend to set up some clients to write to a master database, while other clients only read from asynchronous replicas, then yes, you'll be in good shape. You should be able to get amazing scalability that way. Wouldn't this require zero replication lag? Even if the read replicas had only a second or two delay (or spike delays caused by large writes?), would there not potentially be quite serious problems? What I think shane meant was to have clients that only read data (eg anonymous users) so they can read old data without causing problems and clients that edit content use the primary database. Seems to be a bad way to improve performance, the only advantage I would see from using cache is if you somehow have logged in users that don't edit data and have most/all of the pages dependent on the logged user, for example a forum. -- Leonardo Santagada ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage: Write on master, read from slave(s)?
On 11/29/2010 12:55 PM, Leonardo Santagada wrote: On Mon, Nov 29, 2010 at 5:17 PM, Chris Withersch...@simplistix.co.uk wrote: Wouldn't this require zero replication lag? No. Even if the read replicas had only a second or two delay (or spike delays caused by large writes?), would there not potentially be quite serious problems? No. What I think shane meant was to have clients that only read data (eg anonymous users) so they can read old data without causing problems and clients that edit content use the primary database. Correct, although old means up to a few seconds old. Seems to be a bad way to improve performance, the only advantage I would see from using cache is if you somehow have logged in users that don't edit data and have most/all of the pages dependent on the logged user, for example a forum. This strategy would be bad for a site that expects most users to log in and edit, such as Facebook or Twitter, but it is a proven strategy for forums, wikis, blogs, news sites, stores, and many other kinds of sites. It is more reliable than caching, since failed caches often lead to cascading failures. (See the recent Facebook outage.) Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] RelStorage: Write on master, read from slave(s)?
Hi, In order to scale an application using RelStorage I was thinking about seperating reads and writes accross databases. Writes would go to a Mysql master and the app would read from one or more Mysql slaves. It looks like RelStorage already keeps seperate connections for read and write (even though they are not exposed through configuration), so it is probably easy to set up. Would it work or is there a risk of getting inconsistent data? Or is that entirely up to the ability of doing synchronous replication of the underlying database? /Anton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage: Write on master, read from slave(s)?
On 10/26/2010 04:04 AM, Anton Stonor wrote: In order to scale an application using RelStorage I was thinking about seperating reads and writes accross databases. Writes would go to a Mysql master and the app would read from one or more Mysql slaves. If you mean that you intend to set up some clients to write to a master database, while other clients only read from asynchronous replicas, then yes, you'll be in good shape. You should be able to get amazing scalability that way. It looks like RelStorage already keeps seperate connections for read and write (even though they are not exposed through configuration), so it is probably easy to set up. Would it work or is there a risk of getting inconsistent data? Or is that entirely up to the ability of doing synchronous replication of the underlying database? These questions make me wonder if you're thinking of something more complicated than the plan I described above. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage: Write on master, read from slave(s)?
Hi Shane, In order to scale an application using RelStorage I was thinking about seperating reads and writes accross databases. Writes would go to a Mysql master and the app would read from one or more Mysql slaves. If you mean that you intend to set up some clients to write to a master database, while other clients only read from asynchronous replicas, then yes, you'll be in good shape. You should be able to get amazing scalability that way. That is what I want. Sounds great. My assumption was that if a client was reading from a relica before fresh data was replicated I would get old data. That was why I was talking about synchronous replication. /Anton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
