Hi Mark, On 3/23/10 4:55 AM PDT, "MARK CALLAGHAN" <mdcal...@gmail.com> wrote:
> On Mon, Mar 22, 2010 at 7:52 PM, Jeremy Zawodny <jer...@zawodny.com> wrote: >> Robert... >> >> On Mon, Mar 22, 2010 at 12:43 PM, Robert Hodges >> <robert.hod...@continuent.com> wrote: >>> >>> Hi All, >>> >>> 1.) What are the "big" problems in replication? Let's say we have things >>> like availability and basic read scaling basically handled. What's next >>> on >>> the list? (Big data, No-SQL, replication/database impedance mismatch due >>> to >>> faster hardware, complex topologies, management, etc., all suggestions are >>> welcome.) >> >> Latency. Specifically I want muli-threaded replication so I can better take >> advantage of mutli-core slaves. I'm willing to assume the "risk" associated >> with saying "yes, I'm sure that each of these logical dbs are logically >> separate, and no statements will cross them." > > Paul started it, so I will add another plug. > > Are your slaves IO bound? If so, have you tried mk-slave-prefetch? I > think it can fix the replication lag problems for IO bound slaves. It > needs more people to test and use it. It can be used right now. > > I don't know if it good enough yet. We have sponsored a lot of work on > it because we think it can be good enough. > > It won't do anything for workloads with crazy and gigantic joins or > deletes/updates with full-table scans on large tables. Those problems > require education, not faster replication. > Just a follow-on about some problems that aren't solved by prefetching, because the issue is not fundamentally a problem of speeding up single-thread I/O. Also, education does not help. I'm curious how you, Jeremy, and others have addressed these in the past. 1.) Slave catch up. Loading a backup and then synchronizing is painfully slow for large data sets. This is a great place to look at parallelization if you know the data are sharded. 2.) Shared resource computing. Keeping tenants in multi-tenant systems (including both SaaS as well as ISPs) from hogging resources is a big problem. For example, I'm working on an engagement now where single tenants can do operations that induce a couple of hours of slave lag, thereby blocking replication for all other tenants as well. This problem is sufficiently widespread that you want replication to handle it through configuration rather than app changes, which are often either very costly or difficult/impossible to do. Cheers, Robert _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : drizzle-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
