Hi Padraig and Arjen, We have implemented a filtering feature in the Tungsten Replicator that can be adapted to keep only events from some databases and not others, etc. Both master and slave have a chain of filter plugins that get to look at events as they come up out of the binlog on the master as well as before they are applied in the slave. Our filtering is more general I think than what you are describing here, because instead of having a single filter that has a lot of options, you have several simple little ones instead. Also, filters can alter events, for example to add metadata or (in future) to select a channel to be used for parallel replication.
When filtering in this way, it's also important to have filters on the slave side. For example, you can use a filter to implement time-delay replication--the filter looks at the master timestamp and waits until some pre-configured time has elapsed before allowing the event to be applied on the slave. This works surprisingly well and is trivially easy to implement once the filtering mechanism is in. Anyway, I thought you might want to look at this while thinking about Drizzle. We will of course have this available for everybody once we integrate full support for Drizzle. Cheers, Robert On 7/23/09 4:57 PM PDT, "Padraig O'Sullivan" <[email protected]> wrote: > On Thu, Jul 23, 2009 at 7:39 PM, Arjen Lentz<[email protected]> wrote: >> Hi Padraig, >> >> On 24/07/2009, at 9:04 AM, Padraig O'Sullivan wrote: >>> >>> Yesterday, I created a filtered replicator plugin based on the default >>> replicator that Jay developed. It is a simple filtered replicator that >>> can currently filter events based on a schema or table name. A user >>> can specify a list of tables or schemas to filter replication events >>> by. If an event is on a schema or table that is to be filtered, then >>> the event will not be passed on to an applier by the replicator. >> >> Ehm, is that on the side of master, or slave? > > It is on the side of the master. Sorry, I should have been clearer. > >> >> As discussed previously, filtering should be done on master, not slave, e.g. >> before sending events to slave. >> This for security as well as traffic and load considerations. > > This is exactly how the filtered replicator currently works. I pushed > the branch to launchpad if you want to look at it: > > lp:~posulliv/drizzle/filtered-replicator > >> Within Drizzle, you could create a filter plugin that can have several >> filter profiles, and a slave would subscribe using one (or several?) filter >> profiles. >> >> If the current replication concept doesn't allow this, we should redesign it >> ;-) >> >> >> Cheers, >> Arjen. >> -- >> Arjen Lentz, Director @ Open Query (http://openquery.com) >> Exceptional Services for MySQL at a fixed budget. >> >> Follow our blog at http://openquery.com/blog/ >> OurDelta: free enhanced builds for MySQL @ http://ourdelta.org >> >> > > _______________________________________________ > Mailing list: https://launchpad.net/~drizzle-discuss > Post to : [email protected] > Unsubscribe : https://launchpad.net/~drizzle-discuss > More help : https://help.launchpad.net/ListHelp > -- Robert Hodges, CTO, Continuent, Inc. Email: [email protected] Mobile: +1-510-501-3728 Skype: hodgesrm _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
