Hi Jay, If you are going to implement checksums you need to put in a feature to allow checksums on chunks of the table. Otherwise you end up with long and often unnecessary checksum runs on large tables. The phrase ³bide a wee² comes to mind but it could be simply overwork.
We implemented checksums in Tungsten by having a ³magic table² that we recognize as containing checksum requests. The replicator logs in as a client to run the master side request and then generate an appropriate update. We intercept and convert these to checksum events using a master-side replication filter. There is a chunking feature and eventually we will be able to use algorithms other than MD5, which is a bit pokey. This seems like a better way as the checksum command itself should not be logged. Incidentally, Maatkit has a very nice feature set for table checksums. However, it depends on statement replication and therefore will not work on Drizzle. What it does is great though. It inspired our implementation. Cheers, Robert On 8/18/09 7:33 AM PDT, "Jay Pipes" <[email protected]> wrote: > Monty Taylor wrote: >> Brian Aker wrote: >>> Hi! >>> >>> What is the value of CHECKSUM TABLE? Should this have been done as an >>> I_S feature? >>> >>> I was looking through the bug reports and it looks to give out fairly >>> inconsistent, and sometimes wrong, answers. Is this being used in >>> production? >> >> It isn't - or shouldn't be. BUT - if it actually worked well, like >> Baron's checksumming thing from maatkit - then it would be a good thing >> for many people, I imagine. > > My plan was to rip the current implementation out, as it is specific > only to MyISAM, and replace with a storage-engine agnostic CHECKSUM > command, taking the form: > > CHECKSUM (ALL | TABLE table_identifier | TABLES table_list); > > with the ALL representing all tables in the server. > > This CHECKSUM command would be able to be manually run on a publisher > server (a master in MySQL world) and whenever the command hit a > subscriber (slave), the subscriber would checksum itself and, depending > on an option set up in the configuration file, would either: > > a) Abort replication immediately and stop subscriber feed > b) Continue replication but provide an email or warning of some kind > > The Google Protobuffer message format I will propose is as follows: > > message Checksum > { > required uint32 checksum; /* crc32 value */ > repeated string table; /* contains table_list or table_identifier */ > } > > The Command message will be modified as follows: > > message Command > { > ... > optional Checksum checksum_record; /* A CHECKSUM SQL command result */ > } > > NOTE: There is no need for a timestamp in the Checksum message, as it is > already available in the Command message, of which Checksum is an > encapsulted sub-message. > > This CHECKSUM command is in addition to the existing CRC32 checksum that > the command log already keeps for its segments. > > Thoughts? > > Jay > > _______________________________________________ > Mailing list: https://launchpad.net/~drizzle-discuss > Post to : [email protected] > Unsubscribe : https://launchpad.net/~drizzle-discuss > More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
