Hi Jay
On 19/08/2009, at 12:33 AM, Jay Pipes wrote:
Monty Taylor wrote:
Brian Aker wrote:
What is the value of CHECKSUM TABLE? Should this have been done as
an
I_S feature?
I was looking through the bug reports and it looks to give out
fairly
inconsistent, and sometimes wrong, answers. Is this being used in
production?
It isn't - or shouldn't be. BUT - if it actually worked well, like
Baron's checksumming thing from maatkit - then it would be a good
thing
for many people, I imagine.
My plan was to rip the current implementation out, as it is specific
only to MyISAM, and replace with a storage-engine agnostic CHECKSUM
command, taking the form:
CHECKSUM (ALL | TABLE table_identifier | TABLES table_list);
with the ALL representing all tables in the server.
This CHECKSUM command would be able to be manually run on a
publisher server (a master in MySQL world) and whenever the command
hit a subscriber (slave), the subscriber would checksum itself and,
depending on an option set up in the configuration file, would either:
a) Abort replication immediately and stop subscriber feed
b) Continue replication but provide an email or warning of some kind
The Google Protobuffer message format I will propose is as follows:
message Checksum
{
required uint32 checksum; /* crc32 value */
repeated string table; /* contains table_list or table_identifier */
}
The Command message will be modified as follows:
message Command
{
...
optional Checksum checksum_record; /* A CHECKSUM SQL command result
*/
}
NOTE: There is no need for a timestamp in the Checksum message, as
it is already available in the Command message, of which Checksum is
an encapsulted sub-message.
This CHECKSUM command is in addition to the existing CRC32 checksum
that the command log already keeps for its segments.
Thoughts?
If doing this, then a proper fingerprint like SHA1 would be appropriate.
CRCs are not suitable for comparing two potentially distinct datasets,
it's entirely possible (and expected) to have them have the same CRC;
a CRC was designed to verify the integrity of a single block of data
through a transmission medium, generally a serial/modem line or in
some cases storage.
A fingerprint can be used in that case also, but a CRC cannot be used
where you have potentially distinct datasets. It causes false positives.
That said, I'm not sure it should be an internal thing. Maatkit has
tools for it, which use different calculation algorithms, sorting
methods and chunking depending on needs. There's no single one that
fits all cases.
The MyISAM use would be to simply verify table integrity after say
copying; but with replication you'd have to pinpoint a specific time
to get the exact same checksum. With chunks, you can mitigate that...
chunks also allow sync checks between servers, which is quite
important with replication.
My suggestion would be to ditch it from Drizzle; if you do want to
build something, it'd have to be a pluggable, with pretty much per
table/db settings including fingerprint/calc algorithm, chunking
options, etc; I'm not sure that makes sense, since tools like Maatkit
are around anyway.
Cheers,
Arjen.
--
Arjen Lentz, Director @ Open Query (http://openquery.com)
Exceptional Services for MySQL at a fixed budget.
Follow our blog at http://openquery.com/blog/
OurDelta: free enhanced builds for MySQL @ http://ourdelta.org
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
