I hadn’t seen maatkit before. I’ll have to look into it. It may be a good
starting point to think about data diffs.
Nathan
From: Michael Cheung [mailto:[email protected]]
Sent: Friday, June 12, 2009 10:58 AM
To: [email protected]
Cc: [email protected]
Subject: Re: [Liquibase-user] Diffing data
Hi
I currently working on a project to resolve this. The requirement was to diff
the schema and data to generate a script to apply in an env during release. We
did many research and we end up using liquibase for schema diff and maatkit for
data diff. Maatkit is not prefect and only specific for mysql. It also requires
you to pick the right algorithm (chuck, nibble, stream, groupby) for diffing. A
wrong algorithm can generate duplicate record. It requires many trail and error
to get it right.
On Jun 12, 2009, at 8:24 AM, "Voxland, Nathan"
<[email protected]> wrote:
The reason we don’t do data diffing currently is because if you are
trying to solve the problem in a general manner, you quickly run into many
difficult problems (size of dataset, what really constitutes a “different” cell
value? How should you handle data that is the same, but with different PKs?,
How do you handle data that you expect to be different? etc.)
I have heard of people dong a text database dump, then just using a
regular text diff tool between the files. You may be able to use liquibase’s
generateChangeLog to get the data out in a standardized and nicer-to-diff
format. Depending on how your data is changing, there may be a poor
signal-to-noise ratio.
I haven’t worked with content management systems like drupal much, but
I have recognized that tracking of data changes in them is a huge problem.
Does anyone else have any experience with them and have suggestions? Anything
liquibase can do for that niche in particular? Even if we do not support data
diffs in general, it may be worth creating a drupal diff extension project in
the upcoming (next 2 weeks) liquibase plug-in site.
Nathan
From: Romberg Christo [mailto:[email protected]]
Sent: Thursday, June 11, 2009 12:51 AM
To: [email protected]
Subject: [Liquibase-user] Diffing data
Hello!
On Controlling Checks (since 1.8)
(http://www.liquibase.org/manual/diff) it’s stated that not only can you diff
the database structure, but also the
data itself. However, later in that section it’s stated that the
diffTypes command is only available when using generateChangeLog, not when
using the diff or diffChangeLog commands.
Is there any way that you could get the data diffed as well (using
diffChangeLog)?
Since we’re developing in Drupal, where unconscious database changes
are a common phenomena, we would need a reliable database differentiating
system.
If the diff cannot differentiate the data between the tables, I guess
the only solution left is 1) Up Front Design/Development, 2) doing all the
base work in one database,
3) using database dumps ? Any ideas on this?
Regards,
Christo
------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
Liquibase-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/liquibase-user
------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
Liquibase-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/liquibase-user
