- one problem with the "Build System B and migrate to it" approach is
that you might build system B from scratch and think that it's really
great, but discover that the migration isn't really possible once you
try it.
+1......... You never really know the gremlins that live deep in that
database that make ETL or migration a complete fiasco until you run your
migration scripts several hundred times trying to get all your data to
look right to the end users. If it was a custom app, you never know when
some programmer, or several personalities, a couple of years ago decided
to change the date format and has included a patch in his code to deal
with the two or more formats but didn't tell you. You go to key off
that date and you find your scripts inexplicably failing. You look
deeper and find 6 different date formats in the same column. Who would
have guessed we store dates in 6 different ways? Hmm.... then you
discover that not all accounts have all information because columns were
added as time went on and requirements grew.
It seems so easy to move data from flat files (access) to a relational
database in concept. In a trivial system it might be easy, however the
cost associated do not grow linearly. It seems to grow much faster then
the perceived complexity of the problem, ask Airbus. They just had some
wires that were the wrong length.
Then you start to try to take advantage of RDMS and you find that
incomplete data or data that is not keyed properly doesn't move
gracefully. You find that the original design of the flat files falls
short and needs to changed to support the new app. So you start to
transform the data to fit the new schema and you find that keys you
think would be no brainers don't exist or are incomplete.
On top of that, you design all these new data structures and classes to
do really neat things. Your very proud of yourself and the way the
application behaves with test data. Then you shoe horn the old data in
and your application looks more like a cubist painting then a crisp
photo from a Canon 20D with an L lens. Then you spend some more time
massaging the data. In the end it is beautiful, but it requires a high
level of expertise and the constitution of Conan.
I speak from direct personal experience working with data collected over
the course of 20 years. It is not impossible, I know, we are
successfully doing it here. I happen to be blessed with working with a
very talented team of people. But you have to be very very aware that
there are things that you don't know that you don't know. Ha ha ha....
It is a very painful lesson to stand in front of the CEO and CFO and
explain that you just spent 160 man hours working on how to retrieve and
correctly key broken data from the old system. Data everyone thought
was locked down until someone in sales says they can't find some over
looked piece of data when he is testing. Your story may not be the
same, but your feelings will be.
Hans Kaspersetz
NJ Web Development by http://www.cyberxdesigns.com
_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk
NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com
Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php
