Hi Stefan,
I'm trying to investigate the problem.
Could you give some more information about what happened to you.
- what is your cluster size
- amount of data
- how long did dfs ran without restarting the name node before upgrading
we discussed the three kinds of data looses, hardware, software or
human errors.
I would love to figure out what was my problem today. :)
Looks like you are not alone :-(
Scenario:
+ updated from hadoop .2.1 to .4.
+ problems to get all datanodes started
what was the problem with datanodes?
+ downgrade to hadoop .3.1
+ error message of incompatible dfs (I guess . already had started to
write to the log)
What is the message?
All transaction done with .2 during last hours lost. Means the data I
created and copy was not in the dfs any more.
I guess the update / downgrade process destroyed the log but the
image itself was still ok.
I ended up with a complete corrupted dfs - I guess since the lost dfs
namenode transaction log.
Wouldn't be better in case we discover a version conflict in the dfs,
that the user need to do manually confirm that the data have to
converted into a new format.
Any thoughts?
Thanks,
Konstantin
