Filed HADOOP-5798.
On Wed, May 6, 2009 at 9:53 PM, Raghu Angadi rang...@yahoo-inc.com wrote:
Tamir Kamara wrote:
Hi Raghu,
The thread you posted is my original post written when this problem first
happened on my cluster. I can file a JIRA but I wouldn't be able to
provide
information
Hi.
Yes, this was probably it.
The strangest part, that the HDFS somehow worked even with all files empty
in the NN directory.
Go figure...
Regards.
2009/5/5 Raghu Angadi rang...@yahoo-inc.com
the image is stored in two files : fsimage and edits
(under namenode-directory/current/).
Tamir Kamara wrote:
Hi Raghu,
The thread you posted is my original post written when this problem first
happened on my cluster. I can file a JIRA but I wouldn't be able to provide
information other than what I already posted and I don't have the logs from
that time. Should I still file ?
yes.
Hi.
This quite worry-some issue.
Can anyone advice on this? I'm really concerned it could appear in
production, and cause a huge data loss.
Is there any way to recover from this?
Regards.
2009/5/5 Tamir Kamara tamirkam...@gmail.com
I didn't have a space problem which led to it (I think).
Stas,
This is indeed a serious issue.
Did you happen to store the the corrupt image? Can this be reproduced
using the image?
Usually you can recover manually from a corrupt or truncated image. But
more importantly we want to find how it got in to this state.
Raghu.
Stas Oskin wrote:
Tamir,
Please file a jira on the problem you are seeing with 'saveLeases'. In
the past there have been multiple fixes in this area (HADOOP-3418,
HADOOP-3724, and more mentioned in HADOOP-3724).
Also refer the thread you started
Hi Raghu.
The only lead I have, is that my root mount has filled-up completely.
This in itself should not have caused the metadata corruption, as it has
been stored on another mount point, which had plenty of space.
But perhaps the fact that NameNode/SecNameNode didn't have enough space for
Actually, we discovered today an annoying bug in our test-app, which might
have moved some of the HDFS files to the cluster, including the metadata
files.
I presume it could be the possible reason for such behavior? :)
2009/5/5 Stas Oskin stas.os...@gmail.com
Hi Raghu.
The only lead I have,
Stas Oskin wrote:
Actually, we discovered today an annoying bug in our test-app, which might
have moved some of the HDFS files to the cluster, including the metadata
files.
oops! presumably it could have removed the image file itself.
I presume it could be the possible reason for such
Hi.
2009/5/5 Raghu Angadi rang...@yahoo-inc.com
Stas Oskin wrote:
Actually, we discovered today an annoying bug in our test-app, which might
have moved some of the HDFS files to the cluster, including the metadata
files.
oops! presumably it could have removed the image file itself.
I
the image is stored in two files : fsimage and edits
(under namenode-directory/current/).
Stas Oskin wrote:
Well, it definitely caused the SecondaryNameNode to crash, and also seems to
have triggered some strange issues today as well.
By the way, how the image file is named?
Hi Raghu,
The thread you posted is my original post written when this problem first
happened on my cluster. I can file a JIRA but I wouldn't be able to provide
information other than what I already posted and I don't have the logs from
that time. Should I still file ?
Thanks,
Tamir
On Tue, May
Hi.
After rebooting the NameNode server, I found out the NameNode doesn't start
anymore.
The logs contained this error:
FSNamesystem initialization failed
I suspected filesystem corruption, so I tried to recover from
SecondaryNameNode. Problem is, it was completely empty!
I had an issue that
I didn't have a space problem which led to it (I think). The corruption
started after I bounced the cluster.
At the time, I tried to investigate what led to the corruption but didn't
find anything useful in the logs besides this line:
saveLeases found path
14 matches
Mail list logo
