Eisuke Umeda created HDFS-10815:
-----------------------------------
Summary: The state of the EC file is erroneously recognized when
you restart the NameNode.
Key: HDFS-10815
URL: https://issues.apache.org/jira/browse/HDFS-10815
Project: Hadoop HDFS
Issue Type: Bug
Components: erasure-coding
Affects Versions: 3.0.0-alpha1
Environment: 2 NameNodes, 5 DataNodes, Erasured code policy is set as
"RS-DEFAULT-3-2-64k"
Reporter: Eisuke Umeda
After carrying out an examination in the following procedures, an EC files came
to be recognized as corrupt files.
These files were able to get in "hdfs dfs -get".
NameNode might be causing the false recognition.
NameNodes:
namenode1(active)
namenode2(standby)
The directory which there is "Under-erasure-coded block groups":
/tmp/tpcds-generate/test
{code}
$ sudo -u hdfs hdfs erasurecode -getPolicy /tmp/tpcds-generate/test
ErasureCodingPolicy=[Name=RS-DEFAULT-3-2-64k,
Schema=[ECSchema=[Codec=rs-default, numDataUnits=3, numParityUnits=2]],
CellSize=65536 ]
{code}
The following is the steps to reproduce:
1) hdfs dfs -cp /tmp/tpcds-generate/25/store_sales/* /tmp/tpcds-generate/test
2) datanode1: (in the middle of the copy) sudo pkill -9 -f datanode
3) start a process of datanode1 two minutes later
4) carry out hdfs fsck and confirm that Under-Replicated Blocks occurred
5) wait until Under-Replicated Blocks becomes 0
6) (namenode1) /etc/init.d/hadoop-hdfs-namenode restart
7) (namenode2) /etc/init.d/hadoop-hdfs-namenode restart
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
