[
https://issues.apache.org/jira/browse/HDFS-15431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ludun updated HDFS-15431:
-------------------------
Description:
a file with two replications and keep it opening.
first it writes to DN1 and DN2.
{code}
2020-06-23 14:22:51,379 | DEBUG | pipeline =
[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK],
DatanodeInfoWithStorage[DN2:25009,DS-7e434b35-0b10-44fa-9d3b-c3c938f1724d,DISK]]
| DataStreamer.java:1757
{code}
after DN2 restart, it writes to DN1 and DN3,
{code}
2020-06-23 14:24:04,559 | DEBUG | pipeline =
[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK],
DatanodeInfoWithStorage[DN3:25009,DS-1810c3d5-b6e8-4403-a0fc-071ea6e5489f,DISK]]
| DataStreamer.java:1757
{code}
after DN1 restart. it writes to DN3 and DN4.
{code}
2020-06-23 14:25:21,340 | DEBUG | pipeline =
[DatanodeInfoWithStorage[DN3:25009,DS-1810c3d5-b6e8-4403-a0fc-071ea6e5489f,DISK],
DatanodeInfoWithStorage[DN4:25009,DS-5fbb2232-e7c8-4186-8eb9-87a6aff86cef,DISK]]
| DataStreamer.java:1757
{code}
restart Active NameNode. then try to get the file.
NameNode return locatedblocks with DN1 and DN2. Can not obtain block Exception
occurred.
{code}
20/06/20 17:57:06 DEBUG hdfs.DFSClient: newInfo = LocatedBlocks{
fileLength=0
underConstruction=true
blocks=[LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796;
getBlockSize()=53; corrupt=false; offset=0;
locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK],
DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}]
lastLocatedBlock=LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796;
getBlockSize()=53; corrupt=false; offset=0;
locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK],
DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}
isLastBlockComplete=false}
{code}
was:
a file with two replications and keep it opening.
first it writes to DN1 and DN2.
after DN2 restart, it writes to DN1 and DN3,
after DN1 restart. it writes to DN3 and DN4.
restart Active NameNode. then try to get the file.
NameNode return locatedblocks with DN1 and DN2. Can not obtain block Exception
occurred.
{code}
20/06/20 17:57:06 DEBUG hdfs.DFSClient: newInfo = LocatedBlocks{
fileLength=0
underConstruction=true
blocks=[LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796;
getBlockSize()=53; corrupt=false; offset=0;
locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK],
DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}]
lastLocatedBlock=LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796;
getBlockSize()=53; corrupt=false; offset=0;
locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK],
DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}
isLastBlockComplete=false}
{code}
> Can not read a opening file after NameNode failover if pipeline recover
> occuered
> --------------------------------------------------------------------------------
>
> Key: HDFS-15431
> URL: https://issues.apache.org/jira/browse/HDFS-15431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Reporter: ludun
> Priority: Major
>
> a file with two replications and keep it opening.
> first it writes to DN1 and DN2.
> {code}
> 2020-06-23 14:22:51,379 | DEBUG | pipeline =
> [DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK],
>
> DatanodeInfoWithStorage[DN2:25009,DS-7e434b35-0b10-44fa-9d3b-c3c938f1724d,DISK]]
> | DataStreamer.java:1757
> {code}
> after DN2 restart, it writes to DN1 and DN3,
> {code}
> 2020-06-23 14:24:04,559 | DEBUG | pipeline =
> [DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK],
>
> DatanodeInfoWithStorage[DN3:25009,DS-1810c3d5-b6e8-4403-a0fc-071ea6e5489f,DISK]]
> | DataStreamer.java:1757
> {code}
> after DN1 restart. it writes to DN3 and DN4.
> {code}
> 2020-06-23 14:25:21,340 | DEBUG | pipeline =
> [DatanodeInfoWithStorage[DN3:25009,DS-1810c3d5-b6e8-4403-a0fc-071ea6e5489f,DISK],
>
> DatanodeInfoWithStorage[DN4:25009,DS-5fbb2232-e7c8-4186-8eb9-87a6aff86cef,DISK]]
> | DataStreamer.java:1757
> {code}
> restart Active NameNode. then try to get the file.
> NameNode return locatedblocks with DN1 and DN2. Can not obtain block
> Exception occurred.
> {code}
> 20/06/20 17:57:06 DEBUG hdfs.DFSClient: newInfo = LocatedBlocks{
> fileLength=0
> underConstruction=true
>
> blocks=[LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796;
> getBlockSize()=53; corrupt=false; offset=0;
> locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK],
>
> DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}]
>
> lastLocatedBlock=LocatedBlock{BP-1590194288-10.162.26.113-1587096223927:blk_1073895975_155796;
> getBlockSize()=53; corrupt=false; offset=0;
> locs=[DatanodeInfoWithStorage[DN1:25009,DS-1dcbe5bd-f69a-422c-bea6-a41bda773084,DISK],
>
> DatanodeInfoWithStorage[DN2:25009,DS-cd06a4f9-c25d-42ab-887b-f129707dba17,DISK]]}
> isLastBlockComplete=false}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
