[
https://issues.apache.org/jira/browse/HDFS-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987846#comment-15987846
]
Lukas Majercak commented on HDFS-11713:
---------------------------------------
Exception in {{TestRollingUpgrade.testRollback}} :
{code}
from
E:\workspace\Hadoop\hadoop-hdfs-project\hadoop-hdfs\target\test\data\dfs\data\data1\current\BP-107441638-10.123.116.86-1493168359567\tmp\subdir0\subdir0\blk_1073741826_1002.meta
to
E:\workspace\Hadoop\hadoop-hdfs-project\hadoop-hdfs\target\test\data\dfs\data\data1\current\BP-107441638-10.123.116.86-1493168359567\current\finalized\subdir0\subdir0\blk_1073741826_1002.meta
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.moveBlockFiles(FsDatasetImpl.java:882)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addBlock(BlockPoolSlice.java:286)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.addFinalizedBlock(FsVolumeImpl.java:799)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.finalizeReplica(FsDatasetImpl.java:1621)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.updateReplicaUnderRecovery(FsDatasetImpl.java:2526)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.updateReplicaUnderRecovery(FsDatasetImpl.java:2451)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.updateReplicaUnderRecovery(DataNode.java:2633)
at
org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$BlockRecord.updateReplicaUnderRecovery(BlockRecoveryWorker.java:87)
at
org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$BlockRecord.access$500(BlockRecoveryWorker.java:70)
at
org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$RecoveryTaskContiguous.syncBlock(BlockRecoveryWorker.java:241)
at
org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$RecoveryTaskContiguous.recover(BlockRecoveryWorker.java:153)
at
org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$1.run(BlockRecoveryWorker.java:541)
at java.lang.Thread.run(Thread.java:745)
Caused by: 183: Cannot create a file when that file already exists.
at org.apache.hadoop.io.nativeio.NativeIO.renameTo0(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO.renameTo(NativeIO.java:902)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.moveBlockFiles(FsDatasetImpl.java:880)
... 12 more
{code}
> Use MoveFileEx to allow renaming a file when the destination exists
> -------------------------------------------------------------------
>
> Key: HDFS-11713
> URL: https://issues.apache.org/jira/browse/HDFS-11713
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, native, rolling upgrades
> Affects Versions: 2.7.1, 2.7.3, 3.0.0-alpha1, 3.0.0-alpha2
> Reporter: Lukas Majercak
> Assignee: Lukas Majercak
> Labels: windows
> Fix For: 3.0.0-alpha2
>
> Attachments: HDFS-11713.001.patch
>
>
> The {{NativeIO.c#renameTo0}} currently uses {{MoveFile}} Windows system call,
> which fails when renaming a file to a destination that already exists.
> This makes the {{TestRollingUpgrade.testRollback}} test fail on Windows, as
> during that execution, a DataNode tries to rename block's meta file to a
> destination that exists.
> The proposal is to change to using {{MoveFileEx}} Windows call, and passing
> in {{MOVEFILE_REPLACE_EXISTING}} flag to force the renaming.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
