[
https://issues.apache.org/jira/browse/HDFS-13586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479903#comment-16479903
]
Anbang Hu commented on HDFS-13586:
----------------------------------
[~lukmajercak]'s patch can fix the following tests that complain about "All
datanodes are bad" as well (referring to daily Windows build 469):
*
[TestHdfsCryptoStreams|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs.crypto/TestHdfsCryptoStreams/]
*
[TestOverReplicatedBlocks|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs.server.blockmanagement/TestOverReplicatedBlocks/]
*
[TestBlockRecovery|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs.server.datanode/TestBlockRecovery/]
*
[TestDataNodeMetrics|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs.server.datanode/TestDataNodeMetrics/]
*
[TestStorageMover|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs.server.mover/TestStorageMover/]
*
[TestFSImage|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs.server.namenode/TestFSImage/]
*
[TestFSImageWithSnapshot|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs.server.namenode/TestFSImageWithSnapshot/]
*
[TestNamenodeCapacityReport|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs.server.namenode/TestNamenodeCapacityReport/]
*
[TestFileLengthOnClusterRestart|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs/TestFileLengthOnClusterRestart/]
*
[TestFileCreation|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs/TestFileCreation/]
*
[TestHFlush|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs/TestHFlush/]
*
[TestLeaseRecovery|https://builds.apache.org/job/hadoop-trunk-win/469/testReport/org.apache.hadoop.hdfs/TestLeaseRecovery/]
> Fsync fails on directories on Windows
> -------------------------------------
>
> Key: HDFS-13586
> URL: https://issues.apache.org/jira/browse/HDFS-13586
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode, hdfs
> Environment: JDK 1.8.0_144
> Hadoop 2.9+
> Reporter: Lukas Majercak
> Assignee: Lukas Majercak
> Priority: Critical
> Labels: Windows
> Attachments: HDFS-13586.000.patch
>
>
> HDFS-11915 added a fsync call on DataNode's rbw directory on the first
> hsync() call. IOUtils.fsync first tries to get a FileChannel on the directory
> using FileChannel.open(READ). This call fails on Windows for any directory
> and throws an AccessDeniedException, see discussion here:
> [http://mail.openjdk.java.net/pipermail/nio-dev/2015-May/003140.html].
>
> {code:java}
> java.io.IOException: Failed to sync
> E:\workspace\OSSHadoop\hadoop-hdfs-project\hadoop-hdfs\target\test\data\dfs\data\data1\current\BP-571178992-10.123.152.148-1526591934139\current\rbw
> at
> org.apache.hadoop.hdfs.server.datanode.DatanodeUtil.fsyncDirectory(DatanodeUtil.java:160)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.flushOrSync(BlockReceiver.java:430)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:807)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:873)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:291)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.nio.file.AccessDeniedException:
> E:\workspace\OSSHadoop\hadoop-hdfs-project\hadoop-hdfs\target\test\data\dfs\data\data1\current\BP-571178992-10.123.152.148-1526591934139\current\rbw
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
> at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
> at
> sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
> at java.nio.channels.FileChannel.open(FileChannel.java:287)
> at java.nio.channels.FileChannel.open(FileChannel.java:335)
> at org.apache.hadoop.io.IOUtils.fsync(IOUtils.java:405)
> at
> org.apache.hadoop.hdfs.server.datanode.FileIoProvider.dirSync(FileIoProvider.java:169)
> at
> org.apache.hadoop.hdfs.server.datanode.DatanodeUtil.fsyncDirectory(DatanodeUtil.java:158)
> ... 8 more
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
