[
https://issues.apache.org/jira/browse/HDFS-13586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lukas Majercak updated HDFS-13586:
----------------------------------
Description:
HDFS-11915 added a fsync call on DataNode's rbw directory on the first hsync()
call. IOUtils.fsync first tries to get a FileChannel on the directory using
FileChannel.open(READ). This call fails on Windows for any directory and throws
an AccessDeniedException, see discussion here:
[http://mail.openjdk.java.net/pipermail/nio-dev/2015-May/003140.html].
2018-05-17 14:09:32,089 [DataXceiver for client
DFSClient_NONMAPREDUCE_149875897_1 at /127.0.0.1:12645 [Receiving block
BP-1245968727-10.123.152.148-1526591311603:blk_1073741825_1001]] ERROR
datanode.DataNode (DataXceiver.java:run(324)) - 127.0.0.1:12591:DataXceiver
error processing WRITE_BLOCK operation src: /127.0.0.1:12645 dst:
/127.0.0.1:12591
java.io.IOException: Failed to sync
E:\workspace\OSSHadoop\hadoop-hdfs-project\hadoop-hdfs\target\test\data\dfs\data\data1\current\BP-1245968727-10.123.152.148-1526591311603\current\rbw
at
org.apache.hadoop.hdfs.server.datanode.DatanodeUtil.fsyncDirectory(DatanodeUtil.java:163)
at
org.apache.hadoop.hdfs.server.datanode.BlockReceiver.flushOrSync(BlockReceiver.java:430)
at
org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:807)
at
org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:873)
at
org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166)
at
org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:291)
was:HDFS-11915 added a fsync call on DataNode's rbw directory on the first
hsync() call. IOUtils.fsync first tries to get a FileChannel on the directory
using FileChannel.open(READ). This call fails on Windows for any directory and
throws an AccessDeniedException, see discussion here:
[http://mail.openjdk.java.net/pipermail/nio-dev/2015-May/003140.html].
> Fsync fails on directories on Windows
> -------------------------------------
>
> Key: HDFS-13586
> URL: https://issues.apache.org/jira/browse/HDFS-13586
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode, hdfs
> Reporter: Lukas Majercak
> Assignee: Lukas Majercak
> Priority: Critical
>
> HDFS-11915 added a fsync call on DataNode's rbw directory on the first
> hsync() call. IOUtils.fsync first tries to get a FileChannel on the directory
> using FileChannel.open(READ). This call fails on Windows for any directory
> and throws an AccessDeniedException, see discussion here:
> [http://mail.openjdk.java.net/pipermail/nio-dev/2015-May/003140.html].
>
>
> 2018-05-17 14:09:32,089 [DataXceiver for client
> DFSClient_NONMAPREDUCE_149875897_1 at /127.0.0.1:12645 [Receiving block
> BP-1245968727-10.123.152.148-1526591311603:blk_1073741825_1001]] ERROR
> datanode.DataNode (DataXceiver.java:run(324)) - 127.0.0.1:12591:DataXceiver
> error processing WRITE_BLOCK operation src: /127.0.0.1:12645 dst:
> /127.0.0.1:12591
> java.io.IOException: Failed to sync
> E:\workspace\OSSHadoop\hadoop-hdfs-project\hadoop-hdfs\target\test\data\dfs\data\data1\current\BP-1245968727-10.123.152.148-1526591311603\current\rbw
> at
> org.apache.hadoop.hdfs.server.datanode.DatanodeUtil.fsyncDirectory(DatanodeUtil.java:163)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.flushOrSync(BlockReceiver.java:430)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:807)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:873)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:291)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
