[
https://issues.apache.org/jira/browse/HDFS-14111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17186020#comment-17186020
]
Mania Abdi edited comment on HDFS-14111 at 8/27/20, 5:56 PM:
-------------------------------------------------------------
Hi
[~tlipcon] and
[~stakiar] I am researching on debugging distributed system, I am building a
tool that allows developers to find the frequent processing patterns within the
workflow graphs of applications. This bug seems like a case study for my
proposed research. Would it be possible to let me know what was the process of
catching this bugs for you? and how can I reproduce this bug, what was the
test that leads to this bug?
Regards
Mania
was (Author: maniaabdi):
Hi
[~tlipcon] and
[~stakiar] I am researching on debugging distributed system, I am building a
tool that allows developers to find the frequent processing patterns within the
workflow graphs of applications. This bug seems like a case study for my
proposed research. Would it be possible to let me know what was the process of
catching this bugs for you? and how can I reproduce this bug?
Regards
Mania
> hdfsOpenFile on HDFS causes unnecessary IO from file offset 0
> -------------------------------------------------------------
>
> Key: HDFS-14111
> URL: https://issues.apache.org/jira/browse/HDFS-14111
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client, libhdfs
> Affects Versions: 3.2.0
> Reporter: Todd Lipcon
> Assignee: Sahil Takiar
> Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-14111.001.patch, HDFS-14111.002.patch,
> HDFS-14111.003.patch
>
>
> hdfsOpenFile() calls readDirect() with a 0-length argument in order to check
> whether the underlying stream supports bytebuffer reads. With DFSInputStream,
> the read(0) isn't short circuited, and results in the DFSClient opening a
> block reader. In the case of a remote block, the block reader will actually
> issue a read of the whole block, causing the datanode to perform unnecessary
> IO and network transfers in order to fill up the client's TCP buffers. This
> causes performance degradation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
