[
https://issues.apache.org/jira/browse/HDFS-12971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
muyannian updated HDFS-12971:
-----------------------------
Description:
when i used hdfs hedged read , i found DFSInputStream hang on
getFirstToComplete method.
the reason is when something has exception on datanode,or namenode , for
example FileNotFoundException, that may cause hang up for ever. future has
finished but still call get "future = hedgedService.take()" method ,so cause
hang up.
in the attatch file is my jstack and java patch.
<property>
<name>dfs.client.hedged.read.threadpool.size</name>
<value>512</value>
</property>
<property>
<name>dfs.client.hedged.read.threshold.millis</name>
<value>300</value>
</property>
was:
when i used hdfs hedged read , i found DFSInputStream hang on
getFirstToComplete method.
the reason is when something has exception on datanode,or namenode , for
example FileNotFoundException, that may cause hang up for ever. future has
finished but still call get "future = hedgedService.take()" method ,so cause
hang up.
in the attatch file is my jstack and java patch.
> DfsClint hang on hedged getFromOneDataNode
> ------------------------------------------
>
> Key: HDFS-12971
> URL: https://issues.apache.org/jira/browse/HDFS-12971
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, hdfs-client
> Affects Versions: 2.6.0, 2.7.0
> Environment: hadoop 2.6.0 (dfs.client.hedged.read.threadpool.size>0)
> Reporter: muyannian
> Attachments: 1.jpg, 2.jpg, 3.jpg, 4.jpg,
> DFSInputStream-afterpatch.java, DFSInputStream-beforepatch.java,
> DFSInputStream.java.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> when i used hdfs hedged read , i found DFSInputStream hang on
> getFirstToComplete method.
> the reason is when something has exception on datanode,or namenode , for
> example FileNotFoundException, that may cause hang up for ever. future has
> finished but still call get "future = hedgedService.take()" method ,so cause
> hang up.
> in the attatch file is my jstack and java patch.
> <property>
> <name>dfs.client.hedged.read.threadpool.size</name>
> <value>512</value>
> </property>
>
> <property>
> <name>dfs.client.hedged.read.threshold.millis</name>
> <value>300</value>
> </property>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
