[
https://issues.apache.org/jira/browse/HDFS-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065288#comment-14065288
]
Colin Patrick McCabe commented on HDFS-6695:
--------------------------------------------
bq. I did not look through AsynchronousfileChannel impl, but per the above link
saying: " This class also defines read and write methods that initiate
asynchronous operations, returning a Future to represent the pending result of
the operation. The Future may be used to check if the operation has completed,
wait for its completion, and retrieve the result. ", seems does the same idea
like my pseudo code...
Yes, we could definitely set up a thread pool ourselves, but we don't want to
do that because it would be slow. Passing large amounts of data between
threads can mean moving it between CPU cores, which really kills performance.
It also introduces latency from the context switches. I was hoping that Java7
had support for the kernel's native AIO interface... I guess we'll have to see.
We might be able to do something interesting by having a thread pool where
certain threads were locked to certain physical CPU cores. Of course, that
requires JNI to pull off, at the moment, and the overheads of that might wipe
out any gain...
> Investigate using Java 7's nonblocking file I/O in BlockReaderLocal to
> implement read timeouts
> ----------------------------------------------------------------------------------------------
>
> Key: HDFS-6695
> URL: https://issues.apache.org/jira/browse/HDFS-6695
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Colin Patrick McCabe
>
> In BlockReaderLocal, the "read" system call could block for a long time if
> the disk drive is having problems, or there is a huge amount of I/O
> contention. This might cause poor latency performance.
> In the remote block readers, we have implemented a read timeout, but we don't
> have one for the local block reader, since {{FileChannel#read}} doesn't
> support this.
> Once we move to JDK 7, we should investigate the {{java.nio.file}}
> nonblocking file I/O package to see if it could be used to implement read
> timeouts.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
