[
https://issues.apache.org/jira/browse/HDFS-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13891378#comment-13891378
]
Haohui Mai commented on HDFS-5182:
----------------------------------
I should have said it more concretely. What I'm proposing is that the DN passes
the file descriptor to the client (e.g., using {{ioctl()}} in Linux).
It seems to me that with this approach (1) the OS takes cares about the
resource management, and (2) the client has more flexibility. The client can
access the file using the {{read()}} and {{write()}} system calls. The client,
of course, can calls {{mmap()}} of the descriptor to implement zero-copy reads
with respect to its process boundary. It can also call {{ioctl()}} and
{{madvise()}} to specify the OS buffer cache policy of the file. The additional
flexibility can be quite useful to for implementing databases on HDFS.
> BlockReaderLocal must allow zero-copy reads only when the DN believes it's
> valid
> ---------------------------------------------------------------------------------
>
> Key: HDFS-5182
> URL: https://issues.apache.org/jira/browse/HDFS-5182
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.0
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
>
> BlockReaderLocal must allow zero-copy reads only when the DN believes it's
> valid. This implies adding a new field to the response to
> REQUEST_SHORT_CIRCUIT_FDS. We also need some kind of heartbeat from the
> client to the DN, so that the DN can inform the client when the mapped region
> is no longer locked into memory.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
