[
https://issues.apache.org/jira/browse/HDFS-5957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13918542#comment-13918542
]
Todd Lipcon commented on HDFS-5957:
-----------------------------------
bq. That was attempted & we do follow that codepath for the remote reads. That
is slower than the zero copy read because to produce a direct byte buffer, the
JVM has to defragment memory & produce a contiguous large memory region - this
triggered a full GC pass, which caused stragglers with container reuse.
This doesn't make sense to me -- direct buffers are off-heap and allocating one
is really just a wrapper around Unsafe.allocateMemory(), which itself is just a
regular old malloc.
Maybe something else was going on there?
> Provide support for different mmap cache retention policies in
> ShortCircuitCache.
> ---------------------------------------------------------------------------------
>
> Key: HDFS-5957
> URL: https://issues.apache.org/jira/browse/HDFS-5957
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 2.3.0
> Reporter: Chris Nauroth
>
> Currently, the {{ShortCircuitCache}} retains {{mmap}} regions for reuse by
> multiple reads of the same block or by multiple threads. The eventual
> {{munmap}} executes on a background thread after an expiration period. Some
> client usage patterns would prefer strict bounds on this cache and
> deterministic cleanup by calling {{munmap}}. This issue proposes additional
> support for different caching policies that better fit these usage patterns.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
