[
https://issues.apache.org/jira/browse/HDFS-5050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747977#comment-13747977
]
Colin Patrick McCabe commented on HDFS-5050:
--------------------------------------------
FSDataSetSpi#cache: It would be nicer to see a separate function that caches a
single block, that is called from the main function taking an array. That
would avoid the need for 'continue' in the loop.
> These threads consume both I/O and CPU, so can affect
> normal datanode operations.
Should be "this can affect."
> public long getNumBytes() {
> return blockSize + metaSize;
> }
You aren't locking the meta file, so this should not be counted. Also don't
need metaSize itself...
> metaMapped = metaChannel.map(MapMode.READ_ONLY, metaChannel.position(),
> metaSize - metaChannel.position());
I think we should avoid mmaping the meta file here (and also avoid reading the
block file via mmap here). Rationale: when a read from an mmap region fails,
it causes a SIGBUS which normally takes down the application (the datanode in
this case). It's better to use the regular read API and get an error we can
handle. mmap will also not be substantially faster here, because this the
first (and only?) time we're reading it, so we'll take a lot of page faults
creating PTEs.
Also, we can close the fileChannel (and associated file descriptor) as soon as
we get the mmap. The mmap increments the reference count on the FD
> Add DataNode support for mlock and munlock
> ------------------------------------------
>
> Key: HDFS-5050
> URL: https://issues.apache.org/jira/browse/HDFS-5050
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Reporter: Colin Patrick McCabe
> Assignee: Andrew Wang
> Attachments: hdfs-5050-1.patch, hdfs-5050-2.patch
>
>
> Add DataNode support for mlock and munlock. The DataNodes should respond to
> RPCs telling them to mlock and munlock blocks. Blocks should be uncached
> when the NameNode asks for them to be moved or deleted. For now, we should
> cache only completed blocks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
