[
https://issues.apache.org/jira/browse/HDFS-5050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HDFS-5050:
------------------------------
Attachment: hdfs-5050-1.patch
Patch attached.
Big picture:
- New {{FsDatasetCache}} class manages a map of {{MmapBlock}}, which is a
{{Block}} with mmap, mlock, and checksum operations. The expensive operation of
adding a new block to the cache happens asynchronously via a set of per-volume
worker pools. It's modeled after {{FsDatasetAsyncDiskService}}.
- Plumbing to hook up heartbeat responses to the {{FsDatasetCache}}
- Reasonable tests with a spoofed NN, we're getting close!
Small picture:
- Refactored FsDatasetAsyncDiskService since I'd rather key my thread pools off
of the {{FsVolumeSpi#getBasePath}} string than the File from
{{FsVolumeImpl#getCurrentDir}}. This isn't strictly necessary, but I think it's
a nice change.
- I tried making FsDatasetCache lock-free, so a careful eye toward concurrency
issues would be appreciated.
Known todos:
- Statistics regarding # of cache attempts, successes, failures, and making
sure the FsDatasetMBean stats are getting exposed. Tests for these too.
- More tests for FsDatasetCache, not hitting all the failure modes.
> Add DataNode support for mlock and munlock
> ------------------------------------------
>
> Key: HDFS-5050
> URL: https://issues.apache.org/jira/browse/HDFS-5050
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Reporter: Colin Patrick McCabe
> Assignee: Andrew Wang
> Attachments: hdfs-5050-1.patch
>
>
> Add DataNode support for mlock and munlock. The DataNodes should respond to
> RPCs telling them to mlock and munlock blocks. Blocks should be uncached
> when the NameNode asks for them to be moved or deleted. For now, we should
> cache only completed blocks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
