[
https://issues.apache.org/jira/browse/HDFS-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183662#comment-14183662
]
Xiaoyu Yao commented on HDFS-6934:
----------------------------------
Thanks [~cnauroth] for working on this. The fix looks good to me. Below are
some of my comments:
1. FSOutputSummer.java
If FSOutputSummer#getChecksumSize() is added to reduce individual call
sum.getChecksumSize() in FSOutputSummer, there are still three
Sum.getChecksumSize() calls that can be replaced with getChecksumSize().
2. DataChecksum.java
NIT: Can you add spaces between operators of * and + in
DataChecksum#getChecksumSize().
3. BlockReceiver.java
If you put Line 582~583 before Line 574, you can use {code} If
(shouldNotWriteChecksum) {code} instead of duplicating the same logic.
4. DFSInputStream.java
Line 954: This can be moved out of the loop
StorageType[] storageTypes = block.getStorageTypes();
Line 955: Can you add comment on the relationship between index of nodes (i)
to the storage type selection
if (storageTypes != null && i < storageTypes.length) {
storageType = storageTypes[i];
}
5. TestLazyPersistFiles.java
We have a separate test class for SCR related tests in
TestScrLazyPersistFiles.java so that the temporary domain socket does not need
to be created for non-SCR cases. Can put the new SCR cases there?
I notice the test timeout attribute is removed. Is there a particular reason
for that?
DFS_DATANODE_RAM_DISK_LOW_WATERMARK_REPLICAS has been changed to
DFS_DATANODE_RAM_DISK_LOW_WATERMARK_BYTES by HDFS-6988, can you sync with the
trunk and update the patch?
> Move checksum computation off the hot path when writing to RAM disk
> -------------------------------------------------------------------
>
> Key: HDFS-6934
> URL: https://issues.apache.org/jira/browse/HDFS-6934
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Reporter: Arpit Agarwal
> Assignee: Chris Nauroth
> Attachments: HDFS-6934.3.patch, h6934_20141003b.patch,
> h6934_20141005.patch
>
>
> Since local RAM is considered reliable we can avoid writing checksums on the
> hot path when replicas are being written to a local RAM disk.
> The checksum can be computed by the lazy writer when moving replicas to disk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
