[
https://issues.apache.org/jira/browse/HDFS-7112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159672#comment-14159672
]
Arpit Agarwal commented on HDFS-7112:
-------------------------------------
Thanks for taking this up Xiaoyu! The overall approach looks good to me. Few
comments below:
# FsDatasetImpl.java: We should avoid adding volumes to asyncDiskService unless
there is a RAM_DISK volume configured. This avoids creating unnecessary thread
pools on most deployments which will not have RAM_DISK.
# We don't need the counter and logging in {{addExecutorForVolume}}. We will
never add more than one thread per volume.
# {{onStartLazyPersist}} should be called by {{saveNextReplica}} before it
calls {{submitLazyPersistTask}}.
# {{onFailLazyPersist}} should never be called unless {{onStartLazyPersist}}
has been called. I think it can be called on some failure paths even if
submitLazyPersistTask was not called.
# {{BlockPoolSlice#lazyPersistReplica}} is unused, can be removed.
# {{RamDiskAsyncLazyPersistService#countPendingTasks}} is unused.
# It would be good if RamDiskAsyncLazyPersistService did not have a dependency
on DataNode/FsDatasetImpl. It can accept the success and failure callbacks as
parameters. But it's okay to fix it later in a separate Jira.
> LazyWriter should use either async IO or one thread per physical disk
> ---------------------------------------------------------------------
>
> Key: HDFS-7112
> URL: https://issues.apache.org/jira/browse/HDFS-7112
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: HDFS-6581
> Reporter: Arpit Agarwal
> Assignee: Xiaoyu Yao
> Fix For: 2.6.0
>
> Attachments: HDFS-7112.0.patch
>
>
> The LazyWriter currently uses synchronous IO and a single thread. This limits
> the throughput to that of a single disk. Using either async overlapped IO or
> one thread per physical disk will improve the write throughput.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
