[
https://issues.apache.org/jira/browse/HDFS-9422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003986#comment-15003986
]
Kai Zheng commented on HDFS-9422:
---------------------------------
Hi [~heliangliang],
Thanks for finding and reporting the issue. I took a look at your patch and I
agree we'll need a better lock/sync scheme here, since the scan of disk may
take much longer time than previously assumed and holding of the global lock
during the time will affect other operations. However, some sync mechanism may
be still needed because otherwise the diff computation result may be out of
sync or incorrect if the *dataset* block map is changed during the time. We may
limit the lock scope or use a fine-grained lock instead.
> Unnecessary FsDatasetImpl locking in DirectoryScanner cause periodic datanode
> pauses
> ------------------------------------------------------------------------------------
>
> Key: HDFS-9422
> URL: https://issues.apache.org/jira/browse/HDFS-9422
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: He Liangliang
> Assignee: He Liangliang
> Attachments: HDFS-9422.patch
>
>
> In DirectoryScanner, the scan call hold the global dataset lock for quite
> long time (typically > 5 secs for 500k blocks). This will stuck the client
> which need acquire this lock. For applications like HBase, this will affect
> the latency. In fact, this lock is unnecessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
