[
https://issues.apache.org/jira/browse/HDFS-15000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160540#comment-17160540
]
dmichal commented on HDFS-15000:
--------------------------------
I have one more idea about how this issue could be solved. In addition to
global locks (shared for all blocks), block-specific locks can be introduced.
In such a case {{FsDatasetImpl.createRbw()}} could work as follows:
{code:java}
block-specific write lock {
global lock {
1. check if block_id exists
2. other checks
3. ...
}
4. perform the IO
global lock {
5. update the volume map
}
}
{code}
Then the race conditions mentioned by [~sodonnell] in [~Aiphag0]'s solution
won't occur, since no other thread attempting to access the same block will be
allowed.
> Improve FsDatasetImpl to avoid IO operation in datasetLock
> ----------------------------------------------------------
>
> Key: HDFS-15000
> URL: https://issues.apache.org/jira/browse/HDFS-15000
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Xiaoqiao He
> Assignee: Aiphago
> Priority: Major
> Attachments: HDFS-15000.001.patch
>
>
> As HDFS-14997 mentioned, some methods in #FsDatasetImpl such as
> #finalizeBlock, #finalizeReplica, #createRbw includes IO operation in the
> datasetLock, It will block some logic when IO load is very high. We should
> reduce grain fineness or move IO operation out of datasetLock.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
