[
https://issues.apache.org/jira/browse/HDFS-16432?focusedWorklogId=719840&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-719840
]
ASF GitHub Bot logged work on HDFS-16432:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Feb/22 02:48
Start Date: 03/Feb/22 02:48
Worklog Time Spent: 10m
Work Description: jojochuang commented on pull request #3907:
URL: https://github.com/apache/hadoop/pull/3907#issuecomment-1028551793
Thanks for opening a PR.
However, I am not so sure if this is the right approach. The whole design
namenode assumes a global namespace lock and within it, data structure access
is consistent. Once you yield in the middle, there's no such guarantee and it
could run into data races.
As for the block report taking time -- is it still a problem after removing
the folded tree set structure (HDFS-13671)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 719840)
Time Spent: 1h 20m (was: 1h 10m)
> Namenode block report add yield to avoid holding write lock too long
> --------------------------------------------------------------------
>
> Key: HDFS-16432
> URL: https://issues.apache.org/jira/browse/HDFS-16432
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: qinyuren
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-01-20-15-19-28-384.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> !image-2022-01-20-15-19-28-384.png|width=683,height=132!
> In our cluster, namenode block report will held write lock for a long time if
> the storage block number more than 100000. So we want to add a yield
> mechanism in block reporting process to avoid holding write lock too long.
> # Ensure that the processing of the same block is in the same write lock.
> # Because StorageInfo.addBlock will moves the block to the head of
> blockList, so we can collect blocks that have not been reported by delimiter
> block.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
