[
https://issues.apache.org/jira/browse/HDFS-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuanbo Liu updated HDFS-16433:
------------------------------
Description:
The code in IncrementalBlockReportManager.java
{code:java}
synchronized void waitTillNextIBR(long waitTime) {
if (waitTime > 0 && !sendImmediately()) {
try {
wait(ibrInterval > 0 && ibrInterval < waitTime? ibrInterval: waitTime);
} catch (InterruptedException ie) {
LOG.warn(getClass().getSimpleName() + " interrupted");
}
}
} {code}
We can see that wait(waitime) will hold synchronized, if ibr interval is
enabled or heartbeat time is not reached. Then the lock will block
notifyNamenodeBlock function which is widely used in
deleting/creating/finalizing/opening blocks, then the speed of DataNode IO will
slow down. Here is the graph of blocking relationship
!image-2022-01-21-22-52-52-912.png|width=976,height=299!
was:
The code in IncrementalBlockReportManager.java
{code:java}
synchronized void waitTillNextIBR(long waitTime) {
if (waitTime > 0 && !sendImmediately()) {
try {
wait(ibrInterval > 0 && ibrInterval < waitTime? ibrInterval: waitTime);
} catch (InterruptedException ie) {
LOG.warn(getClass().getSimpleName() + " interrupted");
}
}
} {code}
We can see that wait(waitime) will hold synchronized, if ibr interval is
enabled or heartbeat time is not reached. Then the lock will block
notifyNamenodeBlock function which is widely used in
deleting/creating/finalizing/opening blocks, then the speed of DataNode IO will
slow down. Here is the graph of blocking relationship
!image-2022-01-21-22-52-52-912.png!
> The synchronized lock of IncrementalBlockReportManager will slow down the
> speed of creating/deleting/finalizing block
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-16433
> URL: https://issues.apache.org/jira/browse/HDFS-16433
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Yuanbo Liu
> Priority: Critical
> Attachments: image-2022-01-21-22-52-52-912.png
>
>
> The code in IncrementalBlockReportManager.java
> {code:java}
> synchronized void waitTillNextIBR(long waitTime) {
> if (waitTime > 0 && !sendImmediately()) {
> try {
> wait(ibrInterval > 0 && ibrInterval < waitTime? ibrInterval: waitTime);
> } catch (InterruptedException ie) {
> LOG.warn(getClass().getSimpleName() + " interrupted");
> }
> }
> } {code}
> We can see that wait(waitime) will hold synchronized, if ibr interval is
> enabled or heartbeat time is not reached. Then the lock will block
> notifyNamenodeBlock function which is widely used in
> deleting/creating/finalizing/opening blocks, then the speed of DataNode IO
> will slow down. Here is the graph of blocking relationship
> !image-2022-01-21-22-52-52-912.png|width=976,height=299!
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
