[
https://issues.apache.org/jira/browse/HDFS-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078015#comment-14078015
]
Esteban Gutierrez commented on HDFS-6766:
-----------------------------------------
+1. should be the condition timeout configurable? So far I think this is a
really good fix [~xieliang007] and this will improve responsiveness of HBase
Region Servers when there are issues in the HDFS pipeline. Thanks for the fix!
> optimize ack notify mechanism to avoid thundering herd issue
> ------------------------------------------------------------
>
> Key: HDFS-6766
> URL: https://issues.apache.org/jira/browse/HDFS-6766
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.0
> Reporter: Liang Xie
> Assignee: Liang Xie
> Attachments: HDFS-6766.txt
>
>
> Currently, DFSOutputStream uses wait/notifyAll to coordinate ack receiving
> and ack waiting, etc..
> say there're 5 threads(t1,t2,t3,t4,t5) wait for ack seq no: 1,2,3,4,5, once
> the no. 1 ack arrived, the "notifyAll" be called, so t2/t3/t4/t5 could do
> nothing except wait again.
> we can rewrite it with Condition class, with a fair policy(fifo), we can just
> make t1 be notified, so a number of context switch be saved.
> It's possible more than one thread waiting on the same ack seq no(e.g. no
> more data be written between two flush operations), so once it happened, we
> need to notify those threads, so i introduced a set to remember this seq no.
> In a simple HBase ycsb testing, the context switch number per second was
> reduced about 15%, and reduced sys cpu% about 6%(My HBase has new write model
> patch, i think the benefit will be higher if w/o it)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
