[
https://issues.apache.org/jira/browse/HDFS-14497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16841164#comment-16841164
]
He Xiaoqiao commented on HDFS-14497:
------------------------------------
[~jojochuang] Thanks for your quick response.
{quote}IIRC it's around several minutes range for large clusters.
{quote}
the worst case I meet is above 300s. But I believe this is not the worst case,
since this cost is related with number of DataNodes, number of blocks
information which need to print out and also related with load of local storage
of NameNode. If the meta save is invoke just after ha-switch it will cost very
very long time, because number of postponed blocks could very huge for large
cluster, and it will bring series risks.
{quote}I don't think switching to read lock would help much. I've seen
read-only RPCs holding lock for too long and cause failover.
{quote}
I agree that it could not solve problem completely, however it may improve
something, for instance, heartbeat could be process by namenode when we switch
to read lock, especially it could be very useful if there no lifeline
installation. Please correct if something I missed.
Thanks [~jojochuang] again
> Write lock hold by metasave impact following RPC processing
> -----------------------------------------------------------
>
> Key: HDFS-14497
> URL: https://issues.apache.org/jira/browse/HDFS-14497
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: He Xiaoqiao
> Assignee: He Xiaoqiao
> Priority: Major
>
> NameNode meta save hold global write lock currently, so following RPC r/w
> request or inner-thread of NameNode could be paused if they try to acquire
> global read/write lock and have to wait before metasave release it.
> I propose to change write lock to read lock and let some read request could
> be process normally. I think it could not change informations which meta save
> try to get if we try to open read request.
> Actually, we need ensure that there are only one thread to execute metaSave,
> otherwise, output streams could meet exception especially both streams hold
> the same file handle or some other same output stream.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
