[
https://issues.apache.org/jira/browse/HDFS-16848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637051#comment-17637051
]
ASF GitHub Bot commented on HDFS-16848:
---------------------------------------
howzi commented on PR #5147:
URL: https://github.com/apache/hadoop/pull/5147#issuecomment-1323243368
> > Actually it is an obvious performance problem, it takes over 3 mins to
refresh the state store cache in our enviroment. Different deployment of ZK may
cause a diffrent choice. For example, we have a exclusive ZK cluster for
router, it's not a big problem for us.
> > Anyway, I realized this is not a good feature for everyone, I will
change it to an optional configuration, that must be better.
>
> Can we provide some information to explain this problem? it will help to
understand this change better.
>
> Thank you very much for helping to review this pr! @ZanderXu
We have thousands of mount points and
> > Actually it is an obvious performance problem, it takes over 3 mins to
refresh the state store cache in our enviroment. Different deployment of ZK may
cause a diffrent choice. For example, we have a exclusive ZK cluster for
router, it's not a big problem for us.
> > Anyway, I realized this is not a good feature for everyone, I will
change it to an optional configuration, that must be better.
>
> Can we provide some information to explain this problem? it will help to
understand this change better.
>
> Thank you very much for helping to review this pr! @ZanderXu
We have thousands of mount points on zk, and the network delay between zk
and router become lager because of multi-region deployment. So we are going to
fix it by this way.
> RBF: Improve StateStoreZookeeperImpl
> -------------------------------------
>
> Key: HDFS-16848
> URL: https://issues.apache.org/jira/browse/HDFS-16848
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rbf
> Reporter: Sun Hao
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> Currently, router is getting/updating state from zk sequentially. It will
> slowdown router load/update state cache especially for a large cluster or a
> multi region cluster.
> We propose adding a threadpool to deal with zk state synchronization。
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
