[
https://issues.apache.org/jira/browse/HDFS-16848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17635816#comment-17635816
]
ASF GitHub Bot commented on HDFS-16848:
---------------------------------------
howzi commented on PR #5147:
URL: https://github.com/apache/hadoop/pull/5147#issuecomment-1319852875
> > Thank you for reply, it exactly create more pressure on zk for some
moment, but didn't create more read/write requests. Maybe it shouldn't be a
default configuration
>
> From my personal point of view, ZK is not a high-concurrency component. At
the same time, ZK plays a role in HDFS, YARN HA, YARN App management, etc. If
we have no obvious performance problems, it is not recommended to modify this
part of the code.
>
> Let's wait for other partners' suggestions.
>
> Thanks again for your contribution!
> > Thank you for reply, it exactly create more pressure on zk for some
moment, but didn't create more read/write requests. Maybe it shouldn't be a
default configuration
>
> From my personal point of view, ZK is not a high-concurrency component. At
the same time, ZK plays a role in HDFS, YARN HA, YARN App management, etc. If
we have no obvious performance problems, it is not recommended to modify this
part of the code.
>
> Let's wait for other partners' suggestions.
>
> Thanks again for your contribution!
Actually it is an obvious performance problem, it takes over 3 mins to
refresh the state store cache in our enviroment. Different deployment of ZK may
cause a diffrent choice. For example, we have a exclusive ZK cluster for
router, it's not a big problem for us.
Anyway, I realized this is not a good feature for everyone, I will change it
to an optional configuration, that must be better.
> RBF:Improving StateStoreZookeeperImpl
> --------------------------------------
>
> Key: HDFS-16848
> URL: https://issues.apache.org/jira/browse/HDFS-16848
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rbf
> Reporter: Sun Hao
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> Currently, router is getting/updating state from zk sequentially. It will
> slowdown router load/update state cache especially for a large cluster or a
> multi region cluster.
> We propose adding a threadpool to deal with zk state synchronization。
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
