Hi, ZhangJian He. Thanks for your feedback. > autorecovery on every bookie, and not run it as a part of a bookie process. > I'd go as far as to call it the best practice. As Andrey said, I think that's > the best practice to maintain big bookkeeper clusters. It is the the best practice to deploy a dedicated AutoRecovery cluster. But there are still users do not want to maintain another cluster for some reason. For example, if the scale of the bookkeeper cluster is not that large (maybe with 20 nodes like the experiment described in the pr), user may not want to deploy another AutoRecovery cluster. But 20 replicators are suffice to cause great pressure on Zookeeper as the experimental data shows. As for us, it is for historical reason. If this improvement can’t fix the problem, we will think about deploying another AutoRecovery cluster. But i am glad that it work pretty well. There is no need to add up the complexity of the whole deployment and increase the difficulty of maintenance.
> +0.5 for this change. And this is a zookeeper specified change, and refers to > existing configuration params. I think `zkReplicationTaskRateLimit` might be > better. I have renamed the config to `zkReplicationTaskRateLimit`, thanks for feedback. Thanks, thetumbled. On 2024/04/29 23:40:08 ZhangJian He wrote: > >> In such configuration (and in general) I'd recommend not to run > autorecovery on every bookie, and not run it as a part of a bookie process. > I'd go as far as to call it the best practice. > As Andrey said, I think that's the best practice to maintain big bookkeeper > clusters. > > +0.5 for this change. And this is a zookeeper specified change, and refers > to existing configuration params. I think `zkReplicationTaskRateLimit` > might be better. > > Thanks > ZhangJian He > > > On Tue, Apr 30, 2024 at 2:07 AM Andrey Yegorov > <an...@datastax.com.invalid> wrote: > > > +1 > > > > I am ok with the change if we maintain default behavior the same as now (no > > throttling). > > I left comments on implementation PR. > > > > On Mon, Apr 1, 2024 at 1:54 AM thetumbled <wo...@qq.com.invalid> wrote: > > > > > Hi all, > > > I create a new BP to relieve the pressure of Zookeeper caused by auto > > > recovery, Looking forward to your feedback. > > > https://github.com/apache/bookkeeper/issues/4257 > > > > > > Best Regards, > > > thetumbled. > > > > > > > > -- > > Andrey Yegorov > > >
