Hi RocketMQ Community:

After the release of RocketMQ 4.5.0, the DLedger mode (raft) was introduced. 
The raft commitlog under this architecture is used to replace the original 
commitlog so that it has the ability to failover. However, there are some 
disadvantages going with this architecture due to the raft capability on 
replication, including:

1. To have failover ability, the number of replicas in the broker group must be 
3 or more.

2. Acks from replicas need to strictly follow the majority rule of the Raft 
protocol, that is, 3-replica architecture requires acks from 2 replicas to 
return, and 5-replica architecture requires acks from 3 to return.

3. Since the store repository relies on OpenMessaging DLedger in DLedger mode, 
native storage and replication capabilities of RocketMQ (such as 
transientStorePool and zero-copy capabilities) cannot be reused, and 
maintenance becomes difficult as well.

To handle those mentioned problems, I would like to start an email thread to 
discuss the RIP-44 Support DLedger Controller. With this improvement, DLedger 
(Raft) capability will be abstracted onto the upper layer, becoming an optional 
and loosely coupled coordination component named DLedger Controller.

After the deployment of DLedger Controller, the master-slave architecture will 
also equip with failover capability. The DLedger Controller can optionally be 
embedded into the NameServer (the NameServer itself remains stateless and 
cannot provide electoral capabilities when the majority is down), or it can be 
deployed independently.

DLedger controller is an optional component that does not change the previous 
operation and maintenance mode. Compared with other components, its downtime 
will not affect online services. In addition, RIP-44 unifies the storage and 
replication of RocketMQ, resulting in lower maintenance costs and faster 
development iterations. In terms of compatibility, the master-slave 
architecture can upgrade without compatibility problems.

I've already done part of the work with Zhangheng Huang. Our proposals are 
provided at the links 
below:https://docs.google.com/document/d/1tSJkor_3Js4NBaVA0UENGyM8Mh0SrRMXszRyI91hjJ8/edit?usp=sharing

Chinese version:
https://shimo.im/docs/N2A1Mz9QZltQZoAD/


Please welcome to reply to this email or comment on the proposal if you have 
any questions or suggestions.




Thanks,
RongtongJin

Reply via email to