Status ________________________________ * Current Status: Draft * Authors: [aaron-ai](https://github.com/aaron-ai)
* Shepherds: Zhanhui Li<mailto:[email protected]>, Yukon<mailto:[email protected]>, duhengforever<mailto:[email protected]> * Mailing List discussion: <[email protected]<mailto:[email protected]>> * Pull Request: #PR_NUMBER * Released: <released_version> Background & Motivation ________________________________ What do we need to do * will we add a new module? yes. * will we add new APIs? no. * will we add a new feature? yes. Why should we do that * Are there any problems with our current project? Currently, the RocketMQ kernel supports only one transport layer protocol available, aka, Remoting. Despite its simplicity and high performance, the remoting protocol has some disadvantages: 1. Lack of formal protocol specification. Remoting is loosely defined, lacking a necessary specification document to describe each protocol detail. This indicates much uncertainty, obstructing community developers, from different backgrounds, from building uniformly-behaved SDKs and further raises the bar of participation because they have to read the existing source codebase before understanding the ins and outs. 2. Remoting is built on top of Netty, not every language has a Netty-like library. Languages like C/C++ have to pave a lot of "preparation" to stand at the same start-line. It would be awesome for RocketMQ to have gRPC, which provides a unified foundation and concepts. Based on this unified foundation, SDKs of different languages can be built with shared models and designs. 3. Remoting builds virtually everything from the ground, including auto-retry, connection-multiplexing, coding/decoding, and more. Note these all have been properly defined and implemented in gRPC-like libraries. Further, these common parts bring a significant portion of the SDK complexity, whereas they are already available in gRPC for all mainstream programming languages in the field. * What can we benefit from the proposed changes? In the future, RocketMQ is supposed to have multi-protocol support, to embrace new technology trends, service mesh for example, and enhance interoperability among different communities, like gRPC, HTTP, MQTT, etc. So we shall introduce a multi-protocol mechanism and add gRPC as a new supported protocol. As we all know, gRPC is widely adopted, and now is a CNCF incubation project. It has a great yet mature ecosystem, including a number of successful community projects. The gRPC-based protocol will bring us more productive features: 1. Build RocketMQ Client SDK with a unified interface. 2. Focus on the logic of RocketMQ itself without much effort on the transportation layer. 3. The HTTP2-based gRPC protocol makes RocketMQ more accessible to cloud-native ecology. Goals * What problem is this proposal designed to solve? Introduce a multi-protocol mechanism and add gRPC as a new supported protocol * To what degree should we solve the problem? Support gRPC and Remoting protocol both Non-Goals * What problem is this proposal NOT designed to solve? Nothing specific. * Are there any limits of this proposal? Nothing specific. Changes ________________________________ Architecture Nothing specific. Interface Design/Change * Method signature changes. No * Method behavior changes. No * CLI command changes. No * Log format or content changes. No Compatibility, Deprecation, and Migration Plan * Are backward and forward compatibility taken into consideration? Yes, new broker server will support both two protocols * Are there deprecated APIs? Nothing specific. * How do we do the migration? Nothing specific. Implementation Outline We will implement the proposed changes by two phases. Phase 1 1. Define the new protocol based on gRPC and reach a consensus in the community. 2. Implement the gRPC server in the rocketmq kernel 1. Implement Java SDK based on gRPC protocol Phase 2 1. Implement multilingual SDK, like CPP, Golang, python, C#, etc.
