+1 I partially agree with vongosling that this proposal requires more feedback before initiating a vote. But taking advantage of gRPC does save a lot of work, especially those blocking issues caused by FastJSON due to its non-strict conformance to JSON standards. Review how cumbersome it is to implement the reset-offset-to-playback feature for Non-Java SDK. In addition, ProtoBuffer supports protocol evolution pretty well in the long term.
Actually, this proposal, IMO, is not to replace existing SDKs, instead, it is to make the transport and serialization layer pluggable and thus configurable, which is similar to MySQL X Protocol: https://dev.mysql.com/doc/internals/en/x-protocol.html On Wed, Jun 16, 2021 at 11:14 AM vongosling <[email protected]> wrote: > -1 I think some pro and cron including tech details should be clarified. > I would like to get feedback on this RIP proposal. 不要着急投票 ~ > > > > 我其实是部分支持这个Proposal的，但是新协议如何和老协议兼容，这是一个令人望而却步的工作。我之前提过，如果希望解决多语言的问题，可以参考下dapr目前的做法。在云原生时代，坦率的说，我们不需要富客户端，但是瘦客户端能满足我们80%的主要场景吗？这就跟之前我们一直在强调pull的好处，而远离push，结果呢？如果我们能够以始为终认真看待这个问题， > 近2年多年来 > ，社区前仆后继接近30号同学一直致力于多语言的改进优化，我希望看到工作的延续性。这里，我希望能够慎重的看到这个问题，希望看到更多类似Yin > James的分享和讨论。 > > heng du <[email protected]> 于2021年6月15日周二 上午11:37写道： > > > Hi RocketMQ Community, > > > > This is the vote for the kickoff of RIP-23 RocketMQ gRPC protocol > support. > > > > On one hand RocketMQ. remoting module is too complicated to maintain, > gRPC > > makes it easier to establish a robust communication layer, the current > > remoting module would be simplified radically. > > On the other hand, gRPC has been the de-facto standard in CloudNative, > > service mesh would be easily applied if gRPC is enabled. > > > > So in this RIP, we want to support gRPC's protocol, simplify the current > > communication layer of RocketMQ, make it easier to adapt to other > > languages, which is not limited to CPP/GO/C#/GO。 > > > > The vote will be open for at least 72 hours or until a necessary number > > of votes are reached. > > > > Please vote accordingly: > > > > [ ] +1 approve > > [ ] +0 no opinion > > [ ] -1 disapprove with the reason > > > > > > i yangkun <[email protected]> 于2021年6月8日周二 下午8:28写道： > > > > > Background & Motivation > > > What do we need to do > > > > > > > > > * Will we add a new module? > > > maybe. > > > * Will we add new APIs? > > > Yes. > > > > > > * Will we add new feature? > > > Yes. > > > > > > > > > Why should we do that > > > > > > > > > * Are there any problems of our current project? > > > > > > a. Remoting module is too complicated to maintain, gRPC makes it easier > > to > > > establish a robust communication layer, the current remoting module > would > > > be simplified radically. > > > > > > b. gRPC has been the de-facto standard in CloudNative, service mesh > would > > > be easily applied if gRPC is enabled. > > > > > > c. The private protocol of RocketMQ depends on the FastJson, it is > > > difficult to adapt for other language. > > > > > > On the other side, since the pop consumer has been merged, we could > > > implement new SDK based on gRPC and pop, which is easier to develop and > > > maintain. > > > > > > Chinese Version: > > > > > > a. Remoting 模块对于长期的维护而言过于复杂了，我们可以使用 gRPC 更轻松地建立起一个健壮的通信层，这会使得现有的 > remoting > > > 模块从根本上得到简化。 > > > > > > b. gRPC 目前已经是云原生时代的事实标准，使用 gRPC 可以使得我们天然获取一些云原生的能力，比如 Service Mesh。 > > > > > > c. 目前 RocketMQ 的私有协议强烈依赖 FastJson，多语言的适配将会变得困难。 > > > > > > > > > 从另外一个角度来说，鉴于 pop 消费者已经被合并，我们可以基于 gRPC 和 pop 实现新的 SDK，新的 SDK > 将会更加易于开发和维护。 > > > > > > Goals > > > > > > > > > * What problem is this proposal designed to solve? > > > > > > Support gRPC's protocol, simplify current communication layer oof > > > RocketMQ, make it easier to adapt for other language, which is not > > limited > > > to CPP/GO/C#/GO。 > > > > > > Chinese Version: > > > > > > 支持 gRPC 协议，简化 RocketMQ 现有的通信层，复用 gRPC 的能力，简化多语言适配成本，不限于 CPP/GO/C#/GO。 > > > > > > * To what degree should we solve the problem? > > > This RIP must guarantee below point: > > > > > > 1. Compatibility: Both of gRPC and RemotingCommand should be > > supported. > > > 2. High performance: This implementation does not affects latency > and > > > throughput. > > > > > > > > > Chinese Version: > > > > > > 新方案需要保证两点： > > > > > > 1. 兼容性：同时支持 gRPC 和 RemotingCommand 协议，不影响现有功能。 > > > 2. 高性能：基于 gRPC 的实现不影响整理的延时和吞吐量。 > > > > > > > > > Non-Goals > > > > > > > > > * What problem is this proposal NOT designed to solve? > > > Nothing specific. > > > * Are there any limits of this proposal? > > > Nothing specific. > > > > > > > > > Changes > > > Architecture > > > > > > > > > Current broker processor and client. > > > > > > [ > > > > > > https://intranetproxy.alipay.com/skylark/lark/0/2021/png/200096/1623142547507-128b85f5-98f4-4568-85f8-28ef32982b7c.png > > > ] > > > > > > Proposed gRPC processor and client. > > > > > > [ > > > > > > https://intranetproxy.alipay.com/skylark/lark/0/2021/png/200096/1623142552491-a7f58ac0-cd7d-4ddd-936e-fb296b667196.png > > > ] > > > > > > Broker would provide a protocol negotiate procedure to distinguish > > > RemotingCommand from gRPC protocol. two kinds or processor in broker > > would > > > re-use the same port to serve for RPC from different SDK. > > > > > > > > > Chinese Version: > > > > > > broker 本身提供协议协商机制用于区分 RemotingCommnad 和 gRPC 协议，broker 针对 gRPC 和 > > > RemotingCommand 提供不同的 processor 为各自的 SDK 服务。 > > > > > > Interface Design/Change > > > > > > > > > * Method signature changes > > > Nothing specific. > > > * Method behavior changes > > > Nothing specific. > > > > > > * CLI command changes > > > Nothing specific. > > > * Log format or content changes > > > Nothing specific. > > > > > > > > > Compatibility, Deprecation, and Migration Plan > > > > > > > > > * Are backward and forward compatibility taken into consideration? > > > > > > Broker support processor of RemotingCommand and gRPC simultaneously, so > > > there are one compatibility situations: > > > > > > If user migrates from original SDK to gRPC SDK in push mode, the > > > re-balance policy should make sure that it would not cause repeated > > > consumption for a lot of messages. > > > > > > * Are there deprecated APIs? > > > Nothing specific. > > > * How do we do migration? > > > Nothing specific. > > > > > > > > > Implementation Outline > > > > > > > > > We will implement the proposed changes by 4 phases. > > > > > > > > > Phase 1 > > > > > > 1. Provides gRPC protocol definition(IDL) > > > > > > Phase 2 > > > > > > 1. Implement gRPC processor of broker. > > > 2. Implement protocol negotiation of two kinds of protocol(gRPC and > > > RemotingCommand) > > > > > > Phase 3 > > > > > > 1. Implement new JAVA and CPP native SDK based on gRPC > > > > > > Phase 4 > > > > > > 1. Implement native SDK base on gRPC for other language. > > > > > > > > > Rejected Alternatives > > > > > > > > > How does alternatives solve the issue you proposed? > > > > > > > > > Thrift? not so much impact as gRPC in community. > > > > > > > > > Pros and Cons of alternatives > > > > > > > > > Nothing specific. > > > > > > Why should we reject above alternatives > > > > > > > > > > > -- > Best Regards :-) >
