countang opened a new issue, #9662: URL: https://github.com/apache/rocketmq/issues/9662
### Is Your Feature Request Related to a Problem? Currently, RocketMQ uses JSON as the primary serialization format for communication between components such as the Proxy and Broker, especially in newer architectures like the stateless Proxy introduced in RocketMQ 5.x. While JSON is human-readable and widely supported, it has notable performance overhead in terms of both serialization/deserialization latency and CPU usage. In high-throughput or low-latency scenarios, the cost of parsing large JSON payloads can become a bottleneck, particularly in the Proxy layer where message routing, metadata handling, and protocol translation happen frequently. ### Describe the Solution You'd Like We propose adding support for more efficient serialization protocols, such as Protocol Buffers (Protobuf), Apache Avro, or FlatBuffers, to reduce serialization overhead and improve overall system performance. Specifically, we suggest: Introduce a pluggable serialization framework in the Proxy and Broker modules, allowing multiple serialization formats (e.g., JSON, Protobuf) to coexist. Support Protobuf as an optional or default protocol for internal communication (e.g., between Proxy and Broker, or in client-Proxy interactions). Provide configuration options to enable/disable or prioritize specific serialization formats per endpoint or deployment. Maintain backward compatibility by supporting JSON as a fallback. ### Describe Alternatives You've Considered We have evaluated several alternative approaches to address the serialization performance bottleneck: - Optimizing JSON usage (e.g., using faster JSON parsers like Jackson with annotations, or Jsoniter) While this can provide moderate performance improvements (e.g., 10–20% faster parsing), it does not fundamentally solve the inherent inefficiencies of text-based formats, such as larger payload size, higher GC pressure, and lack of schema enforcement. It is also limited by JSON's dynamic nature and reflection overhead. - Custom binary format with manual serialization Designing a lightweight binary protocol tailored to RocketMQ’s internal messages could achieve maximum efficiency. However, this approach sacrifices interoperability, increases maintenance cost, and makes debugging and evolution more difficult. It also lacks tooling support (e.g., schema evolution, code generation). Using Apache Avro or FlatBuffers - Avro provides schema-based serialization and good performance, but requires a schema registry for efficient operation, adding operational complexity. FlatBuffers enables zero-copy parsing and excellent read performance, but has limited language support and higher complexity in message construction. While both are viable, they are less mature in the Java ecosystem compared to Protobuf and have smaller community adoption in messaging systems. - Sticking with JSON but compressing payloads (e.g., GZIP) Compression can reduce network bandwidth usage, but it adds CPU overhead and does not eliminate the cost of parsing — the deserialization still requires full decompression and parsing into objects. ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
