Hi devs, I have finished the improvements for integrating RocketMQ with Apache Storm. Main changes are as following:
- Upgraded RocketMQ version to 4.2.0 which brings improvements and new features like batch sending - Imporved retry policy for RocketMQ consumer push mode to avoid data loss in some scenes - Batch sending supported for bolt and trident state - Allow running several consumer instances in one process, that is to say, different topics in one worker is possible PR: https://github.com/apache/storm/pull/2518 And I submit the `RocketMQ-Serializer` patch several days ago. This module includes several serialization formats, especially Apache Avro which I stated before[1]. - Raw String - JSON - Avro Generic - Avro Specified PR: https://github.com/apache/rocketmq-externals/pull/42 Any comments for these PRs are welcome. BTW here is the state of rocketmq streaming integration: Module Status * RocketMQ-Storm Patch Available * RocketMQ-Spark To Refactor * RocketMQ-Flink Patch Available Soon * RocketMQ-Serializer Patch Available Anybody who is also interested in these tasks, please join me. Let's fight together. [1] https://issues.apache.org/jira/browse/ROCKETMQ-157 Thanks, Xin Wang (vesense)
