lizhimins opened a new issue, #10680: URL: https://github.com/apache/rocketmq/issues/10680
# Prepare to release Apache RocketMQ 5.5.1 ### Motivation Prepare to release Apache RocketMQ 5.5.1. ### Prerequisites / Blockers - Waiting for the dLedger repository to completely remove the fastjson1 dependency before finalizing this release. ### Summary - **Stability & correctness**: Fixed 30+ bugs including pop consumer CK record loss, TLS certificate hot-reload native memory leak, tiered storage concurrency issues, ServiceThread data race, master election comparator overflow, and CombineConsumeQueue long-polling notification miss. - **Lite messaging evolution**: Server-side Lite Simple Consumer support, exclusive subscription eviction via tombstones, lite topic reset offset fix, and removal of the legacy enableLiteEventMode switch. - **Performance**: Reduced per-message and per-RPC allocations (MessageId, Remoting header encoding, metrics AttributeKey, CommitLog scratch buffer), added madvise(MADV_RANDOM) for offset binary search, and introduced dispatch memory backpressure. - **Tiered storage hardening**: Dual TTL read-ahead cache, selective double-write in CombineConsumeQueueStore, resource leak fixes, and standardized logging/error handling. - **Operations & developer experience**: Batch topic/subscription-group deletion, gRPC keepalive configuration, LMQ gauge metric, native CqCompactionFilter JNI shim, and Bazel CI alignment. ### 变更摘要 - **稳定性与正确性**：修复 30+ 个 Bug，包括 Pop 消费 CK 记录丢失、TLS 证书热加载 native 内存泄漏、分级存储并发问题、ServiceThread 数据竞争、Master 选举比较器溢出、CombineConsumeQueue 长轮询通知丢失等。 - **Lite 消息演进**：服务端支持 Lite Simple Consumer、基于墓碑机制的独占订阅驱逐、Lite Topic 重置 offset 修复、移除遗留 enableLiteEventMode 开关。 - **性能优化**：减少单消息和单 RPC 内存分配（MessageId、Remoting 头编码、Metrics AttributeKey、CommitLog 线程缓冲），offset 二分查找引入 madvise(MADV_RANDOM)，新增 dispatch 内存反压。 - **分级存储加固**：双 TTL 预读缓存、CombineConsumeQueueStore 选择性双写、资源泄漏修复、日志与错误处理标准化。 - **运维与开发体验**：批量删除 Topic/订阅组、gRPC keepalive 配置、LMQ 数量 Gauge 指标、原生 CqCompactionFilter JNI shim、Bazel CI 对齐。 ### Changelog since 5.5.0 **Bug Fixes:** - Reject delayed transactional messages in gRPC send path (#10261) - Fix incorrect time range file selection in IndexStoreService.queryAsync (#10269) - Fix OOM caused by OpenTelemetry 1.44 OtlpGrpcMetricExporter pool race (#10267) - Fix stale wildcardGroupMap entries caused by incorrect parsing during LiteTopic wildcard unregistration (#10254) - Fix ForwardMessageToDeadLetterQueue request mapping descriptor (#10275) - Fix PopConsumerService changeInvisibilityDuration losing CK record when visibilityTimeout collision (#10277) - Fix consumerOffset deserialization error (#10287) - Set brokerName for messages returned by proxy local POP path (#10298) - Fix queryMessage default index type (#10292) - Fix preserve popTime when rebuilding AckResult (#10286) - Restore opaque when proxy remoting forward fails (#10281) - Fix race condition between deleteTopic and FlushConsumeQueueService (#10376) - Fix outdated FAQ documentation URL in FAQUrl (#10394) - Fix native memory leak on TLS certificate hot-reload (#10399) - Fix shared ClientChannelInfo preventing inactive channel expiry (#10437) - Fix unexpected decrement of lmq counter (#10454) - Fix potential NPE in EscapeBridge when topicPublishInfo is null (#10507) - Fix TransactionMetricsFlushService busy spin bug (#10517) - Fix NPE when class filter route data is missing (#10420) - Fix Timer message rocksdb use wrong cache key (#10422) - Fix stale minOffset after consumeQueue truncation (#10418) - Fix lite topic reset offset: memory leak, FIFO block bypass, and offset-0 reset failure (#10550) - Resolve data race in ServiceThread wakeup mechanism (#10544) - Fix master election maxOffset comparator overflow - Fix TopicConfig DataVersion not persisted in split registration (#10605) - Fix missing long-polling notification under CombineConsumeQueue selective double-write (#10616) - Fix async request-reply RequestCallback firing multiple times (#10614) - Fix bitsarray off by one (#10506) - Fix buildTopicConfigSerializeWrapper exposing live topicConfigTable reference (#10439) - Fix concurrency bugs in tiered storage (#10471) - Fix resource leaks and lifecycle issues in tiered storage (#10476) - Fix RocketmqColdCtr logger level (#10447) - Preserve lite topic for auto-renew receipt handles (#10346) - Unregister all producer topics on gRPC client termination (#10662) - Treat blank local serve address as unset (#10663) **New Features & Improvements:** - Make Pop RocksDB BlockCache size configurable via MessageStoreConfig (#10271) - Support native CqCompactionFilter with cross-platform JNI shim (#10335, #10371) - Make HashedWheelTimer parameters configurable in QueueLevelConsumerOrderInfoLockManager (#10382) - Enforce Lite exclusive subscription eviction via server-side tombstones (#10386) - Support runtime hot-reload of maxClientEventCount in LiteEventDispatcher.ClientEventSet (#10388) - Add memory backpressure to dispatch (#10470) - Selective double-write in CombineConsumeQueueStore (#10452) - Support batch deletion of topics and subscription groups in broker (#10448) - Support Lite Simple Consumer on server side (#10570) - Add dual TTL policy for tiered storage read-ahead cache (#10573) - Add gRPC server permit keepalive configuration (#10516) - Add LMQ number gauge metric to Broker observability (#10645) - Remove enableLiteEventMode config switch (#10561) - Resolve ConsumerOffsetManager dynamically in LiteEventDispatcher (#10623) **Performance Optimizations:** - Cache Version.values() in MQVersion to avoid repeated array allocation (#10401) - Optimize LiteConsumerLagCalculator.getLagCountTopK by deferring getStoreTimestamp (#10424) - Pre-size ProxyContext map to avoid HashMap resize on request path (#10461) - Reduce per-message allocation in createUniqID and valueOfMagicCode (#10469) - Optimize thread pool sizes for commit and fetch executors (#10472) - Replace Enum.values() loop with static array lookup in LanguageCode and SerializeType (#10513) - Add getMessageType(Map) overload to eliminate redundant properties decode (#10487) - Use madvise(MADV_RANDOM) to disable kernel read-ahead during correctMinOffset binary search (#10523) - Reduce Remoting header encoding allocation via FastCodesHeader/RocketMQSerializable optimizations - Reduce per-RPC allocation in metrics by caching static AttributeKey instances (#10443) - Reuse a per-thread scratch buffer in CommitLog.checkMessageAndReturnSize (#10640) **CI / Build / Test:** - Fix Bazel CI build failure caused by Maven Central rate limiting (#10342) - Quarantine flaky tests and add detection plan docs (#10374, #10379) - Align Bazel dependencies with Maven pom.xml (#10404) - Fix or ignore several flaky tests of PopPriorityIT (#10390) - Configure GitHub workflows to use concurrency cancel-in-progress (#10445) - Add unit tests for srvutil package (#10431) - Add unit tests for CleanupPolicyUtils - Fix RocksDBOptionsFactoryTest compile failure (#10411) - Optimize PopConsumerServiceTest to avoid unnecessary background threads (#10563) - Fix flaky HATest semi-sync replication (#10495) - Validate message batch input (#10459) **Documentation & Misc:** - Update example dependency version from 4.3.0 to 5.5.0 (#10349, #10358) - Fix typos and dead links across docs (#10360, #10483, #10484, #10505, #10501, #10504, #10449, #10551) - Add debug logging for swallowed exceptions in StatsAllSubCommand (#10493) - Rename CLusterSendMsgRTCommand to ClusterSendMsgRTCommand (#10594) - Add default implementation for rejectRequest() in NettyRequestProcessor (#10547) - Remove duplicate remove call in InvocationChannel (#10248) - Standardize log format across tiered storage module (#10474) - Improve error handling in tiered storage (#10473) Full changelog: https://github.com/apache/rocketmq/compare/rocketmq-all-5.5.0...develop -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
