iloveeyjafjalla opened a new issue, #11429: URL: https://github.com/apache/rocketmq/issues/11429
### Before Creating the Bug Report - [x] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions). - [x] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate. - [x] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Runtime platform environment OS: any (broker with the RocksDB config manager: `enableRocksDBStore=true`, i.e. `RocksDBSubscriptionGroupManager`) ### RocketMQ version branch: develop Git commit id: a7edeecfb ### JDK Version Compiler: Temurin 1.8.0_504 (target 1.8, same as CI) ### Describe the Bug The RocksDB variant of the subscription group manager persists the group's forbidden bits with `JSON.toJSONString(this.getForbiddenTable().get(group))`, and after the last bit of a group is cleared the table no longer holds an entry for that group — so the value written to the `forbidden` column family is the **string `"null"`**: ```java @Override public void updateForbidden(String group, String topic, int forbiddenIndex, boolean setOrClear) { super.updateForbidden(group, topic, forbiddenIndex, setOrClear); this.rocksDBConfigManager.put(FORBIDDEN_COLUMN_FAMILY_NAME, group, JSON.toJSONString(this.getForbiddenTable().get(group))); // -> "null" once the group has no forbidden bits left } ``` The same shape appears in `setForbidden` and `clearForbidden` (`broker/src/main/java/org/apache/rocketmq/broker/config/v1/RocksDBSubscriptionGroupManager.java:287-306`; the `updateForbidden` variant is at `:276-284`). On the next start the stored value is fed back into the loader, which expects a JSON object of topic → bits: ```java protected void decodeForbidden(byte[] key, byte[] body) { JSONObject jsonObject = JSON.parseObject(new String(body, RocksDBConfigManager.CHARSET)); Set<Map.Entry<String, Object>> entries = jsonObject.entrySet(); // NPE for a "null" body ... } ``` (`:263-273`.) `JSON.parseObject("null")` returns `null`, so the `NullPointerException` leaves `loadForbidden` through its `catch` and the method returns `false` (`:101-109`), `loadSubscriptionGroupAndForbidden()` returns `false` (`:95-99`), then `RocksDBSubscriptionGroupManager.load()` returns `false` (`:81-83`), `BrokerController.initializeMetadata()` returns `false` (`BrokerController.java:863`) and `BrokerStartup` answers that with `System.exit(-3)` (`BrokerStartup.java:287-291`) — **the broker does not start** until that key is repaired by hand. The trigger needs nothing more than un-forbidding the last forbidden topic of a group; with a single forbidden topic that is the very first clear, so no multi-topic scenario is required. (The related cross-topic loss when a group has several forbidden topics is reported separately in #11093 — this report is about the value that gets persisted and what it breaks on the next load; the two fixes compose.) ### Steps to Reproduce 1. Start a broker with `enableRocksDBStore=true` and forbid a topic for a group, e.g. via `UPDATE_AND_GET_GROUP_FORBIDDEN` (`AdminBrokerProcessor.updateAndGetGroupForbidden`, `:466-489`) with `readable=false` for `group=g`, `topic=t`. 2. Un-forbid the same topic (`readable=true`). The `forbidden` column family of `group` now holds the literal `null`. 3. Restart the broker: the config load fails and the process exits (`initialize()` false → `System.exit(-3)`), with `loadForbidden exception … java.lang.NullPointerException` at `decodeForbidden`. At the unit level, `RocksDBSubscriptionGroupManagerTest` already injects a mocked `RocksDBConfigManager` (no real RocksDB needed): forbidding then clearing one topic shows the `"null"` value being written today, and calling `decodeForbidden(key, "null".getBytes())` throws. ### What Did You Expect to See? Clearing a group's forbidden state persists "no forbidden state": the key is removed (or at least stores a valid empty JSON object), and the loader tolerates an already written `null` body so that brokers which ran with the broken value can still start after upgrading. ### What Did You See Instead? A `"null"` value is written, and the next broker start aborts because the loader dereferences the parsed `null`; the failure mode is a broker that cannot start from its own persisted configuration. ### Additional Context Minimal fix in `RocksDBSubscriptionGroupManager`: route the three overrides through one helper that deletes the column-family key when the group has no forbidden bits left instead of writing `JSON.toJSONString(null)`, and ignore an empty (`null`) body in `decodeForbidden` so existing stores recover. The sibling JSON mode persists the whole table, which is why it never hits this. I will open a PR with those changes plus regression tests in `RocksDBSubscriptionGroupManagerTest` (red before: a `"null"` value is written / `decodeForbidden` throws; green after: the key is deleted and a `null` body is skipped). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
