imaffe edited a comment on issue #1852: URL: https://github.com/apache/rocketmq/issues/1852#issuecomment-638693095
After I digged deeper into this problem, I found it rather hard to solve given we don't currently have reference to TopicConfigManager in DefaultMessageStore. To know which topics are currently not deleted in DefaultMessageStore, we cannot rely on it's `consumeQueueTable` because `consumeQueueTable` itself is constructed using commitLog(which contains entries that belong to a deleted topic) In order to know if an entry in the commitLog belongs to a deleted topic, we have to query topicConfigManager, )but this would be a very big change as we introduced a dependency in broker level, which complicates the synchronization schema as well). The problem is we can't selectively delete an entry in commitLog (unless they expires themselves), and upon restart the async CQ building process will always try to create a CQ entry for each entry in the commitLog. Seems like not introducing TopicConfigManager in the DefaultMessageStore would be a better choice if we can prove that keeping the consumeQueue for a deleted topic would not hurt safety. () Since consumers interacts directly with consume queues, it might be not safe in this case (as consume queue keeps the deleted topic information even the topic is not in TopicConfigManager). However I didn't verify this in the source code. This exposes another problem. We are using topic Names instead of topicId as unique identifiers for a given topic, we cannot distinguish between a deleted topic and a new topic with a same name. For example, we deleted a topic "user_info" for application A, and we create a new "user_info" for topic B, they are not relevant but consumers can consume messages from older topic. We can avoid this by persuade users don't do so but these kinds of vulnerabilities would still exist. Since using one commitLog for all topics is the core feature of RocketMQ, choices we can make are very limited. Will be so happy to see different opinions~~ ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
