Hi all: Through the construction of [1], we have completed the optional storage implementation of the Master's metadata between BDB and ZooKeeper. Users only need to update the master.ini configuration according to the guidance document of [2] and select the corresponding [meta_zookeeper] or [meta_bdb] configuration unit is enough; before and after the migration, users can export and import the configured metadata through the bin/tubemq-metadata-bru.sh script to complete the metadata migration operation.
The BDB component is an embedded storage component. When using it, the system must use the basic classes in this component to complete related data read and write operations. In TubeMQ, the content of the BDB component is only used, with no code copying, and no API modification. Therefore, by modifying the TubeMQ system this time, all BDB-related code content can be eliminated and exist independently. For compatibility reasons, we still retain the BDB part of the system, because we used BDB in the previous version, there must be users who do not want to make major adjustments and want to continue to use BDB as a storage medium. We leave this option to users. If you find any new problems during use, please let us know through issue or email, and let's see what the problem is together. Thanks! 1. https://github.com/apache/incubator-inlong/issues/2282 2. https://inlong.apache.org/docs/next/modules/tubemq/configure_introduction Goson zhang <[email protected]> 于2022年1月23日周日 16:11写道： > I have submitted an issue [1] for processing and tracking this thread, and > keeping track of related content through this issue. > > Thanks! > > 1. https://github.com/apache/incubator-inlong/issues/2282 > > Goson zhang <[email protected]> 于2022年1月20日周四 10:16写道： > >> Someone gave me a point to consider the compatibility of the system >> switching from berkeleydb-je to ZooKeeper. >> >> I am going to provide tools and help documents so that users can use >> tools to read metadata from the system running on berkeleydb-je, and then >> import it into the new version system based on ZooKeeper. >> >> And I am wondering if the storage implementation of berkeleydb-je can be >> kept in the project so that everyone can choose by themselves. >> I understand that after the metadata storage based on ZooKeeper is >> implemented, the storage of berkeleydb-je has become a non-mandatory >> dependency. >> >> After the implementation is completed, I will ask the mentors for >> suggestions based on the implementation. >> >> Thanks! >> >> Goson zhang <[email protected]> 于2022年1月17日周一 11:32写道： >> >>> Hi all: >>> >>> According to Apache's protocol requirements[1], berkeleydb-je is not a >>> dependent component that meets Apache's open-source requirements. Next, I >>> plan to replace it, before finding a better metadata storage component, >>> with ZooKeeper. >>> >>> Let's know your consideration, in order to avoid the impact before and >>> after the change, we have abstracted the operation of metadata in the early >>> stage, so that the module can seamlessly migrate this part of the code. >>> >>> Thanks! >>> >>> 1. https://lists.apache.org/thread/nchbf7zyg78lwh04op2k0634csj8p5yv >>> >>
