+1 Regads Lokesh
> On 24-May-2022, at 9:56 AM, Uma Maheswara Rao Gangumalla > <umaganguma...@gmail.com> wrote: > > +1 for the merge. > > Thanks to the Tencent team and Sammi for working on this. > > Regards, > Uma > > On Mon, May 23, 2022 at 5:49 AM Sammi Chen <sammic...@apache.org> wrote: > >> Dear Ozone Devs, >> >> I 'm starting this discussion thread for the proposal to merge the Ozone >> Datanode RocksDB merge feature branch (HDDS-3630 >> https://issues.apache.org/jira/browse/HDDS-3630) to the master branch. >> >> Currently there will be one RocksDB for each Container on datanode, which >> leads to hundreds of thousands of RocksDB instances on one datanode. It's >> very challenging to manage this amount of RocksDB instances in one JVM. >> Please refer to the "problem statement" section of the design document[1] >> for challenge details. Unlike the current approach, Datanode RocksDB merge >> feature will use only one RocksDB for each data volume. With far fewer >> RocksDB instances to manage, the write path performance and DN stability >> are improved, Refer to the Micro Benchmark Data section of the design >> document[1]. >> >> The feature has been developed in the last 8 months. And it has also been >> deployed in Tencent production for 5 months. Now we have reviewed all core >> patches, and merged them into the feature branch, including new disk >> layout, new container schema, container replication, container deletion, >> block deletion, container scanner, non-rolling upgrade. There are no >> blocking JIRAs left. >> There are two on-progress JIRAs[2], one is about the feature document md, >> another is add acceptance test case. Hopefully, we will resolve them this >> week. >> >> For the sake of master branch feature stability, we provide a configuration >> to enable/disable this feature. Currently by default, this feature is >> disabled. >> >> For more information, please check out the Merge RocksDB in datanode >> feature wiki page here: >> >> >> https://cwiki.apache.org/confluence/display/OZONE/Merge+Rocksdb+in+Datanode+%28HDDS-3630%29+Merge+Checklist >> >> Great thanks to Mark Gui<guim...@126.com>, majority feature investigation, >> design and core development is contributed by him. Thanks a lot to Runzhi >> Wang, for first finding current schema V2 layout challenges and initiating >> the proposal of merge RocksDB, to Nanda, Mukul, Wei-Chiu and Stephen >> O’Donnell for the help in design review, design discussion and code >> review. If I miss anyone here, sorry for that, but your efforts are well >> appreciated. >> >> >> If there are no objections for the merge, I will start the official vote >> later. >> >> [1] Merge RocksDB in Datanode Design Doc >> < >> https://docs.google.com/document/d/1oPCRUVqTJDbYITbwvoVCOZ2-k108FwOXVhq9ukDKHbQ/edit?usp=sharing >>> >> [2] https://issues.apache.org/jira/browse/HDDS-6790 Add document >> https://issues.apache.org/jira/browse/HDDS-6791 Add smoke test >> >> >> Thanks, >> Sammi >> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org For additional commands, e-mail: dev-h...@ozone.apache.org
