Thanks Mark. In that case let's enable the feature by default so we get more usage.
> On May 24, 2022, at 1:57 AM, guimark <guim...@126.com> wrote: > > +1 for merge. > > > Thanks Sammi for driving this! > > > And for the second question from Arpit, we are running this feature for > months and it looks stable. > We've also recommend this feature to another company from China, after a > continuous test for 1 week they gave positive feedbacks :) > Even they are considering converting all v2 containers to v3 > containers(potentially with an offline cli tool), because they have lots of > small containers due to previous ratis pipeline problems, > and with v2 containers, a lot of rocksdb instances will keep open & close > very often which have negative effects on reads. > > > > > > > > > > > > > > > At 2022-05-24 15:51:58, "mingchao zhao" <captain...@apache.org> wrote: >> +1 for the merge. Thanks Sammi and Mark for working on this. >> >> >> On Tue, May 24, 2022 at 1:35 PM Lokesh Jain <lj...@apache.org> wrote: >> >>> +1 >>> >>> Regads >>> Lokesh >>> >>>> On 24-May-2022, at 9:56 AM, Uma Maheswara Rao Gangumalla < >>> umaganguma...@gmail.com> wrote: >>>> >>>> +1 for the merge. >>>> >>>> Thanks to the Tencent team and Sammi for working on this. >>>> >>>> Regards, >>>> Uma >>>> >>>> On Mon, May 23, 2022 at 5:49 AM Sammi Chen <sammic...@apache.org> wrote: >>>> >>>>> Dear Ozone Devs, >>>>> >>>>> I 'm starting this discussion thread for the proposal to merge the Ozone >>>>> Datanode RocksDB merge feature branch (HDDS-3630 >>>>> https://issues.apache.org/jira/browse/HDDS-3630) to the master branch. >>>>> >>>>> Currently there will be one RocksDB for each Container on datanode, >>> which >>>>> leads to hundreds of thousands of RocksDB instances on one datanode. >>> It's >>>>> very challenging to manage this amount of RocksDB instances in one JVM. >>>>> Please refer to the "problem statement" section of the design >>> document[1] >>>>> for challenge details. Unlike the current approach, Datanode RocksDB >>> merge >>>>> feature will use only one RocksDB for each data volume. With far fewer >>>>> RocksDB instances to manage, the write path performance and DN stability >>>>> are improved, Refer to the Micro Benchmark Data section of the design >>>>> document[1]. >>>>> >>>>> The feature has been developed in the last 8 months. And it has also >>> been >>>>> deployed in Tencent production for 5 months. Now we have reviewed all >>> core >>>>> patches, and merged them into the feature branch, including new disk >>>>> layout, new container schema, container replication, container deletion, >>>>> block deletion, container scanner, non-rolling upgrade. There are no >>>>> blocking JIRAs left. >>>>> There are two on-progress JIRAs[2], one is about the feature document >>> md, >>>>> another is add acceptance test case. Hopefully, we will resolve them >>> this >>>>> week. >>>>> >>>>> For the sake of master branch feature stability, we provide a >>> configuration >>>>> to enable/disable this feature. Currently by default, this feature is >>>>> disabled. >>>>> >>>>> For more information, please check out the Merge RocksDB in datanode >>>>> feature wiki page here: >>>>> >>>>> >>>>> >>> https://cwiki.apache.org/confluence/display/OZONE/Merge+Rocksdb+in+Datanode+%28HDDS-3630%29+Merge+Checklist >>>>> >>>>> Great thanks to Mark Gui<guim...@126.com>, majority feature >>> investigation, >>>>> design and core development is contributed by him. Thanks a lot to >>> Runzhi >>>>> Wang, for first finding current schema V2 layout challenges and >>> initiating >>>>> the proposal of merge RocksDB, to Nanda, Mukul, Wei-Chiu and Stephen >>>>> O’Donnell for the help in design review, design discussion and code >>>>> review. If I miss anyone here, sorry for that, but your efforts are >>> well >>>>> appreciated. >>>>> >>>>> >>>>> If there are no objections for the merge, I will start the official vote >>>>> later. >>>>> >>>>> [1] Merge RocksDB in Datanode Design Doc >>>>> < >>>>> >>> https://docs.google.com/document/d/1oPCRUVqTJDbYITbwvoVCOZ2-k108FwOXVhq9ukDKHbQ/edit?usp=sharing >>>>>> >>>>> [2] https://issues.apache.org/jira/browse/HDDS-6790 Add document >>>>> https://issues.apache.org/jira/browse/HDDS-6791 Add smoke test >>>>> >>>>> >>>>> Thanks, >>>>> Sammi >>>>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org >>> For additional commands, e-mail: dev-h...@ozone.apache.org >>> >>> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org For additional commands, e-mail: dev-h...@ozone.apache.org
