+1. Thanks a lot for working on this! Tsz-Wo
On Wed, May 25, 2022 at 3:08 AM NILOTPAL NANDI <nilotpal.apa...@gmail.com> wrote: > +1 for the merge. > > Thanks, > Nilotpal Nandi > > On Tue, May 24, 2022 at 8:40 PM Hanisha Koneru <koneru.hani...@gmail.com> > wrote: > > > +1 for merge. > > > > Thanks > > Hanisha > > > > On Tue, May 24, 2022 at 5:08 AM Mukul Kumar Singh < > > mksingh.apa...@gmail.com> > > wrote: > > > > > +1 for this feature. > > > > > > On 24/05/22 2:46 pm, jackson yao wrote: > > > > +1 for the merge. Thanks Sammi and Mark for working on this. > > > > > > > > guimark <guim...@126.com> 于2022年5月24日周二 16:58写道： > > > > > > > >> +1 for merge. > > > >> > > > >> > > > >> Thanks Sammi for driving this! > > > >> > > > >> > > > >> And for the second question from Arpit, we are running this feature > > for > > > >> months and it looks stable. > > > >> We've also recommend this feature to another company from China, > > after a > > > >> continuous test for 1 week they gave positive feedbacks :) > > > >> Even they are considering converting all v2 containers to v3 > > > >> containers(potentially with an offline cli tool), because they have > > > lots of > > > >> small containers due to previous ratis pipeline problems, > > > >> and with v2 containers, a lot of rocksdb instances will keep open & > > > close > > > >> very often which have negative effects on reads. > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> At 2022-05-24 15:51:58, "mingchao zhao" <captain...@apache.org> > > wrote: > > > >>> +1 for the merge. Thanks Sammi and Mark for working on this. > > > >>> > > > >>> > > > >>> On Tue, May 24, 2022 at 1:35 PM Lokesh Jain <lj...@apache.org> > > wrote: > > > >>> > > > >>>> +1 > > > >>>> > > > >>>> Regads > > > >>>> Lokesh > > > >>>> > > > >>>>> On 24-May-2022, at 9:56 AM, Uma Maheswara Rao Gangumalla < > > > >>>> umaganguma...@gmail.com> wrote: > > > >>>>> +1 for the merge. > > > >>>>> > > > >>>>> Thanks to the Tencent team and Sammi for working on this. > > > >>>>> > > > >>>>> Regards, > > > >>>>> Uma > > > >>>>> > > > >>>>> On Mon, May 23, 2022 at 5:49 AM Sammi Chen <sammic...@apache.org > > > > > >> wrote: > > > >>>>>> Dear Ozone Devs, > > > >>>>>> > > > >>>>>> I 'm starting this discussion thread for the proposal to merge > the > > > >> Ozone > > > >>>>>> Datanode RocksDB merge feature branch (HDDS-3630 > > > >>>>>> https://issues.apache.org/jira/browse/HDDS-3630) to the master > > > >> branch. > > > >>>>>> Currently there will be one RocksDB for each Container on > > datanode, > > > >>>> which > > > >>>>>> leads to hundreds of thousands of RocksDB instances on one > > datanode. > > > >>>> It's > > > >>>>>> very challenging to manage this amount of RocksDB instances in > one > > > >> JVM. > > > >>>>>> Please refer to the "problem statement" section of the design > > > >>>> document[1] > > > >>>>>> for challenge details. Unlike the current approach, Datanode > > RocksDB > > > >>>> merge > > > >>>>>> feature will use only one RocksDB for each data volume. With far > > > >> fewer > > > >>>>>> RocksDB instances to manage, the write path performance and DN > > > >> stability > > > >>>>>> are improved, Refer to the Micro Benchmark Data section of the > > > design > > > >>>>>> document[1]. > > > >>>>>> > > > >>>>>> The feature has been developed in the last 8 months. And it has > > also > > > >>>> been > > > >>>>>> deployed in Tencent production for 5 months. Now we have > reviewed > > > all > > > >>>> core > > > >>>>>> patches, and merged them into the feature branch, including new > > > disk > > > >>>>>> layout, new container schema, container replication, container > > > >> deletion, > > > >>>>>> block deletion, container scanner, non-rolling upgrade. There > are > > no > > > >>>>>> blocking JIRAs left. > > > >>>>>> There are two on-progress JIRAs[2], one is about the feature > > > document > > > >>>> md, > > > >>>>>> another is add acceptance test case. Hopefully, we will resolve > > them > > > >>>> this > > > >>>>>> week. > > > >>>>>> > > > >>>>>> For the sake of master branch feature stability, we provide a > > > >>>> configuration > > > >>>>>> to enable/disable this feature. Currently by default, this > feature > > > is > > > >>>>>> disabled. > > > >>>>>> > > > >>>>>> For more information, please check out the Merge RocksDB in > > datanode > > > >>>>>> feature wiki page here: > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >> > > > > > > https://cwiki.apache.org/confluence/display/OZONE/Merge+Rocksdb+in+Datanode+%28HDDS-3630%29+Merge+Checklist > > > >>>>>> Great thanks to Mark Gui<guim...@126.com>, majority feature > > > >>>> investigation, > > > >>>>>> design and core development is contributed by him. Thanks a lot > to > > > >>>> Runzhi > > > >>>>>> Wang, for first finding current schema V2 layout challenges and > > > >>>> initiating > > > >>>>>> the proposal of merge RocksDB, to Nanda, Mukul, Wei-Chiu and > > > Stephen > > > >>>>>> O’Donnell for the help in design review, design discussion and > > code > > > >>>>>> review. If I miss anyone here, sorry for that, but your > efforts > > > are > > > >>>> well > > > >>>>>> appreciated. > > > >>>>>> > > > >>>>>> > > > >>>>>> If there are no objections for the merge, I will start the > > official > > > >> vote > > > >>>>>> later. > > > >>>>>> > > > >>>>>> [1] Merge RocksDB in Datanode Design Doc > > > >>>>>> < > > > >>>>>> > > > >> > > > > > > https://docs.google.com/document/d/1oPCRUVqTJDbYITbwvoVCOZ2-k108FwOXVhq9ukDKHbQ/edit?usp=sharing > > > >>>>>> [2] https://issues.apache.org/jira/browse/HDDS-6790 Add > document > > > >>>>>> https://issues.apache.org/jira/browse/HDDS-6791 Add smoke > > > test > > > >>>>>> > > > >>>>>> > > > >>>>>> Thanks, > > > >>>>>> Sammi > > > >>>>>> > > > >>>> > > > >>>> > > --------------------------------------------------------------------- > > > >>>> To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org > > > >>>> For additional commands, e-mail: dev-h...@ozone.apache.org > > > >>>> > > > >>>> > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org > > > For additional commands, e-mail: dev-h...@ozone.apache.org > > > > > > -- > > Thanks > > Hanisha > > >
