+1 for this feature.
On 24/05/22 2:46 pm, jackson yao wrote:
+1 for the merge. Thanks Sammi and Mark for working on this.
guimark <guim...@126.com> 于2022年5月24日周二 16:58写道：
+1 for merge.
Thanks Sammi for driving this!
And for the second question from Arpit, we are running this feature for
months and it looks stable.
We've also recommend this feature to another company from China, after a
continuous test for 1 week they gave positive feedbacks :)
Even they are considering converting all v2 containers to v3
containers(potentially with an offline cli tool), because they have lots of
small containers due to previous ratis pipeline problems,
and with v2 containers, a lot of rocksdb instances will keep open & close
very often which have negative effects on reads.
At 2022-05-24 15:51:58, "mingchao zhao" <captain...@apache.org> wrote:
+1 for the merge. Thanks Sammi and Mark for working on this.
On Tue, May 24, 2022 at 1:35 PM Lokesh Jain <lj...@apache.org> wrote:
+1
Regads
Lokesh
On 24-May-2022, at 9:56 AM, Uma Maheswara Rao Gangumalla <
umaganguma...@gmail.com> wrote:
+1 for the merge.
Thanks to the Tencent team and Sammi for working on this.
Regards,
Uma
On Mon, May 23, 2022 at 5:49 AM Sammi Chen <sammic...@apache.org>
wrote:
Dear Ozone Devs,
I 'm starting this discussion thread for the proposal to merge the
Ozone
Datanode RocksDB merge feature branch (HDDS-3630
https://issues.apache.org/jira/browse/HDDS-3630) to the master
branch.
Currently there will be one RocksDB for each Container on datanode,
which
leads to hundreds of thousands of RocksDB instances on one datanode.
It's
very challenging to manage this amount of RocksDB instances in one
JVM.
Please refer to the "problem statement" section of the design
document[1]
for challenge details. Unlike the current approach, Datanode RocksDB
merge
feature will use only one RocksDB for each data volume. With far
fewer
RocksDB instances to manage, the write path performance and DN
stability
are improved, Refer to the Micro Benchmark Data section of the design
document[1].
The feature has been developed in the last 8 months. And it has also
been
deployed in Tencent production for 5 months. Now we have reviewed all
core
patches, and merged them into the feature branch, including new disk
layout, new container schema, container replication, container
deletion,
block deletion, container scanner, non-rolling upgrade. There are no
blocking JIRAs left.
There are two on-progress JIRAs[2], one is about the feature document
md,
another is add acceptance test case. Hopefully, we will resolve them
this
week.
For the sake of master branch feature stability, we provide a
configuration
to enable/disable this feature. Currently by default, this feature is
disabled.
For more information, please check out the Merge RocksDB in datanode
feature wiki page here:
https://cwiki.apache.org/confluence/display/OZONE/Merge+Rocksdb+in+Datanode+%28HDDS-3630%29+Merge+Checklist
Great thanks to Mark Gui<guim...@126.com>, majority feature
investigation,
design and core development is contributed by him. Thanks a lot to
Runzhi
Wang, for first finding current schema V2 layout challenges and
initiating
the proposal of merge RocksDB, to Nanda, Mukul, Wei-Chiu and Stephen
O’Donnell for the help in design review, design discussion and code
review. If I miss anyone here, sorry for that, but your efforts are
well
appreciated.
If there are no objections for the merge, I will start the official
vote
later.
[1] Merge RocksDB in Datanode Design Doc
<
https://docs.google.com/document/d/1oPCRUVqTJDbYITbwvoVCOZ2-k108FwOXVhq9ukDKHbQ/edit?usp=sharing
[2] https://issues.apache.org/jira/browse/HDDS-6790 Add document
https://issues.apache.org/jira/browse/HDDS-6791 Add smoke test
Thanks,
Sammi
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org
For additional commands, e-mail: dev-h...@ozone.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org
For additional commands, e-mail: dev-h...@ozone.apache.org
