[
https://issues.apache.org/jira/browse/HDDS-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715894#comment-17715894
]
Sumit Agrawal commented on HDDS-8469:
-------------------------------------
manual WAL performance is 3-4 times slower than batch Operation
# Adding 100,000 records: 2.8 times slower
# Add/Delete same records 100,000: 5 times slower
# Delete 100,000 records: 9 times slower
# Overwrite same key 100,000 times: 4.5 times slower
One of reason is, batchOperation has map implementation for add/delete which is
fast, rocksdb udpate happens in flush.
Test code refernce:
[https://github.com/sumitagrawl/ozone/commit/b30a33ccc37af5c4f3cddff8d0c3af6a4d38e138]
> SCM using WAL manual flush of rocksDB
> -------------------------------------
>
> Key: HDDS-8469
> URL: https://issues.apache.org/jira/browse/HDDS-8469
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Sumit Agrawal
> Assignee: Sumit Agrawal
> Priority: Major
> Attachments: SCM WAL manual flush.docx
>
>
> SCM currently make use of transaction buffer with batchOperation. This
> impacts that db update is not available till snapshot is taken which flushes
> batch operation or other related event.
> As part of this, need remove batchOperation and for performance, using WAL
> with manual flush at similar interval,
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
