[
https://issues.apache.org/jira/browse/HDDS-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Elek resolved HDDS-3160.
-------------------------------
Fix Version/s: 0.6.0
Resolution: Fixed
> Disable index and filter block cache for RocksDB
> ------------------------------------------------
>
> Key: HDDS-3160
> URL: https://issues.apache.org/jira/browse/HDDS-3160
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Marton Elek
> Assignee: Marton Elek
> Priority: Minor
> Labels: pull-request-available
> Fix For: 0.6.0
>
> Attachments: key_allocation_after.png, key_allocation_before.png,
> profile.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During preformance tests It was noticed that the OM performance is dropped
> after 10-20 million of keys. (see the screenshot).
> By default cache_index_and_filter_blocks is enabled for all of our RocksDB
> instances (see DBProfile) which is not the best option. (For example see this
> thread: https://github.com/facebook/rocksdb/issues/3961#)
> With turning on this cache the indexes and bloom filters are cached **inside
> the block cache** which makes slower the cache when we have significant data.
> Without turning it on (based on my understanding) all the indexes will remain
> open without any cache. With our current settings we have only a few number
> of sst files (even with million of keys) therefore it seems to be safe to
> turn this option off.
> With turning this option of I was able to write >100M keys with high
> throughput.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
