Yun Tang created IGNITE-16538:
---------------------------------
Summary: Ensure RocksDB to control memory usage better with
stalling write if memory usage exceeds limit
Key: IGNITE-16538
URL: https://issues.apache.org/jira/browse/IGNITE-16538
Project: Ignite
Issue Type: Improvement
Reporter: Yun Tang
Currently, Ignite choose RocksDB-6.20.3 and let all RocksDB table storages
share same {{WriteBufferManager}} within a data region. This could limit the
memory of all storage instances in a way, however, this is still not enough
considerring the case that current {{WriteBufferManager}} cannot not limit the
memory even memory_usage exceeds buffer_size with multi column familes.
We can refer to this feature descriptions:
"When WriteBufferManager is shared across DBs and column families
to maintain memory usage under a limit, OOMs have been observed when flush
cannot
finish but writes continuously insert to memtables.
In order to avoid OOMs, when memory usage goes beyond buffer_limit_ and DBs
tries to write,
this change will stall incoming writers until flush is completed and
memory_usage
drops." (see [the PR summary|https://github.com/facebook/rocksdb/pull/7898])
RocksDB introduce better memory management and public it to java side from
RocksDB-6.27.3 (see
[commit-dc00e4b120|https://github.com/facebook/rocksdb/commit/dc00e4b1206cba9995fe042ec976df54d0f6e76f])
Thus, I suggest to bump the RocksDB version to 6.27.3 and enable this feature
by default.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
