showuon commented on a change in pull request #11528:
URL: https://github.com/apache/kafka/pull/11528#discussion_r754844183
##########
File path: docs/streams/developer-guide/memory-mgmt.html
##########
@@ -195,14 +195,14 @@ <h2><a class="toc-backref" href="#id3">RocksDB</a><a
class="headerlink" href="#r
}
}</code></pre>
<div>
- <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a
fraction of the block cache to set aside for "high priority" (aka index and
filter) blocks, preventing them from being evicted by data blocks. See the full
signature of the <a class="reference external"
href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72";>LRUCache
constructor</a>.
- NOTE: the boolean parameter in the cache constructor lets you
control whether the cache should enforce a strict memory limit by failing the
read or iteration in the rare cases where it might go larger than its capacity.
Due to a
- <a class="reference external"
href="https://github.com/facebook/rocksdb/issues/6247";>bug in RocksDB</a>, this
option cannot be used
- if the write buffer memory is also counted against the cache. If you
set this to true, you should NOT pass the cache in to the
<code>WriteBufferManager</code> and just control the write buffer and cache
memory separately.</sup>
+ <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a
fraction of the block cache to set aside for "high priority" (aka index and
filter) blocks, preventing them from being evicted by data blocks.
+ The boolean parameter in the cache constructor lets you control
whether the cache should enforce a strict memory limit by failing the read or
iteration in the rare cases where it might go larger than its capacity.
+ See the full signature of the LRUCache constructor <a
class="reference external"
href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72";>here</a>.
+ </sup>
<br>
<sup id="fn2">2. This must be set in order for
INDEX_FILTER_BLOCK_RATIO to take effect (see footnote 1) as described in the <a
class="reference external"
href="https://github.com/facebook/rocksdb/wiki/Block-Cache#caching-index-and-filter-blocks";>RocksDB
docs</a></sup>
<br>
- <sup id="fn3">3. You may want to modify the default <a
class="reference external"
href="https://github.com/apache/kafka/blob/2.3/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java#L79";>block
size</a> per these instructions from the <a class="reference external"
href="https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB#indexes-and-filter-blocks";>RocksDB
GitHub</a>. A larger block size means index blocks will be smaller, but the
cached data blocks may contain more cold data that would otherwise be evicted.
+ <sup id="fn3">3. You may want to modify the default <a
class="reference external"
href="https://github.com/apache/kafka/blob/2.3/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java#L79";>block
size</a> per these instructions from the <a class="reference external"
href="https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB#indexes-and-filter-blocks";>RocksDB
docs</a>. A larger block size means index blocks will be smaller, but the
cached data blocks may contain more cold data that would otherwise be evicted.
Review comment:
Update the "RocksDB GIthub" to "RocksDB doc" for consistency above.
You may want to modify the default block size per these instructions from
the RocksDB **GitHub**
-> You may want to modify the default block size per these instructions from
the RocksDB **doc**
##########
File path: docs/streams/developer-guide/memory-mgmt.html
##########
@@ -195,14 +195,14 @@ <h2><a class="toc-backref" href="#id3">RocksDB</a><a
class="headerlink" href="#r
}
}</code></pre>
<div>
- <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a
fraction of the block cache to set aside for "high priority" (aka index and
filter) blocks, preventing them from being evicted by data blocks. See the full
signature of the <a class="reference external"
href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72";>LRUCache
constructor</a>.
- NOTE: the boolean parameter in the cache constructor lets you
control whether the cache should enforce a strict memory limit by failing the
read or iteration in the rare cases where it might go larger than its capacity.
Due to a
- <a class="reference external"
href="https://github.com/facebook/rocksdb/issues/6247";>bug in RocksDB</a>, this
option cannot be used
- if the write buffer memory is also counted against the cache. If you
set this to true, you should NOT pass the cache in to the
<code>WriteBufferManager</code> and just control the write buffer and cache
memory separately.</sup>
+ <sup id="fn1">1. INDEX_FILTER_BLOCK_RATIO can be used to set a
fraction of the block cache to set aside for "high priority" (aka index and
filter) blocks, preventing them from being evicted by data blocks.
+ The boolean parameter in the cache constructor lets you control
whether the cache should enforce a strict memory limit by failing the read or
iteration in the rare cases where it might go larger than its capacity.
+ See the full signature of the LRUCache constructor <a
class="reference external"
href="https://github.com/facebook/rocksdb/blob/master/java/src/main/java/org/rocksdb/LRUCache.java#L72";>here</a>.
+ </sup>
<br>
<sup id="fn2">2. This must be set in order for
INDEX_FILTER_BLOCK_RATIO to take effect (see footnote 1) as described in the <a
class="reference external"
href="https://github.com/facebook/rocksdb/wiki/Block-Cache#caching-index-and-filter-blocks";>RocksDB
docs</a></sup>
<br>
- <sup id="fn3">3. You may want to modify the default <a
class="reference external"
href="https://github.com/apache/kafka/blob/2.3/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java#L79";>block
size</a> per these instructions from the <a class="reference external"
href="https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB#indexes-and-filter-blocks";>RocksDB
GitHub</a>. A larger block size means index blocks will be smaller, but the
cached data blocks may contain more cold data that would otherwise be evicted.
+ <sup id="fn3">3. You may want to modify the default <a
class="reference external"
href="https://github.com/apache/kafka/blob/2.3/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java#L79";>block
size</a> per these instructions from the <a class="reference external"
href="https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB#indexes-and-filter-blocks";>RocksDB
docs</a>. A larger block size means index blocks will be smaller, but the
cached data blocks may contain more cold data that would otherwise be evicted.
Review comment:
Update the "RocksDB GIthub" to "RocksDB docs" for consistency above.
You may want to modify the default block size per these instructions from
the RocksDB **GitHub**
-> You may want to modify the default block size per these instructions from
the RocksDB **docs**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
