anishshri-db commented on code in PR #41042:
URL: https://github.com/apache/spark/pull/41042#discussion_r1185418799
##########
docs/structured-streaming-programming-guide.md:
##########
@@ -2360,8 +2360,35 @@ Here are the configs regarding to RocksDB instance of
the state store provider:
<td>The maximum number of MemTables in RocksDB, both active and immutable.
Value of -1 means that RocksDB internal default values will be used</td>
<td>-1</td>
</tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.boundedMemoryUsage</td>
+ <td>Whether total memory usage for RocksDB state store instances on a
single node is bounded.</td>
+ <td>false</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.maxMemoryUsageMB</td>
+ <td>Total memory limit in MB for RocksDB state store instances on a single
node.</td>
+ <td>500</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.writeBufferCacheRatio</td>
+ <td>Total memory to be occupied by write buffers as a fraction of memory
allocated across all RocksDB instances on a single node.</td>
Review Comment:
Done
##########
docs/structured-streaming-programming-guide.md:
##########
@@ -2360,8 +2360,35 @@ Here are the configs regarding to RocksDB instance of
the state store provider:
<td>The maximum number of MemTables in RocksDB, both active and immutable.
Value of -1 means that RocksDB internal default values will be used</td>
<td>-1</td>
</tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.boundedMemoryUsage</td>
+ <td>Whether total memory usage for RocksDB state store instances on a
single node is bounded.</td>
+ <td>false</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.maxMemoryUsageMB</td>
+ <td>Total memory limit in MB for RocksDB state store instances on a single
node.</td>
+ <td>500</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.writeBufferCacheRatio</td>
+ <td>Total memory to be occupied by write buffers as a fraction of memory
allocated across all RocksDB instances on a single node.</td>
+ <td>0.5</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.highPriorityPoolRatio</td>
+ <td>Total memory to be occupied by filter and index blocks as a fraction
of memory allocated across all RocksDB instances on a single node.</td>
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
