sjwiesman commented on a change in pull request #10498: [FLINK-14495][docs] Add documentation for memory control of RocksDB state backend URL: https://github.com/apache/flink/pull/10498#discussion_r355570820
########## File path: docs/ops/state/large_state_tuning.md ########## @@ -210,6 +210,28 @@ and not from the JVM. Any memory you assign to RocksDB will have to be accounted of the TaskManagers by the same amount. Not doing that may result in YARN/Mesos/etc terminating the JVM processes for allocating more memory than configured. +#### Bound total memory usage of RocksDB instance(s) per slot + +RocksDB allocates native memory without control of JVM, and might lead the process to exceed total memory budget of the container to get killed in container environment (e.g. Kubernetes). +From Flink-1.10, we provide a solution to limit total memory usage for RocksDb instance(s) per slot by leveraging RocksDB's mechanism to Review comment: In general its a good idea to stay away from phrases like "From Flink 1.10". Over many versions, these phrases build up and get confusing. Because we host all docs versions most users just want to know what is possible in the current version, not when a feature was introduced. If they are using an older version then they can refer to that versions docs. For pointing out new features to users I encourage you to add this to the release notes and blog when it is written. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
