mxm opened a new pull request, #778: URL: https://github.com/apache/flink-kubernetes-operator/pull/778
This is the generalized version of #762 in which we tune all directly observable Flink memory pools: heap, managed memory, network, and metaspace. Notable changes: - New metrics to measure managed memory, network, and metaspace usage in addition to heap usage have been added - All memory pools are tuned, i.e. their size is increased or decreased according to their usage. - Max memory size of a pool is no longer limited by the original configured limit for that pool. The only cap is the max TM memory in the spec. - Memory can be freely moved between the pools - The feature to "give back" memory to RocksDB has been removed. The reason is that the new version does that automatically. The available memory is distributed across the memory pools based on their usage. Managed memory is assigned last. In the case of a streaming job without RocksDB, this should result in very low managed memory usage and hence reduce the managed and total memory. However, when RocksDB is used, the managed memory is allowed to grow until the max available memory assigned in the spec. - We assign a minimum of 256Mb (configurable) per memory pool The changes have been verified with an actual Kubernetes deployment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
