[ https://issues.apache.org/jira/browse/FLINK-30073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17635599#comment-17635599 ]
Xintong Song commented on FLINK-30073: -------------------------------------- I'm no more familiar with the background of FLINK-14484 &FLINK-15532 than you are. So I cannot comment whether returning {{false}} from {{useManagedMemory()}} when {{fixed-per-slot > 0}} is a desirable design. Assuming the proposed change is a good design, I see a significant behavior change in this. Let's say fixed-per-slot is configured to X, and a Y sized memory calculated from the weights is reserved for rocksdb due to not setting {{useManagedMemory()}} to {{false}}. Assuming currently the job is running well currently, then either Y >= X and some memory is wasted, or Y < X and other not-fully-used memory (framework.off-heap, jvm-overhead, etc.) is making up for the gap. With the proposed change, Y will be shared by other managed memory use cases, while rocksdb still uses X. That means when upgrading to the new version, to keep a job with fixed-per-slot running with the same resources, user needs to increase the framework.off-heap memory by min(X, Y), and either also increase the total memory or reduce the managed memory. I'm not oppose to this proposal. I don't have enough expertise in the rocksdb state backend to decide whether this is a good design change. Just trying to say that, considering the significant behavior changes, we should be very cautious about this change and apply it only if it's absolutely necessary. cc [~yuanmei] > Managed memory can be wasted if RocksDB memory is fixed-per-slot > ---------------------------------------------------------------- > > Key: FLINK-30073 > URL: https://issues.apache.org/jira/browse/FLINK-30073 > Project: Flink > Issue Type: Bug > Components: Runtime / Task > Affects Versions: 1.16.0, 1.17.0, 1.15.2 > Reporter: Roman Khachatryan > Priority: Major > > When > [state.backend.rocksdb.memory.fixed-per-slot|https://github.com/apache/flink/blob/ba4b182955867fedfa9891bf0bf430e92eeab41a/flink-runtime/src/main/java/org/apache/flink/runtime/util/config/memory/ManagedMemoryUtils.java#L75] > is set, RocksDB does not use managed memory (this option overrides the > 'state.backend.rocksdb.memory.managed'). > However, the runtime [doesn't take this into > account|https://github.com/apache/flink/blob/ba4b182955867fedfa9891bf0bf430e92eeab41a/flink-runtime/src/main/java/org/apache/flink/runtime/util/config/memory/ManagedMemoryUtils.java#L75] > and still reserves the managed memory according to the configured weigths. > cc: [~yunta] -- This message was sent by Atlassian Jira (v8.20.10#820010)