Hi William, If IoTDB uses stateMachineCache in any StateMachine, there's an open problem regarding extended (off-heap) memory utilzation (https://issues.apache.org/jira/browse/RATIS-2093).
Also, an increase in off-heap memory utilization is indeed expected. The reason is, before zero-copy, incoming data is buffered (by netty) in the off-heap area and then is parsed by grpc/protobuf in a copy fashion as on-heap objects. The off-heap buffer is released immediately. With zero-copy optimization, when grpc/protobuf parses the proto objects with binary reference to the original off-heap buffer instead of copying to the heap. The original off-heap buffers are kept longer, hence increasing off-heap utilization. The compensation for this is a reduction of heap usage and copies so you should see less heap utilization and GC activities. The JVM process's resident set size (RSS) is also a fair metric to observe. However, there're a few required adaptations to the StateMachine code. The following APIs are kept alive (deprecated) for backward compatibility, and to avoid corruption, we need to copy data for those APIs. It's required to remove the usage of them to fully utilize the benefit of zero-copy: 1. StateMachine.write(LogEntryProto) 2. StateMachine.write(LogEntryProto, TransactionContext) 3. TransactionContext.getLogEntry() Regarding the release of 3.1.0, please wait until the following issues are resolved: RATIS-2092, RATIS-2093 and RATIS-2094. Thanks, Duong On 2024/05/21 08:02:42 William Song wrote: > Hi dev, > > I would like to propose the release of Apache Ratis version 3.1.0. Our last > release was about 5 months ago and there are a lot of new features, > improvements and bug fixes since then. > > However, I suggest that we exclude the zero-copy feature [RATIS-1931 and its > child issues] from this release. The recent build of Ratis-3.1.0-snapshot > (master branch, commit 192ce48, built in 2024.05.16) showed twice off-heap > memory usage in IoTDB regression tests. (Please let me know if anyone > encounter the same issue, thanks in advance!) > > Best Regards, > William >
