Hi William, Indeed, we've just worked on RATIS-2096, which put zero-copy behind a flag that is disabled by default. It's safe to release Ratis 3.1.0 now.
Thanks for proposing the release. Regards, Duong On 2024/05/23 03:48:45 William Song wrote: > Hi Tsz-Wo and Duong, > > Thanks very much for the response! > > > I agree that we should have a release excluding Zero-Copy > > Sure! Let me prepare for the release starting by cherry-pick changes since > 3.0.1. IoTDB plans to release an official version in June, so I’m hoping to > start the 3.1.0 release process in the early June. > > > For twice off-heap memory usage, could you share more details? > I’m trying to analyze the logs and metrics. I’ll share the details once any > uncommon patterns discovered. > > > Also, an increase in off-heap memory utilization is indeed expected cted. > > The reason is, before zero-copy, incoming data is buffered (by netty) in > > the off-heap area and then is parsed by grpc/protobuf in a copy fashion as > > on-heap objects. The off-heap buffer is released immediately. With > > zero-copy optimization, when grpc/protobuf parses the proto objects with > > binary reference to the original off-heap buffer instead of copying to the > > heap. The original off-heap buffers are kept longer, hence increasing > > off-heap utilization. The compensation for this is a reduction of heap > > usage and copies so you should see less heap utilization and GC activities. > > The JVM process's resident set size (RSS) is also a fair metric to observe. > > Thanks very much for the detailed explanation! It’s good to know the expected > behavior once Zero-Copy introduced. Let me check the usage of > StateMachine.write(). > > Best, > William > > > 2024年5月22日 03:23，Duong Nguyen <[email protected]> 写道： > > > > Hi William, > > > > If IoTDB uses stateMachineCache in any StateMachine, there's an open > > problem regarding extended (off-heap) memory utilzation > > (https://issues.apache.org/jira/browse/RATIS-2093). > > > > Also, an increase in off-heap memory utilization is indeed expected. The > > reason is, before zero-copy, incoming data is buffered (by netty) in the > > off-heap area and then is parsed by grpc/protobuf in a copy fashion as > > on-heap objects. The off-heap buffer is released immediately. > > With zero-copy optimization, when grpc/protobuf parses the proto objects > > with binary reference to the original off-heap buffer instead of copying to > > the heap. The original off-heap buffers are kept longer, hence increasing > > off-heap utilization. The compensation for this is a reduction of heap > > usage and copies so you should see less heap utilization and GC activities. > > The JVM process's resident set size (RSS) is also a fair metric to observe. > > However, there're a few required adaptations to the StateMachine code. The > > following APIs are kept alive (deprecated) for backward compatibility, and > > to avoid corruption, we need to copy data for those APIs. It's required to > > remove the usage of them to fully utilize the benefit of zero-copy: > > 1. StateMachine.write(LogEntryProto) > > 2. StateMachine.write(LogEntryProto, TransactionContext) > > 3. TransactionContext.getLogEntry() > > > > Regarding the release of 3.1.0, please wait until the following issues are > > resolved: RATIS-2092, RATIS-2093 and RATIS-2094. > > > > Thanks, > > Duong > > > > On 2024/05/21 08:02:42 William Song wrote: > >> Hi dev, > >> > >> I would like to propose the release of Apache Ratis version 3.1.0. Our > >> last release was about 5 months ago and there are a lot of new features, > >> improvements and bug fixes since then. > >> > >> However, I suggest that we exclude the zero-copy feature [RATIS-1931 and > >> its child issues] from this release. The recent build of > >> Ratis-3.1.0-snapshot (master branch, commit 192ce48, built in 2024.05.16) > >> showed twice off-heap memory usage in IoTDB regression tests. (Please let > >> me know if anyone encounter the same issue, thanks in advance!) > >> > >> Best Regards, > >> William > >> > >
