Hi, tison We had to search for a long time for gRPC-java-related APIs and issues back then before we finally found the UnsafeByteOperations.unsafeWrap <https://github.com/apache/iotdb/blob/master/iotdb-core/consensus/src/main/java/org/apache/iotdb/consensus/ratis/RequestMessage.java#L49> function to avoid application-level copying . In hindsight, it seems like we should have referred to and learned from Ozone's implementation earlier.
However, it's essential to note that once you pass a buffer using this method, it can no longer be modified, which is also why gRPC declares this API as unsafe. Xinyu Tan Tsz Wo Sze <[email protected]> 于2023年9月16日周六 00:21写道： > HI tison, > > Good to know that IoTDB has avoided buffer copying using the > UnsafeByteOperations provided by protobuf. FYI, Ozone is also doing the > same thing: > https://github.com/apache/ozone/blob/932c52062225f16328e1a553cf1ee43afff4447d/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ByteStringConversion.java#L49 > > Tsz-Wo > > On Fri, Sep 15, 2023 at 7:41 AM Zili Chen <[email protected]> wrote: > >> > have a utility for conversion >> >> Previously I'm afraid that this way may cause copy. But now I found that >> IoTDB provide an example to avoid copy: >> >> UnsafeByteOperations.unsafeWrap(content.asReadOnlyByteBuffer()) >> >> ... this way, we only create a new wrapper class but keep the underneath >> bytebuffer as is without copy. >> >> On 2019/11/14 03:15:38 Mukul Kumar Singh wrote: >> > Hi Tison, >> > >> > Thanks for the interest in Ratis. There are 2 options as you have >> > already noticed. >> > >> > a) Ozone which is a consumer of Ratis, we have used the shaded >> > ByteString version in the Ozone codebase. >> > >> > b) We can have a utility for conversion as you have already pointed out. >> > >> > >> > Thanks, >> > Mukul >> > >> > >> > On 13/11/19 8:28 pm, tison wrote: >> > > Well I find a way to write a utility for convertion. >> > > >> > > Best, >> > > tison. >> > > >> > > >> > > tison <[email protected] <mailto:[email protected]>> >> > > 于2019年11月13日周三 下午10:46写道： >> > > >> > > Hi devs, >> > > >> > > I am trying to develop a filesystem-view storage on ratis, and >> > > here is the problem I meet: >> > > >> > > When I trying to reply within `StateMachine#query` while >> > > generating Message, compiler fails on >> > > ByteString is not compatible with ByteString. I think it is >> > > because ratis use shaded protobuf deps. >> > > >> > > However, how can I instance Message outside ratis project? Shall I >> > > also depends on >> > > ratis-thirdparty-misc? Is there a workaround? Or will ratis >> > > provides its own abstraction for >> > > resolving dep issues? >> > > >> > > Best, >> > > tison. >> > > >> > >> >
