szetszwo commented on pull request #221: URL: https://github.com/apache/incubator-ratis/pull/221#issuecomment-707542547
This is a good question -- In user applications such as Ozone, the may originally use AsyncApi.send to send a message. Suppose the use data is stored as a local file in the client side. The client has to read the entire file (say, 100MB) as a ByteString and then send it out using AsyncApi.send(entire_file_message). Internally, AsyncImpl/RaftClientImpl create a RaftClientRequest containing entire_file_message. In DataStream, the client will first create a DataStreamOutput. Then client read and send chunks (say, 1MB each chunk) of the file in a loop. The client should use the zero buffer copying API in Java to read the file chunk to a direct buffer. If it is the case, DataStream could send it out without any buffer copying. Internally, DataStreamOutputImpl will create a RaftClientRequest and send it out as the header. Note that we allow buffer copying for the RaftClientRequest since it only has metadata. Therefore, we may use Protobuf to encode RaftClientRequest. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
