szetszwo edited a comment on pull request #298:
URL: https://github.com/apache/incubator-ratis/pull/298#issuecomment-733402768
Consider the existing code for async:
```
//FileStoreClient
private static <OUTPUT, THROWABLE extends Throwable> OUTPUT writeImpl(
CheckedFunction<ByteString, OUTPUT, THROWABLE> sendFunction,
String path, long offset, boolean close, ByteBuffer data)
throws THROWABLE {
final WriteRequestHeaderProto.Builder header =
WriteRequestHeaderProto.newBuilder()
.setPath(ProtoUtils.toByteString(path))
.setOffset(offset)
.setLength(data.position())
.setClose(close);
final WriteRequestProto.Builder write = WriteRequestProto.newBuilder()
.setHeader(header)
.setData(ByteString.copyFrom(data));
final FileStoreRequestProto request =
FileStoreRequestProto.newBuilder().setWrite(write).build();
return sendFunction.apply(request.toByteString());
}
```
The idea is to define the streaming message format as below:
```
<header-length><header><data>
```
The entire message could be big, say 100MB. We divide it in chunks, say
1MB, and send them using DataStreamOutput.writeAsync.
We send a WriteRequestHeaderProto per message in Async. In Streaming, we
only need to send a header (say StreamRequestHeaderProto) once for the entire
stream since the entire stream is logically a single message. The
StreamRequestHeaderProto has path and data length.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
