[
https://issues.apache.org/jira/browse/RATIS-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
GuoHao updated RATIS-1938:
--------------------------
Description:
Currently, stream writing requires 3 steps
1. stream init
2. write data
3. close stream
For small files can we optimize this transfer process by combining these three
steps into one and using a single rpc to merge the three steps?
{code:java}
message DataStreamPacketHeaderProto {
enum Type {
STREAM_HEADER = 0;
STREAM_DATA = 1;
STREAM_HEADER_DATA = 2; // Add a new header, the init packet contains the
data and is transmitted and then closed.
}
enum Option {
SYNC = 0;
CLOSE = 1;
}
bytes clientId = 1;
Type type = 2;
uint64 streamId = 3;
uint64 streamOffset = 4;
uint64 dataLength = 5;
repeated Option options = 6;
} {code}
I understand that https://issues.apache.org/jira/browse/RATIS-1157 can optimize
the current process, but for small files it still keeps stream init.
was:
Currently, stream writing requires 3 steps
1. stream init
2. write data
3. close stream
For small files can we optimize this transfer process by combining these three
steps into one and using a single rpc to merge the three steps?
{code:java}
message DataStreamPacketHeaderProto {
enum Type {
STREAM_HEADER = 0;
STREAM_DATA = 1;
STREAM_HEADER_DATA = 2; // Add a new header, the init packet contains the
data and is transmitted and then closed.
}
enum Option {
SYNC = 0;
CLOSE = 1;
}
bytes clientId = 1;
Type type = 2;
uint64 streamId = 3;
uint64 streamOffset = 4;
uint64 dataLength = 5;
repeated Option options = 6;
} {code}
I understand that https://issues.apache.org/jira/browse/RATIS-1157 can optimize
the current process, but for small files it still keeps stream init
> Optimize stream friendly small file transfers
> ---------------------------------------------
>
> Key: RATIS-1938
> URL: https://issues.apache.org/jira/browse/RATIS-1938
> Project: Ratis
> Issue Type: Improvement
> Reporter: GuoHao
> Priority: Major
>
> Currently, stream writing requires 3 steps
> 1. stream init
> 2. write data
> 3. close stream
> For small files can we optimize this transfer process by combining these
> three steps into one and using a single rpc to merge the three steps?
>
> {code:java}
> message DataStreamPacketHeaderProto {
> enum Type {
> STREAM_HEADER = 0;
> STREAM_DATA = 1;
> STREAM_HEADER_DATA = 2; // Add a new header, the init packet contains
> the data and is transmitted and then closed.
> }
> enum Option {
> SYNC = 0;
> CLOSE = 1;
> }
> bytes clientId = 1;
> Type type = 2;
> uint64 streamId = 3;
> uint64 streamOffset = 4;
> uint64 dataLength = 5;
> repeated Option options = 6;
> } {code}
>
> I understand that https://issues.apache.org/jira/browse/RATIS-1157 can
> optimize the current process, but for small files it still keeps stream init.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
