xhumanoid commented on code in PR #3769:
URL: https://github.com/apache/celeborn/pull/3769#discussion_r3718339130
##########
common/src/main/java/org/apache/celeborn/common/network/protocol/Encoders.java:
##########
@@ -20,20 +20,20 @@
import java.nio.charset.StandardCharsets;
import io.netty.buffer.ByteBuf;
+import io.netty.buffer.ByteBufUtil;
/** Provides a canonical set of Encoders for simple types. */
public class Encoders {
/** Strings are encoded with their length followed by UTF-8 bytes. */
public static class Strings {
public static int encodedLength(String s) {
- return 4 + s.getBytes(StandardCharsets.UTF_8).length;
+ return 4 + ByteBufUtil.utf8Bytes(s);
}
public static void encode(ByteBuf buf, String s) {
- byte[] bytes = s.getBytes(StandardCharsets.UTF_8);
- buf.writeInt(bytes.length);
- buf.writeBytes(bytes);
+ buf.writeInt(ByteBufUtil.utf8Bytes(s));
+ ByteBufUtil.writeUtf8(buf, s);
Review Comment:
in agoda we use this version
```
int lenIdx = buf.writerIndex(); - remember current index
buf.writeInt(0); - write 0
int written = ByteBufUtil.writeUtf8(buf, s); - write all data
buf.setInt(lenIdx, written); - update specific offset
with actual size
```
in this case you don't need to traverse string byte[] twice to calculate how
much data you will write
original code
```
buf.writeInt(ByteBufUtil.utf8Bytes(s)); - read string array and
calculate required amount of bytes for each character (utf-8 have variable size
for each)
ByteBufUtil.writeUtf8(buf, s); - write data
```
this double size calculation can be reason of increased time for
`encodeString (256)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
