zhengchenyu commented on code in PR #2202:
URL:
https://github.com/apache/incubator-uniffle/pull/2202#discussion_r1810128492
##########
common/src/main/java/org/apache/uniffle/common/serializer/writable/WritableSerializerInstance.java:
##########
@@ -47,19 +47,28 @@ public <T> T deserialize(DataInputBuffer buffer, Class
vClass) throws IOExceptio
}
@Override
- public <K, V> SerializationStream serializeStream(OutputStream output,
boolean raw) {
+ public <K, V> SerializationStream serializeStream(
+ SerOutputStream output, boolean raw, boolean shared) {
if (raw) {
- return new RawWritableSerializationStream(this, output);
+ if (shared) {
Review Comment:
We have three stream:
* WritableSerializationStream
* RawWritableDeserializationStream
* SharedRawWritableDeserializationStream
`WritableSerializationStream` is used to parse bytes into Java objects,
mainly used on the reduce side of tez and spark.
`RawWritableSerializationStream` directly copies bytes without doing any
actual deserialization. It is mainly used on the reduce side of mr, because mr
requires raw interface.
`SharedRawWritableDeserializationStream` is similar to
`RawWritableSerializationStream`, but uses some memory optimization methods.
Mainly used for server-side merge. So deserialization is no needed.
The `Raw` prefix means that the bytes are copied directly without
unnecessary serialization. I think it should not be deleted.
Now that, I think WritableSerializationStream and
RawWritableDeserializationStream names are not changed, rename
SharedRawWritableDeserializationStream to BufferRawDeserializationStream. How
about this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
