summaryzb commented on code in PR #950:
URL: https://github.com/apache/incubator-uniffle/pull/950#discussion_r1229128247
##########
client-spark/common/src/main/java/org/apache/spark/shuffle/writer/WriteBufferManager.java:
##########
@@ -128,7 +128,11 @@ public WriteBufferManager(
super(taskMemoryManager, taskMemoryManager.pageSizeBytes(),
MemoryMode.ON_HEAP);
this.bufferSize = bufferManagerOptions.getBufferSize();
this.spillSize = bufferManagerOptions.getBufferSpillThreshold();
- this.instance = serializer.newInstance();
+ // columnar shuffle reader use the serialized data directly
Review Comment:
yes, serializer is null for two reason below
1.columnar serializer could not handle the empty byte array
2.when use columnar shuffle writerBufferManager only use columnar data
already serialized
Ok unit test is on the way
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
