johnyangk commented on a change in pull request #222: [NEMO-350] Implement
Off-heap SerializedMemoryStore & [NEMO-384] Implement
DirectByteBufferInputStream for Off-heap SerializedMemoryStore
URL: https://github.com/apache/incubator-nemo/pull/222#discussion_r295146829
##########
File path:
runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/DataUtil.java
##########
@@ -147,12 +149,20 @@ private static void serializePartition(final
EncoderFactory encoderFactory,
for (final SerializedPartition<K> partitionToConvert :
partitionsToConvert) {
final K key = partitionToConvert.getKey();
-
+ if (partitionToConvert.isOffheap()) {
+ try (DirectByteBufferInputStream byteBufferInputStream =
+ new
DirectByteBufferInputStream(partitionToConvert.getBuffer())) {
+ final NonSerializedPartition<K> deserializePartition =
deserializePartition(
Review comment:
Can you reuse code?
L155-L157 and L162-L164
The last argument of `deserializePartition()` is `InputStream`, which both
`DirectByteBufferInputStream` and `ByteArrayInputStream` extend.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
