Lemarais commented on a change in pull request #317:
URL: https://github.com/apache/incubator-nemo/pull/317#discussion_r690063094
##########
File path:
runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/DataUtil.java
##########
@@ -304,6 +305,27 @@ public long getNumEncodedBytes() {
}
return numEncodedBytes;
}
+
+ @Override
+ public long getCurrNumSerializedBytes() {
Review comment:
[getNumSerializedBytes](https://github.com/apache/incubator-nemo/blob/74d4a86b5c99418922317be41f0655d2dc48be6a/runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/DataUtil.java#L484)
should be called after the actual data is taken out of iterator. and it is
used to check whether this task is over or not.
I don't want to make a side effect by modifying getNumSerializedBytes.
So I created a new method
[getCurrNumSerializedBytes](https://github.com/apache/incubator-nemo/blob/74d4a86b5c99418922317be41f0655d2dc48be6a/runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/DataUtil.java#L494)
to get serialized bytes before done.
##########
File path:
runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/DataUtil.java
##########
@@ -304,6 +305,27 @@ public long getNumEncodedBytes() {
}
return numEncodedBytes;
}
+
+ @Override
+ public long getCurrNumSerializedBytes() {
+ if (serializedCountingStream == null) {
+ return numSerializedBytes;
+ }
+ return numSerializedBytes + serializedCountingStream.getCount();
+ }
+
+ @Override
+ public long getCurrNumEncodedBytes() {
Review comment:
It has same definition and reason with getCurrNumSerializedBytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
