lindong28 commented on code in PR #248:
URL: https://github.com/apache/flink-ml/pull/248#discussion_r1297874568
##########
flink-ml-iteration/flink-ml-iteration-common/src/main/java/org/apache/flink/iteration/typeinfo/IterationRecordSerializer.java:
##########
@@ -106,7 +106,7 @@ public int getLength() {
@Override
public void serialize(IterationRecord<T> record, DataOutputView target)
throws IOException {
target.writeByte((byte) record.getType().ordinal());
- serializerNumber(record.getEpoch(), target);
+ target.writeInt(record.getEpoch());
Review Comment:
Regarding "the original serializerNumber cannot serialize Integer.MAX_VALUE
+ 1", can you explain why it is necessary to pass `Integer.MAX_VALUE + 1` as
the serializerNumber? Is it possible to change the caller side code to avoid
that?
I agree the 1-byte overhead is likely not a big deal for common cases where
iteration is needed. On the other hand, if it is possible to change the caller
side code as appropriate without any downside, then that approach is better
than increasing the epoch overhead from 1 byte to 4 bytes.
##########
flink-ml-iteration/flink-ml-iteration-common/src/main/java/org/apache/flink/iteration/typeinfo/IterationRecordSerializer.java:
##########
@@ -106,7 +106,7 @@ public int getLength() {
@Override
public void serialize(IterationRecord<T> record, DataOutputView target)
throws IOException {
target.writeByte((byte) record.getType().ordinal());
- serializerNumber(record.getEpoch(), target);
+ target.writeInt(record.getEpoch());
Review Comment:
Regarding "the original serializerNumber cannot serialize Integer.MAX_VALUE
+ 1", can you explain why it is necessary to pass `Integer.MAX_VALUE + 1` as
the serializerNumber? Is it possible to change the caller side code to avoid
that?
I agree the 1-byte overhead is likely not a big deal for common cases where
iteration is needed. On the other hand, if it is possible to change the caller
side code as appropriate without any downside, then that approach would be
strictly better than increasing the epoch overhead from 1 byte to 4 bytes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
