lindong28 commented on code in PR #234:
URL: https://github.com/apache/flink-ml/pull/234#discussion_r1174844283
##########
flink-ml-iteration/src/test/java/org/apache/flink/iteration/operator/HeadOperatorTest.java:
##########
@@ -318,6 +322,10 @@ public void
testPostponeGloballyAlignedEventsAfterSnapshot() throws Exception {
OperatorUtils.getUniqueSenderId(operatorId, 0)),
0)),
new ArrayList<>(harness.getOutput()));
+ // TODO There might be a potential bug makes the testing
hang, which could be
Review Comment:
The root cause is that `RecordingHeadOperatorFactory.latestHeadOperator` is
not properly closed. The issue can be fixed by having the following code in
`HeadOperatorTest#createHarnessAndRun`:
```
try {
return runnable.apply(harness);
} finally {
RecordingHeadOperatorFactory.latestHeadOperator.close();
}
```
##########
flink-ml-iteration/src/main/java/org/apache/flink/iteration/operator/OperatorUtils.java:
##########
@@ -136,31 +135,20 @@ public static StreamConfig
createWrappedOperatorConfig(StreamConfig config, Clas
wrappedConfig.setTypeSerializerOut(
((IterationRecordSerializer<?>)
typeSerializerOut).getInnerSerializer());
- Stream.concat(
- config.getChainedOutputs(cl).stream(),
- config.getNonChainedOutputs(cl).stream())
+ config.getChainedOutputs(cl)
.forEach(
- edge -> {
- OutputTag<?> outputTag = edge.getOutputTag();
- if (outputTag != null) {
- TypeSerializer<?> typeSerializerSideOut =
-
config.getTypeSerializerSideOut(outputTag, cl);
- checkState(
- typeSerializerSideOut instanceof
IterationRecordSerializer,
- "The serializer of side output with
tag[%s] should be IterationRecordSerializer but it is %s.",
- outputTag,
- typeSerializerSideOut);
- wrappedConfig.setTypeSerializerSideOut(
- new OutputTag<>(
- outputTag.getId(),
- ((IterationRecordTypeInfo<?>)
-
outputTag.getTypeInfo())
- .getInnerTypeInfo()),
- ((IterationRecordSerializer)
typeSerializerSideOut)
- .getInnerSerializer());
- }
+ chainedOutput -> {
+ OutputTag<?> outputTag =
chainedOutput.getOutputTag();
+ setTypeSerializerSideOut(outputTag, config,
wrappedConfig, cl);
+ });
+ config.getOperatorNonChainedOutputs(cl)
Review Comment:
Would it be simpler to keep the previous style and do something like this:
```
Stream.concat(
config.getChainedOutputs(cl).stream(),
config.getOperatorNonChainedOutputs(cl).stream())
.forEach(
output -> {
OutputTag<?> outputTag = output.getOutputTag();
setTypeSerializerSideOut(outputTag, config,
wrappedConfig, cl);
}
);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
