jiangxin369 commented on code in PR #209:
URL: https://github.com/apache/flink-ml/pull/209#discussion_r1101173179
##########
flink-ml-tests/src/test/java/org/apache/flink/test/iteration/BoundedAllRoundCheckpointITCase.java:
##########
@@ -131,16 +129,7 @@ static JobGraph createVariableAndConstantJobGraph(
int maxRound,
int failoverCount,
SinkFunction<OutputRecord<Integer>> sinkFunction) {
- StreamExecutionEnvironment env =
- StreamExecutionEnvironment.getExecutionEnvironment(
- new Configuration() {
- {
- this.set(
- ExecutionCheckpointingOptions
-
.ENABLE_CHECKPOINTS_AFTER_TASKS_FINISH,
- true);
- }
- });
+ StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.getConfig().enableObjectReuse();
Review Comment:
Would it be better if used `TestUtils.getExecutionEnvironment` here to
disable kryo for these tests?
##########
flink-ml-lib/src/test/java/org/apache/flink/ml/feature/HashingTFTest.java:
##########
@@ -83,15 +81,9 @@ public class HashingTFTest extends AbstractTestBase {
@Before
public void before() {
- Configuration config = new Configuration();
-
config.set(ExecutionCheckpointingOptions.ENABLE_CHECKPOINTS_AFTER_TASKS_FINISH,
true);
- env = StreamExecutionEnvironment.getExecutionEnvironment(config);
- env.getConfig().enableObjectReuse();
- env.setParallelism(4);
- env.enableCheckpointing(100);
- env.setRestartStrategy(RestartStrategies.noRestart());
+ env = TestUtils.getExecutionEnvironment();
tEnv = StreamTableEnvironment.create(env);
- DataStream<Row> dataStream = env.fromCollection(INPUT);
+ DataStream<Row> dataStream = env.fromCollection(INPUT,
Types.ROW(Types.LIST(Types.STRING)));
Review Comment:
So users need to specify the type info by themselves, right? if no type info
is specified, then kryo is used.
##########
flink-ml-benchmark/src/main/java/org/apache/flink/ml/benchmark/Benchmark.java:
##########
@@ -78,6 +79,8 @@ private static void executeBenchmarks(CommandLine
commandLine) throws Exception
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.getConfig().enableObjectReuse();
+ env.getConfig().disableGenericTypes();
Review Comment:
Do we need to `disableGenericTypes` in benchmark? First, not all algorithms
work well with this config, e.g. HashingTF. Secondly, it is already guaranteed
that kryo is disabled in unit tests of each algorithm.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
