Xpray commented on a change in pull request #8346: [FLINK-12405] [DataSet]
Introduce BLOCKING_PERSISTENT result partition type
URL: https://github.com/apache/flink/pull/8346#discussion_r286751650
##########
File path:
flink-optimizer/src/test/java/org/apache/flink/optimizer/plantranslate/JobGraphGeneratorTest.java
##########
@@ -265,6 +274,55 @@ public void testArtifactCompression() throws IOException {
assertState(nonExecutableDirEntry, false, true);
}
+ @Test
+ public void testGeneratingJobGraphWithUnconsumedResultPartition() {
+
+ ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
+
+ DataSet<Tuple2<Long, Long>> input = env.fromElements(new
Tuple2<>(1L, 2L))
+ .setParallelism(1);
+
+ DataSet ds = input.map((MapFunction<Tuple2<Long, Long>,
Object>) value -> new Tuple2<>(value.f0 + 1, value.f1))
+ .setParallelism(3);
+
+ AbstractID intermediateDataSetID = new AbstractID();
+
+ // this output branch will be excluded.
+
ds.output(BlockingShuffleOutputFormat.createOutputFormat(intermediateDataSetID))
+ .setParallelism(1);
+
+ // this is the normal output branch.
+ ds.output(new DiscardingOutputFormat())
+ .setParallelism(1);
+
+ Plan plan = env.createProgramPlan();
+ Optimizer pc = new Optimizer(new Configuration());
+ OptimizedPlan op = pc.compile(plan);
+
+ JobGraphGenerator jgg = new JobGraphGenerator();
+ JobGraph jobGraph = jgg.compileJobGraph(op);
+
+ Assert.assertEquals(3,
jobGraph.getVerticesSortedTopologicallyFromSources().size());
+
+ JobVertex inputVertex =
jobGraph.getVerticesSortedTopologicallyFromSources().get(0);
+ JobVertex mapVertex =
jobGraph.getVerticesSortedTopologicallyFromSources().get(1);
+ JobVertex outputVertex =
jobGraph.getVerticesSortedTopologicallyFromSources().get(2);
+
+ Assert.assertTrue(outputVertex instanceof OutputFormatVertex);
Review comment:
> It is better to use `hamcrest` to verify `instanceof` here.
Great, it seemes `hamcrest ` is more readable here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
