[
https://issues.apache.org/jira/browse/FLINK-8974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555665#comment-16555665
]
ASF GitHub Bot commented on FLINK-8974:
---------------------------------------
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/6415#discussion_r205106925
--- Diff:
flink-end-to-end-tests/flink-dataset-allround-test/src/main/java/org/apache/flink/batch/tests/DataSetAllroundTestProgram.java
---
@@ -66,14 +61,21 @@ public static void main(String[] args) throws Exception
{
ParameterTool params = ParameterTool.fromArgs(args);
int loadFactor =
Integer.parseInt(params.getRequired("loadFactor"));
String outputPath = params.getRequired("outputPath");
+ boolean infinite = params.getBoolean("infinite", false);
ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
int numKeys = loadFactor * 128 * 1024;
- DataSet<Tuple2<String, Integer>> x1Keys = env.createInput(new
Generator(numKeys, 1)).setParallelism(4);
+ DataSet<Tuple2<String, Integer>> x1Keys;
DataSet<Tuple2<String, Integer>> x2Keys = env.createInput(new
Generator(numKeys * 32, 2)).setParallelism(4);
DataSet<Tuple2<String, Integer>> x8Keys = env.createInput(new
Generator(numKeys, 8)).setParallelism(4);
+ if (infinite) {
+ x1Keys =
env.createInput(Generator.infinite()).setParallelism(4).filter(t -> t.f1 >= 0);
--- End diff --
This now filters out all data.
> End-to-end test: Run general purpose DataSet job with failures in standalone
> mode
> ---------------------------------------------------------------------------------
>
> Key: FLINK-8974
> URL: https://issues.apache.org/jira/browse/FLINK-8974
> Project: Flink
> Issue Type: Sub-task
> Components: Tests
> Affects Versions: 1.5.0
> Reporter: Till Rohrmann
> Assignee: Tuo Wang
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> Similar to FLINK-8973, we should setup an end-to-end test where we run the
> general purpose DataSet job from FLINK-8972 in a HA standalone setting with
> failures and process kills.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
