[
https://issues.apache.org/jira/browse/FLINK-8974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555711#comment-16555711
]
ASF GitHub Bot commented on FLINK-8974:
---------------------------------------
zentol commented on a change in pull request #6415: [FLINK-8974] Run all-round
DataSet job with failures in HA mode
URL: https://github.com/apache/flink/pull/6415#discussion_r205116733
##########
File path:
flink-end-to-end-tests/flink-dataset-allround-test/src/main/java/org/apache/flink/batch/tests/DataSetAllroundTestProgram.java
##########
@@ -66,14 +61,21 @@ public static void main(String[] args) throws Exception {
ParameterTool params = ParameterTool.fromArgs(args);
int loadFactor =
Integer.parseInt(params.getRequired("loadFactor"));
String outputPath = params.getRequired("outputPath");
+ boolean infinite = params.getBoolean("infinite", false);
ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
int numKeys = loadFactor * 128 * 1024;
- DataSet<Tuple2<String, Integer>> x1Keys = env.createInput(new
Generator(numKeys, 1)).setParallelism(4);
+ DataSet<Tuple2<String, Integer>> x1Keys;
DataSet<Tuple2<String, Integer>> x2Keys = env.createInput(new
Generator(numKeys * 32, 2)).setParallelism(4);
DataSet<Tuple2<String, Integer>> x8Keys = env.createInput(new
Generator(numKeys, 8)).setParallelism(4);
+ if (infinite) {
+ x1Keys =
env.createInput(Generator.infinite()).setParallelism(4).filter(t -> t.f1 >= 0);
Review comment:
that shouldn't be possible due to back-pressure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> End-to-end test: Run general purpose DataSet job with failures in standalone
> mode
> ---------------------------------------------------------------------------------
>
> Key: FLINK-8974
> URL: https://issues.apache.org/jira/browse/FLINK-8974
> Project: Flink
> Issue Type: Sub-task
> Components: Tests
> Affects Versions: 1.5.0
> Reporter: Till Rohrmann
> Assignee: Tuo Wang
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> Similar to FLINK-8973, we should setup an end-to-end test where we run the
> general purpose DataSet job from FLINK-8972 in a HA standalone setting with
> failures and process kills.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
