afedulov commented on code in PR #20757:
URL: https://github.com/apache/flink/pull/20757#discussion_r991387182
##########
flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/datagen/DataGeneratorPerCheckpoint.java:
##########
@@ -40,22 +35,19 @@ public static void main(String[] args) throws Exception {
final StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.enableCheckpointing(3000);
env.setRuntimeMode(RuntimeExecutionMode.STREAMING);
- env.setParallelism(1);
+ env.setParallelism(8);
final String[] elements = new String[] {"a", "b", "c", "d", "e", "f",
"g", "h", "i", "j"};
final int size = elements.length;
final GeneratorFunction<Long, String> generatorFunction =
index -> elements[(int) (index % size)];
- final RateLimiter rateLimiter = new GatedRateLimiter(size);
-
- final SourceReaderFactory<String,
NumberSequenceSource.NumberSequenceSplit> factory =
- context ->
- new RateLimitedSourceReader<>(
- new GeneratingIteratorSourceReader<>(context,
generatorFunction),
- rateLimiter);
final DataGeneratorSource<String> generatorSource =
- new DataGeneratorSource<>(factory, Long.MAX_VALUE,
Types.STRING);
+ new DataGeneratorSource<>(
+ generatorFunction,
+ Long.MAX_VALUE,
+ RateLimiterStrategy.perCheckpoint(4),
Review Comment:
In general, I am not sure if we want to keep it as a separate example.
Probably won't harm to have it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
