One more question. Since I have set the "Maximum Concurrent Checkpoints" to
1. Will cascading effect still be true?
Whenever my sink operator returns to normal (in terms of latency), new
checkpoint after this point should work, right? there are no other
in-flight/concurrent checkpoints still in pr
Stephan, thanks a lot for the explanation. Now everything makes sense to
me. Will set the min pause.
On Sat, Dec 2, 2017 at 8:58 AM, Stephan Ewen wrote:
> Hi Steven!
>
> You are right, there could be some cascading effect from previous
> checkpoints.
> I think the best way to handle that is to s
I use Flink Kafka connector 10 to subscribe topics and get data. Now I want
to specify topics not using String, but regular expression. I want to do
that just because it can recognize future topics added to the Kafka and get
their data.
Spark Kafka connector has a method named SubscribePattern and
Hi Steven!
You are right, there could be some cascading effect from previous
checkpoints.
I think the best way to handle that is to set the "minimum pause between
checkpoints". In fact, I would actually recommend this over the checkpoint
interval parameter.
The pause will allow the job to handle
