I was wondering what is the impact if one of the stream operator function
occasionally takes too long to process the event. Given the following simple
flink job
inputStream
.KeyBy (“tenantId”)
.process ( new MyKeyedProcessFunction())
, if occasionally MyKeyedProcessFunction takes too long (say ~5-10 minutes) to
process an incoming element, what is the impact on overall pipeline? Is the
impact limited to
1. Specific key for which MyKeyedProcessFunction is currently taking too
long to process an element, or
2. Specific Taskslot, where MyKeyedProcessFunction is currently taking too
long to process an element, i.e. impacting multiple keys, or
3. Entire inputstream ?
Also what is the built in resiliency in these cases? Is there a concept of
timeout for each operator function?
Ajay