Hello Flink Community! We have a fairly intensive flink streaming application, processing 8-9 million records a minute, with each record being 10k. One of our steps is a keyBy operation. We are finding that flink lags seriously behind when we introduce the keyBy (presumably because of shuffle across the network).
We are trying to tune it ourselves (size of nodes, memory, network buffers etc), but before we spend way too much time on this; would it be better to hire some “flink tuning expert” to get us through? If so what resources are recommended on this list? Cheers Kumar