Sorry colleagues,I am having a moving target in my Beam app with FlinkRunner in
a Flink Cluster of 2 nodes.Every run produces a different result while we know
what the result MUST be: its an expected fixed number.I checked and see Kafka
is NOT sending any extra records.My first suspect was Redis thread-UNsafe
hashmap objects.I replaced them with Java ConcurrentHashMaps. Wow! it worked
for the very first time every PERFECT.I then re-run exact same thing expecting
the exact same previous result.But its different. Run it again. Another
different wrong result. A different result each time.No code change nothing
different.
I am wondering if some previous data/record gets cached by
Beam/FlinkRunner/KafkaIO invocation etc. somewhere.Sorry for the long email. Am
losing my mind catching this moving target :))Appreciate your kind feedback on
this.Cheers+have a great weekend.Amir-
