Hey, I am new to flink and I have a question and want to see if anyone can help here.
So we have a s3 path that flink is monitoring that path to see new files available. val avroInputStream_activity = env.readFile(format, path, FileProcessingMode.PROCESS_CONTINUOUSLY, 10000) I am doing both internal and external check pointing and let's say there is a bad file came to the path and flink will do several retries. I want to take those bad files and let the process continue. However, since the file path persist in the checkpoint, when I try to resume from external checkpoint, it threw the following error on no file been found. java.io.IOException: Error opening the Input Split s3a://myfile [0,904]: No such file or directory: s3a://myfile Is there a way to skip this bad file and move on? Thanks in advance. Best, Chengzhi Zhao
