Hi, We have created Batch job where we are trying to merge set of S3 directories in TextFormat with the old snapshot in Parquet format.
We are running 50 such jobs daily and found the progress of few random jobs get stuck in between. We have gone through logs of JobManager, TaskManager and could not get any useful information there. Important operators involved, are read using TextInputFormat, read using HadoopInputFormat, FullOuterJoin, write using our BucketingSink code. Please help resolve this issue. Flink Version 1.3.2 deployed on Yarn Container -- Thanks, Amit
