Hi Cam, could you share a bit more details about your job (e.g. which sources are you using, what are your settings, etc.). Ideally you can provide a minimal example in order to better understand the program.
>From a high level perspective, there might be different problems: First of all, Flink does not support checkpointing/taking a savepoint if some of the job's operator have already terminated iirc. But your description points rather into the direction that your bounded source does not terminate. So maybe you are reading a file via StreamExecutionEnvironment.createFileInput with FileProcessingMode.PROCESS_CONTINUOUSLY. But these things are hard to tell without a better understanding of your job. Cheers, Till On Mon, Jan 13, 2020 at 8:35 PM Cam Mach <cammac...@gmail.com> wrote: > Hello Flink expert, > > We have a pipeline that read both bounded and unbounded sources and our > understanding is that when the bounded sources complete they should get a > watermark of +inf and then we should be able to take a savepoint and safely > restart the pipeline. However, we have source that never get watermarks and > we are confused as to what we are seeing and what we should expect > > > Cam Mach > Software Engineer > E-mail: cammac...@gmail.com > Tel: 206 972 2768 > >
