Re: Checkpoint is not triggering as per configuration

Great, I will give a try. Thanks, Tao On Tue, May 15, 2018 at 12:50 AM, Piotr Nowojski wrote: > Hi, > > This one: https://issues.apache.org/jira/browse/FLINK-2491 > > 1. What if you set `org.apache.flink.streaming.api.functions.source. >

Re: Checkpoint is not triggering as per configuration

Hi, This one: https://issues.apache.org/jira/browse/FLINK-2491 1. What if you set `org.apache.flink.streaming.api.functions.source.FileProcessingMode#PROCESS_CONTINUOUSLY`? This will prevent split source from finishing, so checkpointing

Re: Checkpoint is not triggering as per configuration

Thanks for the reply Piotr. Which jira ticket were you refer to? We were trying to use the same code for normal stream process to process very old historical backfill data. The problem for me right now is that, backfill x years of data will be very slow. And I cannot have any checkpoint during the

Re: Checkpoint is not triggering as per configuration

Hi, It’s not considered as a bug, only a missing not yet implemented feature (check my previous responses for the Jira ticket). Generally speaking using file input stream for DataStream programs is not very popular, thus this was so far low on our priority list. Piotrek > On 10 May 2018, at

Re: Checkpoint is not triggering as per configuration

I ran into a similar issue. Since it is a "Custom File Source", the first source just listing folder/file path for all existing files. Next operator "Split Reader" will read the content of the file. "Custom File Source" went to "finished" state after first couple secs. That's way we got this

Re: Checkpoint is not triggering as per configuration

Hi, Please analyse what was going on the TaskManager and JobManager before this “task is not being executed at the moment”. What is the reason why it is not being executed? Was there some exception? Depending on your setup, you might need to check your stdout/stderr files (if your code is