Re: Stream job failed after increasing number retained checkpoints

ick math: > > 2880 checkpoints * 10 task managers * 10 operators in the job * 8 > parallelism per task manager * 500 bytes = ~1GB > > The answer might be that you just need to increase the Job Manager max > heap to retain 2880 checkpoints. > > Piotrek > > On 10 Jan 20

Re: Stream job failed after increasing number retained checkpoints

t; - long GC pauses or other blocking operations (possibly long unnatural > gaps in the logs) > - machine health (CPU usage, disks usage, network connections) > > Piotrek > > On 9 Jan 2018, at 16:38, Jose Miguel Tejedor Fernandez < > jose.fernan...@rovio.com> wrote: > &

Lost data when resuming from savepoint

Hi, I am running a simple stream Flink job (Flink version 1.3.2 and 1.3.1) whose source and sink is a Kafka cluster 0.10.0.1. I am testing savepoints by stopping/resuming the job and when I checked the validity of the data sunk during the stop time I observed that some of the events have been los

set state.checkpoint.dir to execution environment

Hi, After enabling checkpoints and set the property env.setStateBackend(new FsStateBackend(url)) I am the following exception: Caused by: java.lang.IllegalStateException: CheckpointConfig says to persist periodic checkpoints, but no checkpoint directory has been configured. You can configure conf

Modify field topics (KafkaConsumer) during runtime

Hi, I am using Flink version 1.3.1. I am wondering if it is possible to add/delete new topics to FlinkKafkaConsumer during execution of a job? Otherwise, I guess I need to cancel the job and redeploy the new job. Cheers BR

Set Savepoints configuration after cluster bootstrap

Hi, I am using Flink v1.3.1. My question is about how to set the configuration for the savepoints feature. As long as I know the configuration entry `state.savepoints.dir` must be set in the file flink/conf/flink-conf.yaml But I would like to know if it is possible to programmatically set/modif