Admin Client Configs

2023-06-21 Thread Razin Bouzar via user
Looking for some guidance on overriding Kafka admin client configs. The partition discovery feature sometimes fails to connect with kafka and we'd like to implement retries. There is no clear documentation on which settings need to be changed?admin client configs of note: -

Re: How to set hdfs configuration in flink kubernetes operator?

2023-06-21 Thread Dongwoo Kim
Hi leilinee, I'm not sure whether this is the best practice but I would like to share our experience about configuring HDFS as checkpoint storage while using flink kubernetes operator. There are two steps. *Step 1)* Mount krb5-conf & keytab file to flink kubernetes operator pod You have to

How to set hdfs configuration in flink kubernetes operator?

2023-06-21 Thread 李 琳
Hi all, Recently, I have been testing the Flink Kubernetes Operator. In the official example, the checkpoint/savepoint path is configured with a file system: state.savepoints.dir: file:///flink-data/savepoints state.checkpoints.dir: file:///flink-data/checkpoints high-availability:

Re: Changelog fail leads to job fail regardless of tolerable-failed-checkpoints config

2023-06-21 Thread Dongwoo Kim
Hi Yanfei, Thanks for the reply. So uploading changelog doesn't count as checkpointing phase, so I understood that "execution.checkpointing.tolerable-failed-checkpoints" is not related to changelog failure. However, how about making something like a tolerable-failed-changelog configuration? This

RocksDB State Backend GET returns null intermittently

2023-06-21 Thread Prabhu Joseph
Hi, RocksDB State Backend GET call on a key that was PUT into the state like 100 ms earlier but is not returned intermittently. The issue never happened with the HashDB State backend. We are trying to increase block cache size, write buffer size, and enable bloom filter as per the doc: -

Re: Default Log4j properties in Native Kubernetes

2023-06-21 Thread Yang Wang
I assume you are using "*bin/flink run-application*" to submit a Flink application to K8s cluster. Then you could simply update your local log4j-console.properties, it will be shipped and mounted to JobManager/TaskManager pods via ConfigMap. Best, Yang Vladislav Keda 于2023年6月20日周二 22:15写道: >