Re: Buffering when connecting streams

Hi Alexis and David, This actually can not happen. There are mechanisms in the code to make sure none of the input is starved IF there is some data to be read. The only time when input can be blocked is during the alignment phase of aligned checkpoints under back pressure. If there was a back pre

RE: Any way to require .uid(...) calls?

Hi Dan, In case you also want to keep automatic UID assignment, we do something like this (scala): override def run(args: ApplicationArguments): Unit = { require(jobName != null, "a specific jobName needs to be configured, if hosted in Spring Boot, configure 'flink.job.name' in application.ya

Re: flink hang : es_rejected_execution_exception导致的flink 假死问题

Hi, ren I think the root cause is you didn’t set proper FailureHandler for ElasticSearch connector, the `RetryRejectedExecutionFailureHandler` can resolve your issue, you can see ElasticSearch connector docs[1] for more information. You can also set 'connector.failure-handler to 'retry-rejected

Re: enable.auto.commit=true and checkpointing turned on

Hi, 1. Yes, the kafka source will use the Kafka committed offset for the group id to start the job. 2. No, the auto.offset.reset is from Kafka consumer config, which defines what to do when there is no initial offset in

Re: [ANNOUNCE] Open source of remote shuffle project for Flink batch data processing

As one of the contributors of flink remote shuffle, I'm glad to hear all the warm responses! Welcome more people to try the flink remote shuffle and look forward to your feedback. Best， Lijie Yingjie Cao 于2021年12月1日周三 17:50写道： > Hi Jiangang, > > Great to hear that, welcome to work together to m

Re: Flinks DispatcherRestEndpoint Thread stuck even though TaskManager failed to execute job

The thread is blocked in your user-code, so whether we can unblock it depends on what said user-code is doing. On 05/12/2021 19:13, Yuval Itzchakov wrote: Hi, Flink 1.14.0, Scala 2.12 Flink on Kubernetes I use Lyfts FlinkOperator, which sets up a job cluster in Kubernetes and then submits the

Re: PyFlink import internal packages

+ CABvJ6uUPXuaKNayJ-VT7uPg-ZqDq1xzGqV8arP7RYcEosVQouA@- все Hi, thank you!it was very helpful! 03.12.2021, 12:48, "Shuiqiang Chen" :Hi, Actually, you are able to develop your app in the clean python way. It's fine to split the code into multiple files and there is no need to call `env.add_python_fi

Flinks DispatcherRestEndpoint Thread stuck even though TaskManager failed to execute job

Hi, Flink 1.14.0, Scala 2.12 Flink on Kubernetes I use Lyfts FlinkOperator, which sets up a job cluster in Kubernetes and then submits the job via the REST API. At times, the job fails. Specifically one case I am analyzing fails due to invalid state migration. I see the following error when execut

Re: Unable to create new native thread error

Hi David, Thanks for your fast response. Do you think that changing the submission method could solve the problem? Using the CLI instead of the REST API. Another question, I see that the most critical issue (FLINK-25022) is in progress and should be released on with version 1.13.4 , do you kno