Exception thrown during batch job execution on YARN even though job succeeded

2021-09-30 Thread Ken Krugler
Hi all, We’ve upgraded from Flink 1.11 to 1.13, and our workflows are now sometimes failing with an exception, even though the job has succeeded. The stack trace for this bit of the exception is: java.util.concurrent.ExecutionException:

Re: Start Flink cluster, k8s pod behavior

2021-09-30 Thread Qihua Yang
Looks like after script *flink-daemon.sh *complete, it return exit 0. Kubernetes regard it as done. Is that expected? Thanks, Qihua On Thu, Sep 30, 2021 at 11:11 AM Qihua Yang wrote: > Thank you for your reply. > From the log, exit code is 0, and reason is Completed. > Looks like the cluster

Re: Start Flink cluster, k8s pod behavior

2021-09-30 Thread Qihua Yang
Thank you for your reply. >From the log, exit code is 0, and reason is Completed. Looks like the cluster is fine. But why kubenetes restart the pod. As you said, from perspective of Kubernetes everything is done. Then how to prevent the restart? It didn't even give chance to upload and run a

Re: Start Flink cluster, k8s pod behavior

2021-09-30 Thread Qihua Yang
I did check the kubectl describe, it shows below info. Reason is Completed. Ports: 8081/TCP, 6123/TCP, 6124/TCP, 6125/TCP Host Ports:0/TCP, 0/TCP, 0/TCP, 0/TCP Command: /opt/flink/bin/entrypoint.sh Args: /opt/flink/bin/run-job-manager.sh State:

RocksDB: Spike in Memory Usage Post Restart

2021-09-30 Thread Kevin Lam
Hi all, We're debugging an issue with OOMs that occurs on our jobs shortly after a restore from checkpoint. Our application is running on kubernetes and uses RocksDB as it's state backend. We reproduced the issue on a small cluster of 2 task managers. If we killed a single task manager, we

Re: Does Flink 1.12.2 support Zookeeper version 3.6+

2021-09-30 Thread Chesnay Schepler
We only support zk 3.4/3.5 . To try another ZK version you will need to create a flink-shaded-zookeeper artifact, similar to the 3.4/3.5 version that you can find here: https://github.com/apache/flink-shaded/tree/master/flink-shaded-zookeeper-parent Once you have that it theoretically is as

RE: FlinkJobNotFoundException

2021-09-30 Thread Hailu, Andreas
Hi Matthias, the log file is quite large (21MB) so mailing it over in its entirety may have been a challenge. The file is available here [1], and we’re of course happy to share any relevant parts of it with the mailing list. I think since we’ve shared logs with you before in the past, you

Re: [ANNOUNCE] Flink mailing lists archive service has migrated to Apache Archive service

2021-09-30 Thread Robert Metzger
@Matthias Pohl : I've also been annoyed by this 30 days limit, but I'm not aware of a way to globally change the default. I would ask in #asfinfra in the asf slack. On Thu, Sep 30, 2021 at 12:19 PM Till Rohrmann wrote: > Thanks for the hint with the managed search engines Matthias. I think this

Re: [ANNOUNCE] Flink mailing lists archive service has migrated to Apache Archive service

2021-09-30 Thread Robert Metzger
@Matthias Pohl : I've also been annoyed by this 30 days limit, but I'm not aware of a way to globally change the default. I would ask in #asfinfra in the asf slack. On Thu, Sep 30, 2021 at 12:19 PM Till Rohrmann wrote: > Thanks for the hint with the managed search engines Matthias. I think this

Does Flink 1.12.2 support Zookeeper version 3.6+

2021-09-30 Thread Prasanna kumar
Hi , Does Flink 1.12.2 support Zookeeper version 3.6+ ? If we add zookeeper version 3.6 jar in the flink image ,would it be able to connect ? The following link mentions only zk 3.5 or 3.4

退订

2021-09-30 Thread Gauler Tan
麻烦退订一下订阅,谢谢

Re: [ANNOUNCE] Flink mailing lists archive service has migrated to Apache Archive service

2021-09-30 Thread Till Rohrmann
Thanks for the hint with the managed search engines Matthias. I think this is quite helpful. Cheers, Till On Wed, Sep 15, 2021 at 4:27 PM Matthias Pohl wrote: > Thanks Leonard for the announcement. I guess that is helpful. > > @Robert is there any way we can change the default setting to

Re: [ANNOUNCE] Flink mailing lists archive service has migrated to Apache Archive service

2021-09-30 Thread Till Rohrmann
Thanks for the hint with the managed search engines Matthias. I think this is quite helpful. Cheers, Till On Wed, Sep 15, 2021 at 4:27 PM Matthias Pohl wrote: > Thanks Leonard for the announcement. I guess that is helpful. > > @Robert is there any way we can change the default setting to

Re: Flink application mode with no ui , how to start job using k8s ?

2021-09-30 Thread Denis Nutiu
Hi, If you're new to k8s you can try to use Flink Native[1]. It's a CLI tool that can be used to deploy Flink in application mode or session mode but note that Reactive Mode is not supported in Flink Native. To answer your questions a) You need to bundle your jar with the Flink image or mount it

Re: Start Flink cluster, k8s pod behavior

2021-09-30 Thread Chesnay Schepler
Is the run-job-manager.sh script actually blocking? Since you (apparently) use that as an entrypoint, if that scripts exits after starting the JM then from the perspective of Kubernetes everything is done. On 30/09/2021 08:59, Matthias Pohl wrote: Hi Qihua, I guess, looking into kubectl

In flight records on Flink : Newbie question

2021-09-30 Thread Declan Harrison
Hi Guys I've just recently started using Apache Flink to evaluate its suitability for a project I'm working on. First impressions are that the project is great, well documented and has lots of examples and guidance showcasing the multitude of things that it can do. Challenging knowing where to

Re: Unable to connect to Mesos on mesos-appmaster.sh start

2021-09-30 Thread Matthias Pohl
Thanks for sharing. I was wondering why you don't use $PORT0 in your command. And: Are the ports properly configured in the Marathon network configuration [1]? But the error seems to be unrelated to that setting. Other than that, I cannot see any other issue with the configuration. It could be

Re: Start Flink cluster, k8s pod behavior

2021-09-30 Thread Matthias Pohl
Hi Qihua, I guess, looking into kubectl describe and the JobManager logs would help in understanding what's going on. Best, Matthias On Wed, Sep 29, 2021 at 8:37 PM Qihua Yang wrote: > Hi, > I deployed flink in session mode. I didn't run any jobs. I saw below logs. > That is normal, same as

Re: FlinkJobNotFoundException

2021-09-30 Thread Matthias Pohl
I didn't receive any email. But we rather not do individual support. Please share the logs on the mailing list. This way, anyone is able to participate in the discussion. Best, Matthias On Wed, Sep 29, 2021 at 8:12 PM Gusick, Doug S wrote: > Hi Matthias, > > > > Thank you for getting back. We

Re: [ANNOUNCE] Apache Flink 1.14.0 released

2021-09-30 Thread Konstantin Knauf
Big Thanks to David, Joe, Xintong and everyone who contributed to the release! Well done! Cheers, Konstantin On Thu, Sep 30, 2021 at 3:12 AM Yangze Guo wrote: > Thanks, Xintong, Joe, Dawid for the great work, thanks to everyone > involved! > > Best, > Yangze Guo > > On Thu, Sep 30, 2021 at