Re: Using filesystem plugin with MiniCluster

2023-01-03 Thread Yaroslav Tkachenko
Hey Ken, I have flink-s3-fs-hadoop as a provided dependency in my project, and I've configured my IDE to include provided dependencies when starting applications. Works just fine. On Tue, Jan 3, 2023 at 5:06 PM Ken Krugler wrote: > Hi all, > > With Flink 1.15.x, is there a way to use the S3

Using filesystem plugin with MiniCluster

2023-01-03 Thread Ken Krugler
Hi all, With Flink 1.15.x, is there a way to use the S3 Presto plugin when running code on the MiniCluster? I can’t just add that jar as a dependency when testing, as I get: java.lang.NoClassDefFoundError: Could not initialize class com.facebook.presto.hive.s3.PrestoS3FileSystem at

Re: The use of zookeeper in flink

2023-01-03 Thread Yang Wang
The reason why the running jobs try to failover with zookeeper outage is that the JobManager lost leadership. Having a standby JobManager or not makes no difference. Best, Yang Matthias Pohl via user 于2023年1月2日周一 20:51写道: > And I screwed up the reply again. -.- Here's my previous response for

Re: How to get failed streaming Flink job log in Flink Native K8s mode?

2023-01-03 Thread Yang Wang
I think you might need a sidecar container or daemonset to collect the Flink logs and store into a persistent storage. You could find more information here[1]. [1]. https://www.alibabacloud.com/blog/best-practices-of-kubernetes-log-collection_596356 Best, Yang hjw 于2022年12月22日周四 23:28写道: > On