Hi, You can give specific IAM instance roles to the instances running Flink. This way you never expose access keys anywhere. As the docs say, that is the recommended way (and not just for Flink, but for any service you want to use, never set it up with AWS credentials in config). IAM will transparently deal with the security, and you can be extremely restrictive on what policies you attach to the instance roles.
Cheers, Bruno On Thu, 7 Feb 2019 at 13:38, Kostas Kloudas <kklou...@gmail.com> wrote: > Hi Antonio, > > I am cc'ing Till who may have something to say on this. > > Cheers, > Kostas > > On Thu, Feb 7, 2019 at 1:32 PM Antonio Verardi <anto...@yelp.com> wrote: > >> Hi there, >> >> I'm trying out to run Flink on Kubernetes and I run into a problem with >> the way Flink sets up AWS credentials to talk with S3 and the way we manage >> AWS secrets in my company. >> >> To give permissions to Flink I am using AWS keys embedded in flink.conf, >> as per >> https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/aws.html#configure-access-credentials. >> The problem there is that we rotate daily our AWS keys in order to mitigate >> any eventual leak of keys. In order to make Flink pick up the new keys I >> understand I have to restart it, but that means downtime, especially for >> the jobs which have a large state to save. >> >> I know that in Kubernetes land there are these two projects, >> https://github.com/uswitch/kiam and https://github.com/jtblin/kube2iam >> <https://github.com/jtblin/kube2iamm>, that make possible to associate >> IAM policies to pods/containers. But they are not part of the "official" >> Kubernetes software, which kinda surprises me. >> >> Did anyone run into a similar problem? If so, how did you solve it? >> >> Cheers, >> Antonio >> >
