Hi Adam, the flink binary will pick up any configuration from the flink-conf.yaml of its directory. If that is the same as in the cluster, you wouldn't have to pass most of your parameters manually. However, if you prefer not having a flink-conf.yaml in place, you could remove the security.ssl.internal.* parameter from its call since those only affect internal communication.
If the client's connection to the JM is denied, you would actually have this in the JM logs as well which you could check. To check whether your whole setup works, I would suggest to try without security enabled first and then enable it (just to rule out any other issues) >From the commands you mentioned, it looks like you're just missing security.ssl.rest.enabled=true and because of that, the client would not use SSL for the connection. For more information and setup, I recommend reading through [1] which also contains an example at the bottom of the page and how to use curl to test or use the REST endpoint. Nico [1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/security-ssl.html On Tuesday, 25 August 2020 14:40:04 CEST Adam Roberts wrote: > Hey everyone, I've been experimenting with Flink > using https://github.com/GoogleCloudPlatform/flink-on-k8s-operator and I > believe I've successfully deployed a JobManager and TaskManager with > security enabled, and a self-signed certificate (the pods come up great). > However, I can't do much with this - I can't port-forward and access the UI, > nor can I submit jobs to it by running another pod and using the DNS name > lookup of the service. > I always get > > The program finished with the following exception: > org.apache.flink.client.program.ProgramInvocationException: The main method > caused an error: java.util.concurrent.ExecutionException: > org.apache.flink.runtime.client.JobSubmissionException: Failed to submit > JobGraph. > ... > > Caused by: org.apache.flink.runtime.rest.ConnectionClosedException: Channel > became inactive. ... 37 more > > > and this is even with all of the -D security options provided. > > The versions of Flink are the same for both my Job and my FlinkCluster > (1.11.1). > Is this a sensible thing to do? If I weren't using the operator for example, > would users be expected to flink run with all of these options? > Does anything look odd here? My guess is because security's on, the Job > Manager refuses to talk to my submitter. > Running as the flink user in the container, I do > > > securityContext: > > runAsUser: 9999 > > runAsGroup: 9999 > > containers: > > - name: wordcount > > image: adamroberts/mycoolflink:latest > > args: > > - /opt/flink/bin/flink > > - run > > - -D > > - > security.ssl.rest.keystore=/etc/flink-secrets/flink-tls-keystore.key > > - -D > > - > security.ssl.rest.truststore=/etc/flink-secrets/flink-tls-ca-truststore.jks > > - -D > > - security.ssl.rest.keystore-password=thepass # Replace with value > of flink-tls-keystore.password > > - -D > > - security.ssl.rest.key-password=thepass # Replace with value of > tls.p12.password > > - -D > > - security.ssl.rest.truststore-password=thepass # Replace with value > of flink-tls-ca.truststore.password > > - -D > > - > security.ssl.internal.keystore=/etc/flink-secrets/flink-tls-keystore.key > > - -D > > - > security.ssl.internal.truststore=/etc/flink-secrets/flink-tls-ca-truststore > .jks > > - -D > > - security.ssl.internal.keystore-password=thepass # Replace with > value of flink-tls-keystore.password > > - -D > > - security.ssl.internal.key-password=thepass # Replace with value of > flink-tls-keystore.password > > - -D > > - security.ssl.internal.truststore-password=thepass # Replace with > value of flink-tls-truststore.password > > - -m > > - tls-flink-cluster-1-11-jobmanager:8081 > > - /opt/flink/examples/batch/WordCount.jar > > - --input > > - /opt/flink/NOTICE > > > with the secrets mounted in at the above location (if I exec into my > container, I can see they're all there OK). Note that it is a read-only > file system. > adamroberts/mycoolflink (at this time of this email) is just based > on https://github.com/apache/flink-docker. > Thanks! > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire > PO6 3AU
