Hey Nico, thanks for your reply. I gave this a try and unfortunately had no luck.
// ah -----Original Message----- From: Nico Kruber <n...@ververica.com> Sent: Wednesday, April 21, 2021 1:01 PM To: user@flink.apache.org Subject: Re: [1.9.2] Flink SSL on YARN - NoSuchFileException Hi Andreas, judging from [1], it should work if you refer to it via security.ssl.rest.keystore: ./deploy-keys/rest.keystore security.ssl.rest.truststore: ./deploy-keys/rest.truststore Nico [1] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-KAFKA-KEYTAB-Kafkaconsumer-error-Kerberos-td37277.html On Monday, 19 April 2021 16:45:25 CEST Hailu, Andreas [Engineering] wrote: > Hi Flink team, > > I'm trying to configure a Flink on YARN with SSL enabled. I've > followed the documentation's instruction [1] to generate a Keystore > and Truststore locally, and added a the properties to my flink-conf.yaml. > security.ssl.rest.keystore: /home/user/ssl/deploy-keys/rest.keystore > security.ssl.rest.truststore: > /home/user/ssl/deploy-keys/rest.truststore > > I've also added the yarnship option so that the keystore and > truststore are deployed as suggested in [1]. > > -m yarn-cluster --class <class> [...] -yt /home/user/ssl/deploy-keys/ > > However, starting the Flink cluster results in a NoSuchFileException, > Caused by: java.nio.file.NoSuchFileException: > /home/user/ssl/deploy-keys/rest.keystore at > sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvide > r.jav > a:214) at java.nio.file.Files.newByteChannel(Files.java:361) > at java.nio.file.Files.newByteChannel(Files.java:407) > at > java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider > .java > :384) at java.nio.file.Files.newInputStream(Files.java:152) > at > org.apache.flink.runtime.net.SSLUtils.getKeyManagerFactory(SSLUtils.ja > va:26 > 6) at > org.apache.flink.runtime.net.SSLUtils.createRestNettySSLContext(SSLUti > ls.ja > va:392) at > org.apache.flink.runtime.net.SSLUtils.createRestNettySSLContext(SSLUti > ls.ja > va:365) at > org.apache.flink.runtime.net.SSLUtils.createRestServerSSLEngineFactory > (SSLU > tils.java:163) at > org.apache.flink.runtime.rest.RestServerEndpointConfiguration.fromConf > igura > tion(RestServerEndpointConfiguration.java:160) > > I'm able to see in launch_container.sh that the shipped directory was > able to be created successfully: > > mkdir -p deploy-keys > ln -sf > "/fs/htmp/yarn/local/usercache/delp/appcache/application_1618711298408 > _2664 /filecache/16/rest.truststore" "deploy-keys/rest.truststore" > mkdir -p deploy-keys ln -sf > "/fs/htmp/yarn/local/usercache/delp/appcache/application_1618711298408 > _2664 /filecache/13/rest.keystore" "deploy-keys/rest.keystore" > > So given the above logs, I tried editing flink-conf.yaml to reflect > what I > saw: security.ssl.rest.keystore: deploy-keys/rest.keystore > security.ssl.rest.truststore: deploy-keys/rest.truststore > > But that didn't seem to work, either: > Caused by: java.nio.file.NoSuchFileException: deploy-keys/rest.truststore > at > sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvide > r.jav > a:214) at java.nio.file.Files.newByteChannel(Files.java:361) > at java.nio.file.Files.newByteChannel(Files.java:407) > at > java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider > .java > :384) at java.nio.file.Files.newInputStream(Files.java:152) > at > org.apache.flink.runtime.net.SSLUtils.getTrustManagerFactory(SSLUtils.java: > 233) at > org.apache.flink.runtime.net.SSLUtils.createRestNettySSLContext(SSLUti > ls.ja > va:397) at > org.apache.flink.runtime.net.SSLUtils.createRestNettySSLContext(SSLUti > ls.ja > va:365) at > org.apache.flink.runtime.net.SSLUtils.createRestClientSSLEngineFactory > (SSLU > tils.java:181) at > org.apache.flink.runtime.rest.RestClientConfiguration.fromConfiguratio > n(Res > tClientConfiguration.java:106) > > What needs to be done to get the YARN application to point to the > right keystore and truststore? > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/securi > ty-ss l.html#tips-for-yarn--mesos-deployment > > ____________ > > Andreas Hailu > Data Lake Engineering | Goldman Sachs & Co. > > > ________________________________ > > Your Personal Data: We may collect and process information about you > that may be subject to data protection laws. For more information > about how we use and disclose your personal data, how we protect your > information, our legal basis to use your information, your rights and > who you can contact, please refer to: > www.gs.com/privacy-notices<http://www.gs.com/privacy-notices> -- Dr. Nico Kruber | Solutions Architect Follow us @VervericaData Ververica -- Join Flink Forward - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Yip Park Tung Jason, Jinwei (Kevin) Zhang, Karl Anton Wehner ________________________________ Your Personal Data: We may collect and process information about you that may be subject to data protection laws. For more information about how we use and disclose your personal data, how we protect your information, our legal basis to use your information, your rights and who you can contact, please refer to: www.gs.com/privacy-notices<http://www.gs.com/privacy-notices>
