[
https://issues.apache.org/jira/browse/FLINK-28069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SunShun updated FLINK-28069:
----------------------------
Description:
Hi, I intend to connect to a SSL enabled Kafka, which require to attach JKS
file for truststore and keystore, then I am trying to use the keyword {*}-yt,
yarnship{*}, to pass the JKS files, and it's expected to find the find under
the classpath of job manager or task manager.
{code:java}
val truststore_path = "client.truststore.jks"
val keystore_path = "client.keystore.jks"
val source_ddl =
s"""
CREATE TABLE source_table(
`id` BIGINT,
`time` TIMESTAMP(3)
) WITH (
'connector' = 'kafka',
...
'properties.security.protocol' = 'SSL',
'properties.ssl.truststore.location' = '$truststore_path',
'properties.ssl.truststore.password' = '$truststore_pwd',
'properties.ssl.keystore.location' = '$keystore_path',
'properties.ssl.keystore.password' = '$keystore_pwd',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
)
""" {code}
I am using the session mode to launch the job as below:
{code:java}
./bin/yarn-session.sh --detached
./bin/flink run ./job/finance-libra-stats-1.0.jar -yt ./client.keystore.jks
./client.truststore.jks{code}
However, the FileNotFound exception is given when the job initiates, saying
that the JKS files is not found from job manager.
{code:java}
Caused by: java.nio.file.NoSuchFileException: client/client.keystore.jks at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at
sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at java.nio.file.Files.newByteChannel(Files.java:361) at
java.nio.file.Files.newByteChannel(Files.java:407) at
java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
at java.nio.file.Files.newInputStream(Files.java:152) at
org.apache.kafka.common.security.ssl.SslEngineBuilder$SecurityStore.load(SslEngineBuilder.java:285)
... 21 more {code}
Anyone can help tell what's the best practice for such case, and any mistake I
meet during the process.
was:
Hi, I intend to connect to a SSL enabled Kafka, which require to attach JKS
file for truststore and keystore, then I am trying to use the keyword {*}-yt,
yarnship{*}, to pass the JKS files, and it's expected to find the find under
the classpath of job manager or task manager.
However, the FileNotFound exception is given when the job initiates, saying
that the JKS files is not found from job manager.
{code:java}
val truststore_path = "client.truststore.jks"
val keystore_path = "client.keystore.jks"
val source_ddl =
s"""
CREATE TABLE source_table(
`id` BIGINT,
`time` TIMESTAMP(3)
) WITH (
'connector' = 'kafka',
...
'properties.security.protocol' = 'SSL',
'properties.ssl.truststore.location' = '$truststore_path',
'properties.ssl.truststore.password' = '$truststore_pwd',
'properties.ssl.keystore.location' = '$keystore_path',
'properties.ssl.keystore.password' = '$keystore_pwd',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
)
""" {code}
I am using the session mode to launch the job as below:
{code:java}
./bin/yarn-session.sh --detached
./bin/flink run ./job/finance-libra-stats-1.0.jar -yt ./client.keystore.jks
./client.truststore.jks{code}
Anyone can help tell what's the best practice for such case, and any mistake I
meet during the process.
> Cannot attach SSL JKS file for Kafka connector
> ----------------------------------------------
>
> Key: FLINK-28069
> URL: https://issues.apache.org/jira/browse/FLINK-28069
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.14.4
> Environment: Flink on Yarn, session mode
> Reporter: SunShun
> Priority: Blocker
>
> Hi, I intend to connect to a SSL enabled Kafka, which require to attach JKS
> file for truststore and keystore, then I am trying to use the keyword {*}-yt,
> yarnship{*}, to pass the JKS files, and it's expected to find the find under
> the classpath of job manager or task manager.
> {code:java}
> val truststore_path = "client.truststore.jks"
> val keystore_path = "client.keystore.jks"
> val source_ddl =
> s"""
> CREATE TABLE source_table(
> `id` BIGINT,
> `time` TIMESTAMP(3)
> ) WITH (
> 'connector' = 'kafka',
> ...
> 'properties.security.protocol' = 'SSL',
> 'properties.ssl.truststore.location' = '$truststore_path',
> 'properties.ssl.truststore.password' = '$truststore_pwd',
> 'properties.ssl.keystore.location' = '$keystore_path',
> 'properties.ssl.keystore.password' = '$keystore_pwd',
> 'scan.startup.mode' = 'latest-offset',
> 'format' = 'json'
> )
> """ {code}
> I am using the session mode to launch the job as below:
> {code:java}
> ./bin/yarn-session.sh --detached
> ./bin/flink run ./job/finance-libra-stats-1.0.jar -yt ./client.keystore.jks
> ./client.truststore.jks{code}
> However, the FileNotFound exception is given when the job initiates, saying
> that the JKS files is not found from job manager.
> {code:java}
> Caused by: java.nio.file.NoSuchFileException: client/client.keystore.jks
> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
> at java.nio.file.Files.newByteChannel(Files.java:361) at
> java.nio.file.Files.newByteChannel(Files.java:407) at
> java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
> at java.nio.file.Files.newInputStream(Files.java:152) at
> org.apache.kafka.common.security.ssl.SslEngineBuilder$SecurityStore.load(SslEngineBuilder.java:285)
> ... 21 more {code}
> Anyone can help tell what's the best practice for such case, and any mistake
> I meet during the process.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
