GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/6328
[FLINK-9816][network] add option to configure SSL engine provider for TM
communication
## What is the purpose of the change
Netty has the ability to run with different `SSLEngine` implementations but
with our current setup, we are fixed to the JDK implementation, although one
based on OpenSSL is expected to be faster [1].
We should make this configurable and ideally also provide everything needed
to run with OpenSSL in the future (the last part is not part of this PR).
[1]
https://netty.io/wiki/requirements-for-4.x.html#benefits-of-using-openssl
## Brief change log
- allow selecting the SSL engine provider via `security.ssl.provider`
- set up Netty SSL handler with its `SslContextBuilder` (in `NettyConfig`)
to have this flexibility
## Verifying this change
This change can be verified as follows:
- I verified by running an SSL setup with 2 TMs and submitting a job
through the WebUI with the default `JDK` SSL engine and `OPENSSL` using a
custom build using `netty-tcnative` with statically linked boringssl libraries
from http://netty.io/wiki/forked-tomcat-native.html
- there is an end-to-end test in #6327 which is currently blocked on the
CLI submission not working with SSL
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): **no**
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: **no**
- The serializers: **no**
- The runtime per-record code paths (performance sensitive): **no**
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
- The S3 file system connector: **no**
## Documentation
- Does this pull request introduce a new feature? **no**
- If yes, how is the feature documented? **docs, JavaDocs**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-9816
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6328.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6328
----
commit d6fb90a0c4cb1b105c482d982f8ab84744a80dc8
Author: Nico Kruber <nico@...>
Date: 2018-07-11T20:10:22Z
[hotfix][typo] fix a deprecation message
commit 83c00b9cd5e1178a1152feac04bbc7a68213deb3
Author: Nico Kruber <nico@...>
Date: 2018-07-11T21:06:04Z
[hotfix][checkstyle] fix a warning in NettyConfig
commit af8486d587a5dbc553fec42d80180a1f6ecc1571
Author: Nico Kruber <nico@...>
Date: 2018-07-11T21:05:01Z
[FLINK-9816][network] add option to configure SSL engine provider for TM
communication
This prepares Flink to use OpenSSL for TM communication channels via netty.
Currently, there is no easy way to provide the required native libraries,
though. We'll either include these in a future version of flink-shaded or
update
instructions on how to include/build them manually.
----
---
