GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/5243
[FLINK-8362][elasticsearch] shade all dependencies
## What is the purpose of the change
The Elasticsearch connectors have some dependencies that need to be
available and should not conflict with other user or system code. Similarly to
the cassandra connector and the S3 file systems, it should thus shade all its
dependencies and become self-contained.
## Brief change log
- shade dependencies of `flink-connector-elasticsearch`,
`flink-connector-elasticsearch2`, and `flink-connector-elasticsearch5` and
relocate to our namespace
- add a test into `travis_mvn_watchdog.sh` to verify classes are shaded
(one gap though: since `flink-connector-elasticsearch` and
`flink-connector-elasticsearch-base` use the same package, we cannot really
test that the latter is shaded correctly there)
- update documentation
## Verifying this change
This can be verified as follows:
In the shaded jar files,
- check for non-shaded `service` files (also their contents)
- check for non-shaded `.class` files (this test is automated in
`travis_mvn_watchdog.sh`, but `flink-connector-elasticsearch` should be
verified manually as well - see the note above)
- check for unnecessary files in `META-INF/maven` (only
`org.apache.flink/flink-connector-elasticsearch<variant>_<scala_version>/`
should remain)
- manually test the connectors and verify they are working without adding
other dependencies to a user jar (TODO! @tzulitai since you have been working
with the connector in the past, can you chime in here?)
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): **no**
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: **no**
- The serializers: **no**
- The runtime per-record code paths (performance sensitive): **no**
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
- The S3 file system connector: **no**
## Documentation
- Does this pull request introduce a new feature? **no**
- If yes, how is the feature documented? **docs**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-8362
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5243.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5243
----
commit e0c219dadcda5a7771b20c9873006ef8ebc699b0
Author: Nico Kruber <nico@...>
Date: 2018-01-04T19:45:04Z
[build][hotfix] fix s3-fs shading checks not exiting for a non-exiting file
+ improve the output a bit
commit b1dcb6d06fed7096c0e2e8a88882037559b29923
Author: Nico Kruber <nico@...>
Date: 2018-01-04T20:09:09Z
[FLINK-8362][elasticsearch] shade all dependencies
----
---
