azagrebin commented on a change in pull request #10932: [FLINK-15614][docs]
Consolidate Hadoop documentation
URL: https://github.com/apache/flink/pull/10932#discussion_r372394242
##########
File path: docs/ops/deployment/hadoop.md
##########
@@ -66,6 +71,29 @@ in the shell. Note that `hadoop` is the hadoop binary and
that `classpath` is an
Putting the Hadoop configuration in the same class path as the Hadoop
libraries makes Flink pick up that configuration.
+### Adding Hadoop to /lib
+
+The Flink project releases Hadoop distributions for specific versions, that
relocate or exclude several dependencies
+to reduce the risk of dependency clashes.
+These can be found on the [downloads]({{ site.download_url }}).
+For these versions it is sufficient to download the corresponding `Pre-bundled
Hadoop` component and placing it in
+the `/lib` directory of the Flink distribution.
+
+If the used Hadoop version is not listed on the download page (possibly due to
being a Vendor-specific version),
+then it is necessary to build
[flink-shaded](https://github.com/apache/flink-shaded) against this version.
+You can find the source for this project in the [Additional Components]({{
site.download_url }}#additional-components) section of the download page.
+
+<span class="label label-info">Note</span> If you want to build `flink-shaded`
against a vendor specific Hadoop version, you first have to configure the
+vendor-specific maven repository in your local maven setup as described
[here](https://maven.apache.org/guides/mini/guide-multiple-repositories.html).
+
+Run the following command to build and install `flink-shaded` against your
desired Hadoop version (e.g., for version `2.6.5-custom`):
+
+{% highlight bash %}
+mvn clean install -Dhadoop.version=2.6.5-custom
+{% endhighlight %}
+
+After this step is complete, place the `flink-shaded-hadoop-2-uber` jar in the
`/lib` directory of the Flink distribution.
Review comment:
I would also suggest to have an expected path to the jar, like
`target/flink-shaded-hadoop-2-uber.jar` for less advanced users.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
