[
https://issues.apache.org/jira/browse/FLINK-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15139694#comment-15139694
]
ASF GitHub Bot commented on FLINK-3373:
---------------------------------------
GitHub user StephanEwen opened a pull request:
https://github.com/apache/flink/pull/1615
[FLINK-3373] [build] Shade away Hadoop's HTTP Components dependency
This makes the HTTP Components dependency disappear from the core
classpath, allowing users to use their own version of the dependency.
We need shading because we cannot simply bump the HTTP Components version
to the newest version. The YARN test for Hadoop version >= 2.6.0 fail in that
case.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink http_shade
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1615.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1615
----
commit 1be39d12071c7251cd566e692c3a9c7b5440e46d
Author: Stephan Ewen <[email protected]>
Date: 2016-02-09T20:18:43Z
[FLINK-3373] [build] Shade away Hadoop's HTTP Components dependency
----
> Using a newer library of Apache HttpClient than 4.2.6 will get class loading
> problems
> -------------------------------------------------------------------------------------
>
> Key: FLINK-3373
> URL: https://issues.apache.org/jira/browse/FLINK-3373
> Project: Flink
> Issue Type: Bug
> Environment: Latest Flink snapshot 1.0
> Reporter: Jakob Sultan Ericsson
>
> When I trying to use Apache HTTP client 4.5.1 in my flink job it will crash
> with NoClassDefFound.
> This has to do that it load some classes from provided httpclient 4.2.5/6 in
> core flink.
> {noformat}
> 17:05:56,193 INFO org.apache.flink.runtime.taskmanager.Task
> - DuplicateFilter -> InstallKeyLookup (11/16) switched to FAILED with
> exception.
> java.lang.NoSuchFieldError: INSTANCE
> at
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.<clinit>(SSLConnectionSocketFactory.java:144)
> at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.getDefaultRegistry(PoolingHttpClientConnectionManager.java:109)
> at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:116)
> ...<internal classes>
> at
> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
> at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:89)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:305)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:227)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> SSLConnectionSocketFactory and finds an earlier version of the
> AllowAllHostnameVerifier that does have the INSTANCE variable (instance
> variable was probably added in 4.3).
> {noformat}
> jar tvf lib/flink-dist-1.0-SNAPSHOT.jar |grep AllowAllHostnameVerifier
> 791 Thu Dec 17 09:55:46 CET 2015
> org/apache/http/conn/ssl/AllowAllHostnameVerifier.class
> {noformat}
> Solutions would be:
> - Fix the classloader so that my custom job does not conflict with internal
> flink-core classes... pretty hard
> - Remove the dependency somehow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
