GitHub user markgrover opened a pull request:
https://github.com/apache/spark/pull/9876
[SPARK-11796] [WIP] Fixing httpclient and httpcore depedency issues related
to docker-client
I was able to get past this error:
````
*** RUN ABORTED ***
java.lang.NoSuchMethodError:
org.apache.http.impl.client.HttpClientBuilder.setConnectionManagerShared(Z)Lorg/apache/http/impl/client/HttpClientBuilder;
at
org.glassfish.jersey.apache.connector.ApacheConnector.<init>(ApacheConnector.java:240)
at
org.glassfish.jersey.apache.connector.ApacheConnectorProvider.getConnector(ApacheConnectorProvider.java:115)
at
org.glassfish.jersey.client.ClientConfig$State.initRuntime(ClientConfig.java:418)
at
org.glassfish.jersey.client.ClientConfig$State.access$000(ClientConfig.java:88)
at
org.glassfish.jersey.client.ClientConfig$State$3.get(ClientConfig.java:120)
at
org.glassfish.jersey.client.ClientConfig$State$3.get(ClientConfig.java:117)
at
org.glassfish.jersey.internal.util.collection.Values$LazyValueImpl.get(Values.java:340)
at
org.glassfish.jersey.client.ClientConfig.getRuntime(ClientConfig.java:726)
at
org.glassfish.jersey.client.ClientRequest.getConfiguration(ClientRequest.java:285)
at
org.glassfish.jersey.client.JerseyInvocation.validateHttpMethodAndEntity(JerseyInvocation.java:126)
...
````
However, then I ran into some class not found errors like these:
````
*** RUN ABORTED ***
java.lang.NoClassDefFoundError: org/apache/http/ssl/SSLContexts
at
org.apache.http.conn.ssl.SSLConnectionSocketFactory.getSocketFactory(SSLConnectionSocketFactory.java:172)
at
com.spotify.docker.client.DefaultDockerClient.getSchemeRegistry(DefaultDockerClient.java:254)
at
com.spotify.docker.client.DefaultDockerClient.getConnectionManager(DefaultDockerClient.java:242)
at
com.spotify.docker.client.DefaultDockerClient.<init>(DefaultDockerClient.java:205)
at
com.spotify.docker.client.DefaultDockerClient$Builder.build(DefaultDockerClient.java:1370)
at
org.apache.spark.sql.jdbc.DockerJDBCIntegrationSuite.beforeAll(DockerJDBCIntegrationSuite.scala:73)
at
org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
at
org.apache.spark.sql.jdbc.DockerJDBCIntegrationSuite.beforeAll(DockerJDBCIntegrationSuite.scala:58)
at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:253)
at
org.apache.spark.sql.jdbc.DockerJDBCIntegrationSuite.run(DockerJDBCIntegrationSuite.scala:58)
...
Cause: java.lang.ClassNotFoundException: org.apache.http.ssl.SSLContexts
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at
org.apache.http.conn.ssl.SSLConnectionSocketFactory.getSocketFactory(SSLConnectionSocketFactory.java:172)
at
com.spotify.docker.client.DefaultDockerClient.getSchemeRegistry(DefaultDockerClient.java:254)
at
com.spotify.docker.client.DefaultDockerClient.getConnectionManager(DefaultDockerClient.java:242)
````
Anyways, finally after trying a few more things and going through
@JoshRosen 's previous discussion with docker-client folks, this patch is what
I finally came up with.
It goes past all the ClassNotFound errors but on my mac, it gives 404
errors. So, I am trying to post the patch here, in case it doesn't pass on my
local environment.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/markgrover/spark master_docker
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/9876.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #9876
----
commit 6dfff89f7a31087c03f62de03ac952974ec30505
Author: Mark Grover <[email protected]>
Date: 2015-11-20T23:46:24Z
Fixing httpclient and httpcore depedency issues related to docker-client
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
