Hello to all,
after debugging an issue in SPARK
(https://issues.apache.org/jira/browse/SPARK-11085)
I wonder whether there isn't a bug in ivy.

In
https://github.com/apache/spark/blob/0a3026990bd0cbad53f0001da793349201104958/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L904
we have two resolvers, configured as

1. br.setName("central")
2. sp.setName("spark-packages")

When executing*
% spark-shell --packages org.apache.spark:spark-streaming-
kafka_2.11:1.6.1,com.datastax.spark:spark-cassandra-connector_2.11:1.6.1-M2
--driver-java-options "-Dhttp.proxyHost=<proxyHost>
-Dhttp.proxyPort=<proxyPort>"

==== spark-packages: tried
module not found: com.datastax.spark#spark-cassandra-connector_2.11;1.6.1-M2
(http://dl.bintray.com/spark-packages/maven/org/apache/spark/spark-streaming-kafka_2.11/1.6.1/spark-streaming-kafka_2.11-1.6.1.pom
does not exist)

==== central: tried
ERRORS
Server access error at url
https://repo1.maven.org/maven2/org/apache/spark/spark-streaming-kafka_2.11/1.6.1/spark-streaming-kafka_2.11-1.6.1.pom(java.net.ConnectException:
Connection timed out)

After reading the logs and observing a tcpdump, it seems
1. br: did not use the proxy
2. sp: did use the proxy

Do you have an idea why ?
Have a good day !

*: I used that docker https://github.com/Yannael/kafka-sparkstreaming-cassandra


-- 
Ion

Reply via email to