[
https://issues.apache.org/jira/browse/SPARK-12825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098139#comment-15098139
]
Jayadevan M commented on SPARK-12825:
-------------------------------------
Can you tell the full command.
> Spark-submit Jar URL loading fails on redirect
> ----------------------------------------------
>
> Key: SPARK-12825
> URL: https://issues.apache.org/jira/browse/SPARK-12825
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.6.0
> Reporter: Alex Nederlof
> Priority: Minor
>
> When you use spark-submit, and pass the jar as a URL, it fails when the URL
> redirects.
> The log prints:
> {code}
> 16/01/14 14:26:43 INFO Utils: Fetching http://myUrl/my.jar to
> /opt/spark/spark-1.6.0-bin-hadoop2.6/work/driver-20160114142642-0010/fetchFileTemp8495494631100918254.tmp
> {code}
> However, that file doesn't exist, but a file called "redirect" is created,
> with the appropriate content.
> After that, the driver fails with
> {code}
> 16/01/14 14:26:43 WARN Worker: Driver driver-20160114142642-0010 failed with
> unrecoverable exception: java.lang.Exception: Did not see expected jar my.jar
> in /opt/spark/spark-1.6.0-bin-hadoop2.6/work/driver-20160114142642-0010
> {code}
> Here's the related code:
> https://github.com/apache/spark/blob/56cdbd654d54bf07a063a03a5c34c4165818eeb2/core/src/main/scala/org/apache/spark/util/Utils.scala#L583-L603
> My Scala chops aren't up to this challenge, otherwise I would have made a
> patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
