[
https://issues.apache.org/jira/browse/SPARK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lucian Timar updated SPARK-34225:
---------------------------------
Description:
{code:java}
c:>spark-shell --files "c:\Program%20Files\...\myjar.jar"
or
c:>spark-shell --jars"c:\Program%20Files\...\myjar.jar"
or
c:>spark-shell --conf spark.jars="c:\Program%20Files\...\myjar.jar"
any combination produce the same exception
java.io.FileNotFoundException: Jar c:\Program%20Files\........ not found
at
org.apache.spark.SparkContext.addLocalJarFile$1(SparkContext.scala:1833)
at org.apache.spark.SparkContext.addJar(SparkContext.scala:1887)
at org.apache.spark.SparkContext.$anonfun$new$11(SparkContext.scala:490)
at
org.apache.spark.SparkContext.$anonfun$new$11$adapted(SparkContext.scala:490)
at
scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at
scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:490)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2574)
at
org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:934)
at scala.Option.getOrElse(Option.scala:189)
at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:928)
at org.apache.spark.repl.Main$.createSparkSession(Main.scala:106)
{code}
I have noticed that this is caused by
{code:java}
def addJar(path: String): Unit = {
...
} else {
val uri = new Path(path).toUri
{code}
the path as the string is
"file:///C:/Program%20Files/Nokia/....jar"
this call generates the following uri
"file:///C:/Program*%25*20Files/Nokia/....jar"
which results in an invalid file name.
Maybe using
{code:java}
Utils.resolveURI(path){code}
resolves this issue.
Are there any workarounds to solve this issue ?
> Jars or file paths which contain spaces are generating FileNotFoundException
> exception
> --------------------------------------------------------------------------------------
>
> Key: SPARK-34225
> URL: https://issues.apache.org/jira/browse/SPARK-34225
> Project: Spark
> Issue Type: Bug
> Components: Spark Submit, Windows
> Affects Versions: 3.0.1
> Reporter: Lucian Timar
> Priority: Major
>
>
> {code:java}
> c:>spark-shell --files "c:\Program%20Files\...\myjar.jar"
> or
> c:>spark-shell --jars"c:\Program%20Files\...\myjar.jar"
> or
> c:>spark-shell --conf spark.jars="c:\Program%20Files\...\myjar.jar"
> any combination produce the same exception
> java.io.FileNotFoundException: Jar c:\Program%20Files\........ not found
> at
> org.apache.spark.SparkContext.addLocalJarFile$1(SparkContext.scala:1833)
> at org.apache.spark.SparkContext.addJar(SparkContext.scala:1887)
> at
> org.apache.spark.SparkContext.$anonfun$new$11(SparkContext.scala:490)
> at
> org.apache.spark.SparkContext.$anonfun$new$11$adapted(SparkContext.scala:490)
> at
> scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
> at
> scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:490)
> at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2574)
> at
> org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:934)
> at scala.Option.getOrElse(Option.scala:189)
> at
> org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:928)
> at org.apache.spark.repl.Main$.createSparkSession(Main.scala:106)
> {code}
> I have noticed that this is caused by
> {code:java}
> def addJar(path: String): Unit = {
> ...
> } else {
> val uri = new Path(path).toUri
> {code}
> the path as the string is
> "file:///C:/Program%20Files/Nokia/....jar"
> this call generates the following uri
> "file:///C:/Program*%25*20Files/Nokia/....jar"
> which results in an invalid file name.
> Maybe using
> {code:java}
> Utils.resolveURI(path){code}
> resolves this issue.
>
> Are there any workarounds to solve this issue ?
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
