srowen commented on a change in pull request #25910: [SPARK-28762][CORE] Read
JAR main class if JAR is not located in local file system
URL: https://github.com/apache/spark/pull/25910#discussion_r328678641
##########
File path:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
##########
@@ -213,24 +215,19 @@ private[deploy] class SparkSubmitArguments(args:
Seq[String], env: Map[String, S
// Try to set main class from JAR if no --class argument is given
if (mainClass == null && !isPython && !isR && primaryResource != null) {
- val uri = new URI(primaryResource)
- val uriScheme = uri.getScheme()
-
- uriScheme match {
- case "file" =>
- try {
- Utils.tryWithResource(new JarFile(uri.getPath)) { jar =>
- // Note that this might still return null if no main-class is
set; we catch that later
- mainClass =
jar.getManifest.getMainAttributes.getValue("Main-Class")
- }
- } catch {
- case _: Exception =>
- error(s"Cannot load main class from JAR $primaryResource")
- }
- case _ =>
- error(
- s"Cannot load main class from JAR $primaryResource with URI
$uriScheme. " +
- "Please specify a class through --class.")
+ try {
+ val fs = FileSystem.get(
Review comment:
To avoid having to involve Spark/Hadoop config etc in the existing code path
for "file", how about just doing this part in the 'else' case of the current
match? It may be just fine this way too, just might be more conservative. I
don't feel too strongly about it, as long as nothing breaks current
functionality.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
