Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8959#issuecomment-144825265
Sorry, my suggestion wasn't very clear and maybe not completely correct.
Here's the possible values for `mainJar` when that method is called:
- `Some("local:/path")`: in `Client` instance when
`spark.yarn.user.classpath.first` is true
- `Some("/path")`: in `Client` instance when
`spark.yarn.user.classpath.first` is true
- `None`: in cluster when user jar is not `local:`
- `Some("local:/path")`: in cluster when user jar is `local:`
The return value for that method for these cases should be URIs pointing to:
- `"/path"`
- `APP_JAR`
- `APP_JAR`
- `"/path"`
So I think the following code should handle that:
val mainUri = mainJar.flatMap { path =>
val uri = new URI(path)
if (uri.getScheme == LOCAL_SCHEME) Some(new URI(uri.getPath()))
else None
}.getOrElse(new URI(APP_JAR))
Hope that makes better sense now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
