xkrogen commented on a change in pull request #28874:
URL: https://github.com/apache/spark/pull/28874#discussion_r445149855
##########
File path: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala
##########
@@ -1256,19 +1256,19 @@ class SparkSubmitSuite
val jars = conf.get("spark.yarn.dist.jars").split(",").toSet
- def isSchemeBlacklisted(scheme: String) = {
- blacklistSchemes.contains("*") || blacklistSchemes.contains(scheme)
+ def isSchemeForcedDownload(scheme: String) = {
Review comment:
The schemes considered here force a download-and-reupload for use with
the YARN cache as opposed to allowing them to remain in their original
location. I think these lines pretty clearly indicate that this method and the
related variables are really a force-download-list:
```
val forceDownloadArgs = if (blacklistSchemes.nonEmpty) { val
forceDownloadArgs = if (forceDownloadSchemas.nonEmpty) {
Seq("--conf",
s"spark.yarn.dist.forceDownloadSchemes=${blacklistSchemes.mkString(",")}")
```
However I just noticed that I incorrectly named this as
`forceDownloadSchemas` instead of `forceDownloadSchemes`, I will fix this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
