Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19643#discussion_r148520298
--- Diff: R/pkg/R/context.R ---
@@ -319,6 +319,27 @@ spark.addFile <- function(path, recursive = FALSE) {
invisible(callJMethod(sc, "addFile",
suppressWarnings(normalizePath(path)), recursive))
}
+#' Adds a JAR dependency for Spark tasks to be executed in the future.
+#'
+#' The \code{path} passed can be either a local file, a file in HDFS (or
other Hadoop-supported
+#' filesystems), an HTTP, HTTPS or FTP URI, or local:/path for a file on
every worker node.
+#' If \code{addToCurrentClassLoader} is true, add the jar to the current
driver.
+#'
+#' @rdname spark.addJar
+#' @param path The path of the jar to be added
+#' @param addToCurrentClassLoader Whether to add the jar to the current
driver class loader.
+#' @export
+#' @examples
+#'\dontrun{
+#' spark.addJar("/path/to/something.jar", TRUE)
+#'}
+#' @note spark.addJar since 2.3.0
+spark.addJar <- function(path, addToCurrentClassLoader = FALSE) {
+ normalizedPath <- suppressWarnings(normalizePath(path))
--- End diff --
Looks we have a problem here with handling URI, Windows path, although most
of other cases should be fine though:
```r
> normalizePath("file:/C:/a/b/c")
[1] "C:\\Users\\IEUser\\workspace\\spark\\file:\\C:\\a\\b\\c"
Warning message:
In normalizePath(path.expand(path), winslash, mustWork) :
path[1]="file:/C:/a/b/c": The filename, directory name, or volume label
syntax
is incorrect
```
This looks ending up with an weird path like
`"C:\\Users\\IEUser\\workspace\\spark\\file:\\C:\\a\\b\\c"`.
I am not sure how we should handle this as this pattern ` normalizedPath <-
suppressWarnings(normalizePath(path))` looks quite common.
If it is fine, I would like to address this issue separately for other
APIs, for example, `spark.addFile` right above ..
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
