jorisvandenbossche commented on a change in pull request #8058:
URL: https://github.com/apache/arrow/pull/8058#discussion_r486167891
##########
File path: r/R/filesystem.R
##########
@@ -242,11 +242,31 @@ FileSystem <- R6Class("FileSystem", inherit = ArrowObject,
)
)
FileSystem$from_uri <- function(uri) {
+ assert_that(is.string(uri))
out <- fs___FileSystemFromUri(uri)
out$fs <- shared_ptr(FileSystem, out$fs)$..dispatch()
out
}
+get_path_and_filesystem <- function(x, filesystem = NULL) {
+ # Wrapper around FileSystem$from_uri that handles local paths
+ # and an optional explicit filesystem
+ assert_that(is.string(x))
+ if (is_url(x)) {
+ if (!is.null(filesystem)) {
+ # Stop? Can't have URL (which yields a fs) and another fs
+ }
+ FileSystem$from_uri(x)
+ } else {
+ list(
+ fs = filesystem %||% LocalFileSystem$create(),
+ path = clean_path_abs(x)
+ )
+ }
+}
+
+is_url <- function(x) grepl("://", x)
Review comment:
You might want to double check that the `get_path_and_filesystem`
doesn't break local relative file paths (as I did in the python version ..
;-)), but I suppose this `is_url` check before calling `from_uri` will indeed
prevent that issue.
##########
File path: r/R/filesystem.R
##########
@@ -242,11 +242,31 @@ FileSystem <- R6Class("FileSystem", inherit = ArrowObject,
)
)
FileSystem$from_uri <- function(uri) {
+ assert_that(is.string(uri))
out <- fs___FileSystemFromUri(uri)
out$fs <- shared_ptr(FileSystem, out$fs)$..dispatch()
out
}
+get_path_and_filesystem <- function(x, filesystem = NULL) {
+ # Wrapper around FileSystem$from_uri that handles local paths
+ # and an optional explicit filesystem
+ assert_that(is.string(x))
+ if (is_url(x)) {
+ if (!is.null(filesystem)) {
+ # Stop? Can't have URL (which yields a fs) and another fs
Review comment:
Indeed, the user shouldn't pass both a filesystem object and a URI, as
then things like `filesystem$OpenInputFile(uri)` will fail (that was the root
cause of the python parquet s3 test crashes ->
https://github.com/apache/arrow/pull/8139). So I think can either let it pass
here and rely on a later error in eg OpenInputFile, or otherwise raise a
(potentially more informative) error message here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
