1beb opened a new issue, #36007:
URL: https://github.com/apache/arrow/issues/36007
### Describe the bug, including details regarding any error messages,
version, and platform.
R version 4.3.0
Ubuntu 20.04.6
Sometimes it fails, other times it hangs, but it always takes longer than
similar methods using aws.s3 and it's unclear why. Here are some examples of
the errors I see. Otherwise it's just a significant delay like it's running
into a time out somewhere and retrying:
```
Error: IOError: AWS Error NETWORK_CONNECTION during GetObject operation:
curlCode: 28, Timeout was reached
```
```
Error in url(file, open = "rb") : URL scheme unsupported by this method
```
## Reprex:
The two scripts below, in my example point to a small 5mb file. I can run
this on a machine with arrow 8.0.0 or a machine with arrow 12.0.0 and the
result is the same (significant delay with s3:// addresses)
```
library(dotenv)
library(tictoc)
load_dot_env()
tic(); df <- read_parquet("s3://somefile.parquet"); toc()
# 70.211 sec elapsed
```
vs.
```
library(aws.s3)
library(dotenv)
load_dot_env()
load_aws <- function(f) {
read_parquet(
get_object(
object = f,
bucket = "s3://bucket-name",
region = "eu-west-1"
)
)
}
tic(); df <- load_aws("s3://somefile.parquet"); toc()
# 1.394 sec elapsed
```
On the same machines I can run a similar test in Python and the delay is not
there leading me to believe it is something explicitly with the implementation
in R. Happy to help debug! Tell me what you need.
### Component(s)
R
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
