HyukjinKwon commented on a change in pull request #26555: [SPARK-29378][R]
Upgrade SparkR to use Arrow 0.15 API
URL: https://github.com/apache/spark/pull/26555#discussion_r348255537
##########
File path: R/pkg/R/SQLContext.R
##########
@@ -166,9 +166,9 @@ writeToFileInArrow <- function(fileName, rdf,
numPartitions) {
for (rdf_slice in rdf_slices) {
batch <- arrow::record_batch(rdf_slice)
if (is.null(stream_writer)) {
- stream <- arrow::FileOutputStream(fileName)
+ stream <- arrow::FileOutputStream$create(fileName)
schema <- batch$schema
- stream_writer <- arrow::RecordBatchStreamWriter(stream, schema)
+ stream_writer <- arrow::RecordBatchStreamWriter$create(stream,
schema)
Review comment:
Actually, I configured AppVeyor to test Arrow related ones in R for now. So,
at least it's being tested in AppVeyor.
Yes, it would be nice if we have Arrow R installed in Jenkins workers to
test it out; however, I remember it's a bit tricky to install Arrow R library
for now (you cannot just install via CRAN but needs some manual preparation -
see https://github.com/apache/arrow/tree/master/r#installation).
I am worried if it messes up the current Jenkins worker's environment. If
you see any concern, @shaneknapp, you can just don't install it for now. It's
being tested via AppVeyor at least so it's neither a must or urgent.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
