paleolimbot edited a comment on pull request #11730: URL: https://github.com/apache/arrow/pull/11730#issuecomment-976697765
To my reading, the duckdb package is correctly calling `arrow:::ImportRecordBatchReader()` with a pointer (casted to `uintptr_t` casted to `double`) to a `struct ArrowArrayStream` as defined in the C API (happens here: https://github.com/duckdb/duckdb/blob/master/tools/rpkg/src/statement.cpp#L615-L632 ). This gets passed to `arrow_dplyr_query()`, which seems to work for at least a dinky example and error properly if you try to `collect()` twice. Once I do some more experimenting with streams I'll investigate this particular failure in more detail. ``` r drv <- duckdb::duckdb() con <- DBI::dbConnect(drv) res <- DBI::dbSendQuery(con, "SELECT 'Hello, world!' as col1", arrow = TRUE) record_batch_reader <- duckdb::duckdb_fetch_record_batch(res) query <- arrow:::arrow_dplyr_query(record_batch_reader) dplyr::collect(query) #> # A tibble: 1 × 1 #> col1 #> <chr> #> 1 Hello, world! dplyr::collect(query) #> Error: IOError: Query Stream is closed #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/c/bridge.cc:1759 StatusFromCError(stream_.get_next(&stream_, &c_array)) #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/record_batch.h:222 ReadNext(&batch) #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/util/iterator.h:428 it_.Next() #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/compute/exec/exec_plan.cc:417 iterator_.Next() #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/record_batch.cc:318 ReadNext(&batch) #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/record_batch.cc:329 ReadAll(&batches) DBI::dbDisconnect(con, shutdown = TRUE) ``` <sup>Created on 2021-11-23 by the [reprex package](https://reprex.tidyverse.org) (v2.0.1)</sup> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
