paleolimbot commented on PR #985:
URL: https://github.com/apache/arrow-adbc/pull/985#issuecomment-2108574928
Nice! I have a concrete idea in my head about how the first things you
described might go but I'm stuck as to how to compose fetching an indeterminate
number of results by composing promises (without potentially costly overhead of
waiting for a promise to resolve before launching another thread and waiting
for *that*). Apologies if there's a simple solution there!
```r
some_con |>
adbc_async_prepare() %...>%
adbc_async_execute_query() %...>%
# What happens here?
(function(array_stream) { what_next() })
```
> Why do we need to tightly couple execution and data retrieval?
I have a more concrete idea in my head about how one might fetch an entire
result (as a vector of C-level arrays) and then call a callback when that has
completed. For example:
```r
some_con |>
adbc_async_prepare() %...>%
adbc_async_execute_and_read_all() %...>%
# This would be a basic_array_stream whose arrays have already been fetched
nanoarrow::convert_array_stream()
```
Ideally we'd be building up the data frame result on the R thread while
waiting for the next batch (but the infrastructure for that isn't merged into
nanoarrow yet).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
