paleolimbot opened a new pull request, #985: URL: https://github.com/apache/arrow-adbc/pull/985
This is a follow-up to https://github.com/apache/arrow-nanoarrow/pull/211 , which was closed because the async functionality for interacting with the array stream is more closely tied to the potential async functionality of other ADBC methods. This can/should remain in draft form for some time...async is hard and this PR is currently intended to act as a starting place to discuss what this would look like. Thanks to @krlmlr for pushing for this functionality! A reprex to get started: ``` r # pak::pak("apache/arrow-adbc/r/adbcdrivermanager#984") queue <- adbcdrivermanager:::adbc_callback_queue() # Add a few callbacks stream <- nanoarrow::basic_array_stream(list(1:5)) adbcdrivermanager:::adbc_array_stream_get_next_async(stream, function(array) { message("Done 1!") print(as.vector(array)) }, queue = queue) stream <- nanoarrow::basic_array_stream(list(6:10)) adbcdrivermanager:::adbc_array_stream_get_next_async(stream, function(array) { message("Done 2!") print(as.vector(array)) }, queue = queue) stream <- nanoarrow::basic_array_stream(list(11:15)) adbcdrivermanager:::adbc_array_stream_get_next_async(stream, function(array) { message("Done 3!") print(as.vector(array)) }, queue = queue) Sys.sleep(0.5) # Run callback (adbcdrivermanager:::adbc_callback_queue_run_pending(queue)) #> Done 1! #> [1] 1 2 3 4 5 #> Done 2! #> [1] 6 7 8 9 10 #> Done 3! #> [1] 11 12 13 14 15 #> [1] 3 # Make sure gc() doesn't crash gc() #> used (Mb) gc trigger (Mb) limit (Mb) max used (Mb) #> Ncells 712327 38.1 1340807 71.7 NA 1340807 71.7 #> Vcells 1268606 9.7 8388608 64.0 16384 1996965 15.3 ``` <sup>Created on 2023-08-18 with [reprex v2.0.2](https://reprex.tidyverse.org)</sup> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
