paleolimbot commented on issue #323: URL: https://github.com/apache/arrow-nanoarrow/issues/323#issuecomment-1834489176
Ok! I did some debugging here...there is nothing "wrong" happening; however, it should not be this easy to get a crash. ADBC statements MUST outlive the stream. Here, they are sometimes outliving the stream because the garbage collector isn't always running in time (I think we both already knew this bit). At the C level, there's no way for the streams produced to check if their parent statement is valid (also goes for statements checking that their connection is valid and so on). They *could*, but they don't, and even if we made the SQLite/Postgres drivers do this, other future drivers might not because the ADBC spec doesn't say they have to. At the R level, we can count the number of child objects that have been instantiated and count the number that have been released and prevent the C-level release callback from being called if not all child objects have been closed. I will make that change ( https://github.com/apache/arrow-adbc/issues/1128 ). In this case, that would result in `adbc_statement_release()` warning, *not* calling the C-level release for the statement, and letting the garbage collector detect when the strong reference from the stream to the statement is no longer present. If you're writing pure ADBC code, `with_adbc()` probably has you covered. I get that in adbi that's not quite an option for you, though. ``` r library(adbcdrivermanager) db <- adbc_database_init(adbcsqlite::adbcsqlite(), uri = ":memory:") con <- adbc_connection_init(db) i <- 0 while (i < 10) { message(i <- i + 1) stmt <- with_adbc(stmt <- adbc_statement_init(con), { adbc_statement_set_sql_query(stmt, "SELECT $1 + 1.0 AS a") adbc_statement_prepare(stmt) adbc_statement_bind_stream( stmt, data.frame(c(0.5, 1.5, 2.5), fix.empty.names = FALSE) ) res <- nanoarrow::nanoarrow_allocate_array_stream() with_adbc(res, { adbc_statement_execute_query(stmt, res) ptype <- nanoarrow::nanoarrow_array_init( nanoarrow::infer_nanoarrow_schema(res) ) nanoarrow::convert_array_stream(res, function(schema, ptype) ptype) }) }) } #> 1 #> 2 #> 3 #> 4 #> 5 #> 6 #> 7 #> 8 #> 9 #> 10 ``` <sup>Created on 2023-11-30 with [reprex v2.0.2](https://reprex.tidyverse.org)</sup> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
