mailmindlin opened a new pull request, #26263:
URL: https://github.com/apache/datafusion/pull/26263
FFI_TableProvider's async methods (scan, insert_into, delete_from, update,
and truncate) are polled by the consumer's executor, which may have no tokio
context or one from a different copy of tokio. Wrap these futures so the
runtime registered with `FFI_TableProvider::new` is entered on every poll,
matching what execute and record batch stream polling already do. This is a
response to a panic I encountered using datafusion-ffi.
## Which issue does this PR close?
- Closes #26262
## What changes are included in this PR?
- `WithRuntime` / `WithRuntimeExt` in `datafusion/ffi/src/util.rs`: a future
wrapper that switches to the optional runtime when polled. The runtime is
entered and released within each `poll`, so the future stays `Send`.
- Wrap the futures of
`FFI_TableProvider::{scan,insert_into,delete_from,update,truncate}`
- Add `pin-project-lite` to `datafusion-ffi`.
## What is the testing strategy for this PR?
Adds `test_ffi_table_provider_async_methods_enter_runtime` in
`table_provider.rs`. It uses a provider whose async methods return an error
unless a tokio runtime is active, and calls each method through the FFI
boundary from a plain `#[test]` thread with no tokio context. The test fails
without this change.
## Are there any user-facing changes?
No public API changes. Performance cost should be an atomic operation or
two. FFI table providers that need a tokio runtime during planning now work
when called from consumers that have no tokio runtime of their own.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
