yihua opened a new issue, #827:
URL: https://github.com/apache/hudi-rs/issues/827
Registering a Hudi table with Lance base files as a
`HudiDataFusionDataSource` in Python and querying it aborts the interpreter:
```
there is no reactor running, must be called from the context of a Tokio 1.x
runtime
lance-io-11.0.0/src/scheduler.rs:911
thread caused non-unwinding panic. aborting.
```
datafusion-python polls the provider's record-batch stream across the
`datafusion-ffi` boundary on its own tokio runtime. Tokio's current-runtime
context is per compiled copy, so Lance's I/O scheduler, which spawns tasks,
finds no runtime in the binding's copy; the panic happens inside an `extern
"C"` poll and cannot unwind. Parquet tables are unaffected because that path
never asks for a runtime handle, and the same Lance table reads fine through
`HudiTable.read()`.
`FFI_TableProvider::new_with_ffi_codec` takes an optional runtime handle
that datafusion-ffi enters around execution and every poll; the binding passes
`None` today. Reproduced on `main` (0.6.0-dev) with datafusion-python 54 and on
#698 with 55.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
