fangchenli opened a new issue, #1626:
URL: https://github.com/apache/datafusion-python/issues/1626
**Describe the bug**
SessionContext.register_record_batches raises an unhandled Rust
PanicException instead of a normal Python error when given a partition that
contains no record batches — for example, the batches of an empty
pyarrow.Table, whose to_batches() returns [].
The schema is read from the first batch with unchecked indexing
(crates/core/src/context.rs):
```python
let schema = partitions.0[0][0].schema(); // panics when partitions.0[0] is
empty
```
With empty input, partitions.0 is [[]] — one partition holding zero batches
— so [0][0] indexes out of bounds and panics across the Python/Rust boundary.
This is the unfixed sibling of #575 / #613, which hit the identical panic on
from_arrow_table. That fix added an optional schema to create_dataframe and
threads the pyarrow object's schema through, but register_record_batches takes
no schema and was left unguarded.
**To Reproduce**
```python
import datafusion as d, pyarrow as pa
from datafusion import SessionContext
empty = pa.table({"k": pa.array([], pa.string()), "v": pa.array([],
pa.float64())})
print(empty.to_batches()) # -> [] (no batches)
ctx = SessionContext()
ctx.register_record_batches("t", [empty.to_batches()])
pyo3_runtime.PanicException: index out of bounds: the len is 0 but the index
is 0
Observed on datafusion 54.0.0; the offending line is still present on main.
```
**Expected behavior**
A catchable Python error, not a panic. An empty pyarrow.Table loses its
schema through to_batches(), so the schema genuinely cannot be recovered at
this point — a clear error is the right outcome (matching the graceful handling
added for from_arrow_table in #613).
**Additional context**
The same unchecked partitions.0[0][0] also remains in create_dataframe's
no-schema else branch, reachable when it is called directly without a schema.
(Out of scope here; noted for completeness.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
