anentropic opened a new issue, #4915:
URL: https://github.com/apache/arrow-adbc/issues/4915
### Describe the bug
With polars 2.0, `adbc_driver_manager.dbapi.Cursor.fetch_polars()` returns a
`polars.Series` with a single struct column instead of a `polars.DataFrame`.
`_RowIterator.fetch_polars()` calls `polars.from_arrow(self.fetch_arrow())`,
which passes in an `ArrowArrayStreamHandle`. polars 2 now turns an Arrow C
stream given to `from_arrow` into a struct `Series`. polars 1.x returned a
`DataFrame`. The `typing.cast(polars.DataFrame, ...)` around the call hides
this from type checkers.
### Reproduce
```python
import adbc_driver_sqlite.dbapi as d
with d.connect() as c, c.cursor() as cur:
cur.execute("SELECT 1 AS a, 'x' AS b")
r = cur.fetch_polars()
print(type(r).__name__, r.dtype, r.shape)
```
Output with polars 2.0.0:
```
Series Struct({'a': Int64, 'b': String}) (1,)
```
Expected: a `DataFrame` with columns `a` and `b`. That is what polars 1.x
returns.
### Versions
- adbc-driver-manager 1.12.0
- polars 2.0.0
- pyarrow 26.0.0
- Python 3.12, Linux
### Possible fix
One option is to unnest when polars hands back a Series:
```python
result = polars.from_arrow(self.fetch_arrow())
if isinstance(result, polars.Series):
result = result.struct.unnest()
```
Another is to build the frame with `polars.DataFrame(...)` from the stream.
That needs checking against both polars 1.x and 2.x. We've applied the unnest
workaround downstream in adbc-poolhouse 1.6.4, and it keeps the frame's schema,
including for empty results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
