oersted opened a new issue, #2952:
URL: https://github.com/apache/arrow-rs/issues/2952
**Describe the bug**
`serde_arrow::from_record_batch` expects `arrow::record_batch::RecordBatch`
and `arrow::schema::Schema` as inputs, while `ParquetRecordBatchStream` seems
to yield `arrow_array::record_batch::RecordBatch` and similarly `::schema()`
returns `arrow_schema::schema::Schema`.
As far as I can tell, the implementation of the types are exactly the same
but the compiler considers them different types.
**To Reproduce**
```
#[derive(Deserializable)]
struct S {}
let file = File::open(path.as_ref()).await?;
let mut batches = ParquetRecordBatchStreamBuilder::new(file).await?
.with_batch_size(1024) // Default
.build()?;
let schema = batches.schema().as_ref();
while let Some(batch) = batches.next().await {
let batch = batch?;
from_record_batch::<S>(&batch, schema);
}
```
```
error[E0308]: arguments to this function are incorrect
--> src/sign.rs:28:17
|
28 | from_record_batch::<S>(&batch, schema);
| ^^^^^^^^^^^^^^^^^^^^^^^^^ ------ ------ expected
struct `Schema`, found struct `arrow_schema::schema::Schema`
| |
| expected struct
`arrow::record_batch::RecordBatch`, found struct
`arrow_array::record_batch::RecordBatch`
|
= note: expected reference `&arrow::record_batch::RecordBatch`
found reference `&arrow_array::record_batch::RecordBatch`
= note: expected reference `&Schema`
found reference `&arrow_schema::schema::Schema`
```
**Expected behavior**
I would expect the API to return the types exposed by the main `arrow`
namespace rather than `arrow_array` and `arrow_schema`. At least it should be
possible to convert between them with `From`.
Because of this, as far as I can tell, it isn't possible (or at least
ergonomic) to use `serde` to deserialise from `parquet` directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
