jamesmcm opened a new issue, #10639:
URL: https://github.com/apache/datafusion/issues/10639
### Is your feature request related to a problem or challenge?
This job in Ballista fails:
```rust
let avro_file = "gs://...";
let metadata_df = ctx
.read_avro(avro_file, AvroReadOptions::default())
.await?
.select(vec![col("id"), col("nested_array")])?
.unnest_column("nested_array")?
```
```
Error: DataFusionError(Internal("failed to serialize logical plan:
Internal(\"LogicalPlan serde is not yet implemented for Unnest\")"))
```
Due to the lack of ProtoBuf serialisation here -
https://github.com/apache/datafusion/blob/main/datafusion/proto/src/logical_plan/mod.rs#L1527-L1529
### Describe the solution you'd like
ProtoBuf Serialisation would be added so that Unnest will be usable in
Ballista jobs.
### Describe alternatives you've considered
I also couldn't get taking just the head to work:
```rust
let metadata_df = ctx
.read_avro(avro_file, AvroReadOptions::default())
.await?
.select(vec![
col("id"),
col("nested_array")
.index(lit(1)) // Note index starts at 1
.field("array_id_1")
.alias("array_id_1"),
col("nested_array")
.index(lit(1))
.field("array_id_2")
.alias("array_id_2"),
])?;
```
which gave:
```
Error: DataFusionError(Plan("Only ints are valid as an indexed field in a
list"))
```
But really unnesting support is needed anyway.
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
