alamb commented on PR #5545:
URL:
https://github.com/apache/arrow-datafusion/pull/5545#issuecomment-1466517323
> Field contains other information that is irrelevant here like nullability.
We could assert on the correct value though (i.e. not nullable). Also this only
concerns on of the two API changes, the other one (PartitionedFile) is still
silently breaking. To avoid that we would need to rename partition_values to
something else. However I'm not sure if a rename would trigger people to not
only follow the rename but also change the types correctly.
Good point re `Field` and non relevant columns
What about a type that made the dictionary encoding explict, something like
```rust
struct PartitionedColumn {
name: String,
data_type: DataType,
dictionary_index_type: Option<DataType>
}
```
And then have a `new` function that defaults to Dictionary 🤔 But that is
kind of ugly as well and now what happens if the data_type was a dictionary in
the first place ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
