brancz commented on code in PR #10230:
URL: https://github.com/apache/arrow-rs/pull/10230#discussion_r3492634579
##########
arrow-ipc/src/writer.rs:
##########
@@ -536,7 +536,17 @@ impl IpcDataGenerator {
compression_context: &mut CompressionContext,
) -> Result<(), ArrowError> {
match column.data_type() {
- DataType::Dictionary(_key_type, _value_type) => {
+ DataType::Dictionary(_key_type, value_type) => {
+ // A dictionary batch writes this field's dictionary values as
its payload.
+ // If that payload is itself dictionary-encoded, readers
cannot match
+ // the encoded buffers to the schema metadata.
+ if matches!(value_type.as_ref(), DataType::Dictionary(_, _)) {
+ return Err(ArrowError::InvalidArgumentError(format!(
+ "IPC does not support dictionary-encoded values for
dictionary field {:?}",
Review Comment:
Yeah, as far as I can tell, there is nothing that prevents dicts from dicts
on IPC, and some forms of nested dicts are already supported, at least when
they're not direct children (eg. `dict(struct(dict))`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
