How are dictionaries intended to be used in a file with multiple record batches?
I tried saving record-batch-specific dictionaries and got this error from python: > pyarrow.lib.ArrowInvalid: Unsupported dictionary replacement or dictionary delta in IPC file This seems to defeat the purpose of having multiple record batches in a single arrow file; the work around appears to be to either preprocess the entire sequence of datasets to unify the dictionaries or save multiple arrow files.
