carols10cents opened a new pull request #8773: URL: https://github.com/apache/arrow/pull/8773
This adds support for writing non-nested dictionaries in the IPC format in Rust. The first commit in this PR is sort of a bug fix related to correctly supporting dictionaries, but I wanted to draw attention to it specifically and make sure I'm handling that case correctly. The second commit is the majority of the new functionality. The third commit removes the TODO assertion in the integration test tool that errored if dictionaries were encountered in the JSON... which means nothing is reading from the `dictionaries` field of the `ArrowFile` struct. This might mean it can be removed? Or perhaps I'm missing something, or something isn't yet implemented? The fourth commit fixes an issue when the integration testing tool reads JSON files with dictionaries; the null bit buffers weren't being set. This didn't matter when testing Rust-to-Rust as neither side was setting the null bit buffers, so the columns were equal, but this does matter when testing dictionaries with another language and Rust. The fifth commit enables the dictionary and unsigned dictionary integration tests for Rust, but doesn't enable the nested dictionary test as that isn't working yet. The sixth commit makes the Rust integration testing tool less noisy-- it was printing more output than the other languages' integration testing tools. Let me know if this should be done a different way. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
