rustyconover opened a new pull request, #955: URL: https://github.com/apache/arrow-nanoarrow/pull/955
Split out of #928 as requested in review. This PR only adds decoding of dictionary delta batches; the writer changes remain in #928. Previously, `ArrowIpcDecoderDecodeDictionary()` failed with "Dictionary concatenation is not yet supported" when it got a DictionaryBatch with `isDelta` set and the current dictionary was non-empty. Now the delta values are appended: a new array is built from the current dictionary value and the delta using `ArrowArrayAppendStorageFromArrayView()` (#930), and it replaces the stored dictionary. Arrays decoded earlier keep referencing their previous dictionary value. Value types that `ArrowArrayAppendStorageFromArrayView()` can't append (e.g., unions) fail with `ENOTSUP`. Changes from the review of #928: - Removed the string/binary-view-specific handling (IPC doesn't support views yet, #824). - Replaced the `if (result == NANOARROW_OK)` chains with a helper that doesn't own its inputs, plus a single check and release. - Dropped the "try to unshare" optimization and the internal `common/utils.c` helper. Each delta copies the dictionary; appending in place when nothing else references the dictionary can come in a follow-up. - Tests use `enum ArrowType` for parameters, and the dense union rejection test is removed. This also fixes a double release on the error path of `ArrowIpcDecoderDecodeDictionaryInternal()`: `ArrowArrayMoveShared()` already releases its input on failure, so the temporary is now only released if it is still live. Tests: `NanoarrowIpcDecodeDictionaryBatch` covers deltas, including a clone held across a delta keeping its old value. With `NANOARROW_BUILD_TESTS_WITH_ARROW`, streams written by Arrow C++ with `emit_dictionary_deltas = true` are read back for bool, int64 (with nulls), uint64, double, string, binary, decimal128, list, struct and fixed-size list values. Locally (aarch64 Linux, Debug, ZSTD/LZ4), ctest passes with and without Arrow C++ 21.0.0, and the decoder tests are clean under valgrind. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
