rustyconover opened a new pull request, #928: URL: https://github.com/apache/arrow-nanoarrow/pull/928
## Summary - add DictionaryBatch metadata/body encoding and low-level writer APIs - teach the stream writer to discover dictionaries, emit them dependency-first, suppress unchanged values, and emit append-only changes as deltas - advertise and handle dictionary replacements in streams while rejecting replacements in IPC files - decode dictionary deltas across supported value types, including nested and run-end encoded values - preserve dictionary snapshots with copy-on-write while avoiding quadratic delta decoding in the usual streaming loop - expand native, Arrow C++, file, nested-dictionary, replacement, delta, and failure-path coverage This builds on and is intended to supersede #926. Its original commit is retained as the first commit here, preserving authorship and credit to @niekverw. ## IPC behavior Streams may emit replacement dictionaries and advertise `DICTIONARY_REPLACEMENT` in the schema. Append-only dictionaries are emitted as deltas. Files continue to reject dictionary replacement but accept append-only deltas, whose blocks are recorded in footer order. ## Validation - repository pre-commit suite: passed - native CTest suite: 345/345 passed (4 codec tests skipped because those optional codecs were not enabled) - Arrow C++ writer suite: 28/28 passed, including 11 dictionary value-type cases - Arrow C++ decoder suite: 287 passed, with 10 pre-existing unsupported dictionary/extension cases skipped - PyArrow-generated delta streams: string, list, and struct dictionary values decoded successfully - AddressSanitizer dictionary-delta ownership regression: passed - namespaced, bundled C, and bundled C++ builds: passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
