azecevic000 opened a new pull request, #12111: URL: https://github.com/apache/arrow-rs/pull/12111
# Which issue does this PR close? N/A # Rationale for this change arrow-ipc writes delta dictionaries since #8001 when `IpcWriteOptions::dictionary_handling` is `Delta`, and `FlightDataEncoder` passes its `IpcWriteOptions` to the IPC writer. But `arrow_flight::encode::DictionaryHandling` has no option for it, so deltas over Flight work only through an undocumented combination of `Resend` and the IPC option. # What changes are included in this PR? Added a new `Delta` arm in the `DictionaryHandling` enum.`DictionaryHandling::Delta` sends dictionaries like `Resend` and sets the IPC option to `Delta`. Also update the docs that predate #8001. The dictionary tracker compares values, so a dictionary is sent again only when its values change, not with every batch or on every pointer change. # Are these changes tested? Two unit-tests are added: 1. `test_dictionary_delta`: tests the main flow where the dictionary incrementally builds on top of the previous dictionary state 2. `test_dictionary_delta_then_replacement_round_trip`: tests the case that the whole dictionary is resent when it's not a suffix of the previous dictionary state # Are there any significant user-facing changes? Yes, `Delta` is the new enum arm in `DictionaryHandling`, so the change is not forward-compatible. # AI usage disclosure Claude Code was used to implement the change in this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
