asubiotto opened a new pull request, #12071: URL: https://github.com/apache/arrow-rs/pull/12071
# Which issue does this PR close? - Closes apache/arrow-rs#12069 Stacked on #12070, which adds the benchmark; only the last commit is new here. # Rationale for this change For every dictionary batch, `get_dictionary_values` calls `Schema::fields_with_dict_id`, which collects every nested field in the schema into a `Vec` and then keeps only the first match. Reading a stream with many dictionaries pays for that once per dictionary. # What changes are included in this PR? A private helper walks the schema in the same pre-order and returns the first match, without allocating. # Are these changes tested? A new test checks that the helper matches `Schema::fields_with_dict_id(id).first()` across nested types, duplicate IDs, and a `dict_id` on a non-dictionary field. `StreamReader/dictionaries_1000` from #12070, local run: | | main | this PR | | --- | --- | --- | | flat | 18.59 ms | 2.52 ms | | nested | 70.49 ms | 4.75 ms | # Are there any significant user-facing changes? No. The implementation and test were written with AI assistance (Claude) and reviewed by me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
