princejain756 commented on PR #11904: URL: https://github.com/apache/arrow-rs/pull/11904#issuecomment-6433884423
@Rich-T-kid That GKE benchmark ran against `5a1bb36` (before `5132371`). In `5a1bb36`, `row_lengths` evaluated `remap.as_ref().map_or_else(|| k.as_usize(), |r| r[idx])` inside the per-row closure across all 4096 rows, which added a fixed ~4 to 5us overhead even when `values.len() <= dictionary.len()` (`remap = None`, which is the path taken by `string_dictionary` and `string_dictionary_low_cardinality`). In `5132371`, I hoisted `match remap.as_deref()` outside the loop in `row_lengths` (matching `encode_column`), so when `values.len() <= dictionary.len()`, the `None` branch executes the exact same loop as `main`. Could you re-trigger `run benchmark row_format` against `5132371`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
