Rich-T-kid opened a new pull request, #10325: URL: https://github.com/apache/arrow-rs/pull/10325
# Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. --> - Closes #7710. - revised version of https://github.com/apache/arrow-rs/pull/9865 # Rationale for this change for a logical representation of an ree ``` let logical_repr = [1, 1, 0, 0, 1, 1]; let ree_take_results = take(logical_repr,[0,1,4,5]); this produces run_ends = [2,4] values = [1,1] ``` when the result should be `runs: [4], values: [1]` both answers are correct but ree's should be as compact as possible. see #7710 <!-- Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. --> # What changes are included in this PR? - `take()` on `RunEndEncoded` arrays now compares values instead of physical indices when deciding run boundaries, producing a more compact run-end representation and fixing cases where identical values across different runs were not merged. - Introduces `arrow-cmp`, a minimal crate that extracts `make_comparator`/`DynComparator` from `arrow-ord` so `arrow-select` can use slot-wise comparison without a circular dependency. Most of the line diff is code moving (+1,889,-1834), not new logic `arrow-ord` re-exports from` arrow-cmp` so its public API is unchanged. <!-- There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR. --> # Are these changes tested? yes, I included three test to assert the compaction behavior we expect from a Run-end array. <!-- We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? If this PR claims a performance improvement, please include evidence such as benchmark results. --> # Are there any user-facing changes? <!-- If there are user-facing changes then we may require documentation to be updated before approving the PR. If there are any breaking changes to public APIs, please call them out. --> 1. `take()` on `RunEndEncoded` arrays behaves differently. the output may have fewer runs than before. Previously runs were only merged when they hit the same physical index; now runs with equal values are also merged across different physical indices. Any code asserting on the exact run-end structure of take output could break. 2. New `arrow-cmp` crate is published. Users can depend on it directly to get `make_comparator` / `DynComparator` without pulling in all of `arrow-ord`. `arrow-ord` still re-exports both so its API is unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
