neilconway opened a new pull request, #26121: URL: https://github.com/apache/datafusion/pull/26121
## Which issue does this PR close? - N/A ## Rationale for this change `sub_view` was introduced in #26039; it efficiently builds a view that is a substring of another view. Several string functions had private helpers that did the same thing or used `append_view`. This PR refactors those places to use `sub_view` instead. Both `append_view` and the private helpers were built on Arrow's `make_view`. `make_view` has some shortcomings: * For strings of up to 12 bytes, it does an indirect jump on the string's length. For real-world data with variable length strings, this jump is often poorly predicted. * Perhaps because it has a 13-arm match statement, `make_view` was marked as `inline(never)`, to avoid bloating the code at each call-site In contrast, `sub_view` replaces the indirect jump with shifts and masks, and is small enough to inline. Benchmarks show that adopting `sub_view` improves performance by 5-40%, depending on the workload. Along the way, simplify the `Trimmer` interface because `append_view` is no longer used. Benchmarks: (x86, AMD EPYC Milan): - ltrim/spaces: 52.4µs → 37.5µs, −28% - ltrim/heavy_padding: 285.2µs → 215.6µs, −24% - ltrim/nothing_to_trim: 51.8µs → 29.3µs, −43% - ltrim/char_set: 2.80ms → 2.80ms, 0% - rtrim/spaces: 53.6µs → 37.6µs, −30% - rtrim/heavy_padding: 322.0µs → 253.0µs, −21% - rtrim/nothing_to_trim: 51.4µs → 29.0µs, −44% - rtrim/char_set: 2.90ms → 2.80ms, −3% - btrim/spaces: 87.3µs → 56.6µs, −35% - btrim/heavy_padding: 458.6µs → 398.3µs, −13% - btrim/nothing_to_trim: 58.1µs → 42.0µs, −28% - btrim/char_set: 5.70ms → 5.50ms, −4% - substr/short_result: 114.6µs → 100.5µs, −12% - substr/long_result: 198.4µs → 181.8µs, −8% - substr/short_result_long_input: 93.5µs → 95.5µs, +2% - substr/no_count: 84.0µs → 62.3µs, −26% - substr/per_row_start: 149.0µs → 119.5µs, −20% - split_part/short_fields: 140.9µs → 113.7µs, −19% - split_part/long_fields: 137.6µs → 113.3µs, −18% - split_part/many_fields: 497.9µs → 427.6µs, −14% - split_part/negative_position: 98.6µs → 89.0µs, −10% - split_part/multi_char_delimiter: 163.8µs → 152.6µs, −7% - split_part/per_row_position: 458.5µs → 460.2µs, 0% - substr_index/array_single_delimiter: 295.1µs → 292.7µs, −1% - substr_index/array_long_delimiter: 471.7µs → 466.5µs, −1% - substr_index/scalar_single_delimiter_pos: 75.0µs → 72.0µs, −4% - substr_index/scalar_single_delimiter_neg: 79.3µs → 75.7µs, −5% - substr_index/scalar_long_delimiter_pos: 103.1µs → 104.5µs, +1% - substr_index/scalar_long_delimiter_neg: 219.2µs → 207.3µs, −5% - left/short_result: 37.2µs → 37.7µs, +1% - left/long_result: 38.2µs → 38.2µs, 0% - left/short_result_long_input: 50.4µs → 50.3µs, 0% - left/n_exceeds_len: 36.4µs → 34.6µs, −5% - left/negative_n: 34.7µs → 34.6µs, 0% - left/per_row_n: 33.5µs → 33.8µs, +1% - right/short_result: 48.1µs → 48.3µs, 0% - right/long_result: 52.2µs → 45.0µs, −14% - right/short_result_long_input: 63.1µs → 61.1µs, −3% - right/n_exceeds_len: 46.4µs → 41.9µs, −10% - right/negative_n: 45.7µs → 41.0µs, −10% - right/per_row_n: 45.3µs → 43.7µs, −4% ## What changes are included in this PR? * Refactor `trim`, `substr`, `split_part`, and `substring_index` to use `sub_view` * Remove `append_view` (no longer used) * Simplify `Trimmer` interface (byte offset no longer needed) * Optimize `sub_view` for long result strings ## What is the testing strategy for this PR? Covered by existing tests. ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
