andygrove opened a new issue, #6980: URL: https://github.com/apache/datafusion-comet/issues/6980
### What is the problem the feature request solves? [`levenshtein_distance_with_threshold` in string_funcs/levenshtein.rs (L178-L349)](https://github.com/apache/datafusion-comet/blob/7d8d9b1544b734c85a93cca2cc9d5380ad70accd/native/spark-expr/src/string_funcs/levenshtein.rs#L178-L349) contains the same banded dynamic-programming algorithm twice: an ASCII byte path (about L183-250) and a `char` path (about L267-349). The two copies differ only in the element type (`u8` vs `char`) and variable names. Any fix to the band bounds, out-of-band handling or threshold capping must be made in both places, and nothing checks that they stay in sync. ### Describe the potential solution Extract a generic helper, for example `fn banded_levenshtein<T: PartialEq + Copy>(s: &[T], t: &[T], threshold: i32) -> i32`, containing the length checks, the swap, the effective threshold and the DP over the scratch buffers. Keep the `is_ascii` fast path so the byte case still avoids allocating `char` vectors: call the helper with `s.as_bytes()` for ASCII and with collected `Vec<char>` slices otherwise. Run the existing tests and a quick benchmark to confirm performance is unchanged. ### Additional context This is self-contained and a good first issue. Line references are pinned to commit 7d8d9b1544b7. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
