andygrove opened a new pull request, #73: URL: https://github.com/apache/datafusion-spark/pull/73
## Summary - Adds Spark's `regexp_extract(str, regexp[, idx])`, ported from DataFusion Comet's `string_funcs/regexp_extract.rs`. - Returns the text of group `idx` (default 1, 0 is the whole match) of the first match. NULL if any argument is NULL; empty string for no match or a group that did not participate. - `idx` outside `0..=groupCount` raises `INVALID_PARAMETER_VALUE.REGEX_GROUP_INDEX`; an invalid pattern raises `INVALID_PARAMETER_VALUE.PATTERN`, using Spark's messages. - Unlike Comet's version, `regexp` and `idx` do not have to be literals: the last compiled pattern is reused across rows, so a literal pattern compiles once per batch and a column of patterns still works. A NULL pattern or idx gives NULL before any validation, as in Spark. - Adds `regex` as a direct dependency. ## Known gap Patterns are Rust `regex` syntax, not `java.util.regex`. Java-only features (lookaround, backreferences, possessive quantifiers) are rejected as invalid patterns here, and `\d`-style shorthands match Unicode where Java's are ASCII-only. Not addressed in this PR. ## Testing - Unit tests and `string/regexp_extract.slt`. Expected values and error classes come from Spark's `regexp-functions.sql.out` golden file. - `cargo clippy --all-targets --all-features -- -D warnings`, `cargo test -p datafusion-spark` and `cargo test -p datafusion-spark-sqllogictest` pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
