lgbo-ustc opened a new pull request, #13608: URL: https://github.com/apache/gluten/pull/13608
## What changes are proposed in this pull request? Fixes #13607. The ClickHouse backend currently applies Spark/Java-to-RE2 adjustments only to `regexp_extract`; other regex functions can interpret the same pattern differently. Introduce a shared `spark_regexp::re2::translatePattern()` with typed top-level nodes and recursively parsed character classes, and use it for constant patterns in `regexp_extract`, `regexp_extract_all`, `regexp_replace`, `rlike`, and `split`. Preserve capture indices, translate the supported nested class/anchor/dot subset and scoped DOTALL flags, validate the result with RE2, and add CH/Spark comparison and C++ tests. Fix lack of forward progress on zero-width matches in `regexp_extract_all`. The replacement-string `$n` conversion remains separate in the existing Scala transformer. Unsupported constructs are conservatively filtered at the Scala level where recognized and rejected by the native translator otherwise. **Draft: known limitations / review focus** - Full ClickHouse native build and integration suite have not been run locally (required CH build/test environment unavailable). - Java `$` before a final CRLF has a zero-width boundary that the current RE2 lowering does not represent exactly; this and matching semantics for zero-width `regexp_extract_all` need CH/Spark differential tests before this is ready for review. - The Scala prefilter is heuristic, not a parser, so unsupported Java patterns can still pass planning and be rejected by the native translator instead of falling back to Spark. A robust fallback strategy is needed before marking ready. - Java constructs such as lookaround, backreferences, and character-class intersection are not generally lowered. ## How was this patch tested? - Added C++ translation tests and Spark/CH SQL comparison tests. - Compiled and ran the translation assertions against a locally available RE2 library with lightweight CH exception and gtest shims; **this is not a ClickHouse native build**. - `./dev/format-scala-code.sh check` (with a writable local Maven repository) passed, as did `git diff --check`. - Native CH compilation and the Spark/ClickHouse integration tests remain pending. Header checker could not run locally because Python `regex` is missing. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: DeepSeek Harness gpt-6-sol -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
