lgbo-ustc commented on code in PR #13594:
URL: https://github.com/apache/gluten/pull/13594#discussion_r4510002626
##########
backends-clickhouse/src/test/scala/org/apache/gluten/execution/compatibility/GlutenClickhouseStringFunctionsSuite.scala:
##########
@@ -107,6 +107,26 @@ class GlutenClickhouseStringFunctionsSuite extends
GlutenClickHouseWholeStageTra
}
}
+ test("GLUTEN-13593: regexp_extract with escaped bracket in character class")
{
+ withTable("regexp_extract_escaped_bracket") {
+ sql("create table regexp_extract_escaped_bracket(a String) using
parquet")
+ sql("""
+ |insert into regexp_extract_escaped_bracket
+ | values ('[com.example.demo.AlphaController viewDidAppear:]'),
+ | ('[com.example.demo.BetaViewController]'), ('[123-456]')
+ |""".stripMargin)
+
+ val sql_str =
+ """select
+ | regexp_extract(a, '\\[(?:.*\\.)?([^\\.\\s\\]]+)', 1)
+ | , regexp_extract(a, '\\[([0-9-]+)\\]', 1)
Review Comment:
Could we also cover an escaped `[` *inside* a character class here, e.g.
`regexp_extract(a, '[\\[]', 0)` (the SQL literal unescapes to `[\[]`)? Pre-fix
that pattern does not return a wrong value, it makes `adjustSparkRegexpRule()`
throw at planning time, so it exercises a different failure mode than the two
cases above and backs the "escaped brackets inside and outside a character
class" coverage claimed in the description.
##########
cpp-ch/local-engine/Parser/scalar_function_parser/regexp_extract.cpp:
##########
@@ -111,6 +111,21 @@ class FunctionParserRegexpExtract : public FunctionParser
while (!buf.eof())
{
+ // Keep "\x" as an atomic unit, otherwise escaped brackets such as
"\[" and "\]"
Review Comment:
nit: `\x` reads like a regex hex escape here; `\` + any character (or `\X`)
would be less ambiguous. Also, this file already references issues in comments
(`// adaptation for $, see issue #8325`), so pointing at GLUTEN-13593 would
help future readers.
##########
cpp-ch/local-engine/Parser/scalar_function_parser/regexp_extract.cpp:
##########
@@ -111,6 +111,21 @@ class FunctionParserRegexpExtract : public FunctionParser
while (!buf.eof())
{
+ // Keep "\x" as an atomic unit, otherwise escaped brackets such as
"\[" and "\]"
+ // (especially "\]" inside a character class, e.g. "[^\.\s\]]")
are mistaken for
+ // character class boundaries and the whole pattern is corrupted.
+ if (*buf.position() == '\\')
+ {
+ strs.top() += *buf.position();
+ ++buf.position();
+ if (!buf.eof())
+ {
+ strs.top() += *buf.position();
+ ++buf.position();
+ }
+ continue;
+ }
Review Comment:
nit, non-blocking: "copy the current char and advance" is now written four
times. Extracting it makes the escape handling easier to read/verify, e.g.
```cpp
auto copy_current = [&]() {
strs.top() += *buf.position();
++buf.position();
};
if (*buf.position() == '\\')
{
copy_current();
if (!buf.eof())
copy_current();
continue;
}
```
Current code is correct as is, feel free to ignore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]