KushShriv commented on PR #13278:
URL: https://github.com/apache/gluten/pull/13278#issuecomment-6516792962
@philo-he Thanks. Yes, this only rewrites literal patterns. The translation,
and the decision to fall back on constructs RE2 cannot express (lookaround,
backreferences, unpaired surrogates), both happen at plan time, when the
pattern value is known.
Patterns from a column are not affected by this PR. `validateRegexExpr` in
`SubstraitToVeloxPlanValidator.cc` already rejects any non-literal pattern
("Pattern is not string literal for <fn>"), so those expressions fall back to
Spark as before.
Moving the rewrite into Velox would let us translate `\uXXXX` for column
patterns too. But unsupported constructs in a column pattern could then only be
detected during execution, so we could not fall back to Spark and would risk
failing queries that Spark runs fine. I would prefer to keep this PR to the
literal case, which keeps the fall-back-rather-than-fail behavior, and treat
column patterns as a separate follow-up if they turn out to matter. If you
think it is worth tracking, I can open an issue for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
