carloea2 commented on code in PR #8990:
URL: https://github.com/apache/texera/pull/8990#discussion_r4291146423
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/regex/RegexOpDesc.scala:
##########
@@ -94,8 +99,44 @@ class RegexOpDesc extends FilterOpDesc with
StandaloneCodeGenerator {
.flatMap(schema =>
scala.util.Try(schema.getAttribute(attribute)).toOption)
.map(_.getType)
val column = renderedAsText(s"in1df[$attrLit]", declared)
- s"""out1df = in1df[in1df[$attrLit].notna() &
$column.str.contains($pyLiteral, regex=True, case=$caseArg,
na=False)].reset_index(drop=True)"""
+ s"""import re
+ |out1df = in1df[in1df[$attrLit].notna() &
$column.str.contains($pyLiteral, regex=True, case=$caseArg, flags=re.ASCII,
na=False)].reset_index(drop=True)""".stripMargin
Review Comment:
[P2] Keep existing inline Unicode flags working. With pattern (?iu)é on é,
É, x and null, the native executor and previous export keep the first two rows.
This export raises ValueError: ASCII and UNICODE flags are incompatible,
because re.ASCII conflicts with the inline u flag. Please handle explicit
Unicode flags without this conflict and add a regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
