andygrove commented on code in PR #1700:
URL: https://github.com/apache/datafusion-comet/pull/1700#discussion_r2070408978
##########
spark/src/test/scala/org/apache/comet/CometFuzzTestSuite.scala:
##########
@@ -188,6 +188,22 @@ class CometFuzzTestSuite extends CometTestBase with
AdaptiveSparkPlanHelper {
}
}
+ test("regexp_replace") {
+ withSQLConf(CometConf.COMET_REGEXP_ALLOW_INCOMPATIBLE.key -> "true") {
+ val df = spark.read.parquet(filename)
+ df.createOrReplaceTempView("t1")
+ // We want to make sure that the schema generator wasn't modified to
accidentally omit
+ // StringType, since then this test would not run any queries and
silently pass.
+ var testedString = false
+ for (field <- df.schema.fields if field.dataType == StringType) {
+ testedString = true
+ val sql = s"SELECT regexp_replace(${field.name}, 'a', 'b') FROM t1"
Review Comment:
I'm not sure how likely it is that the input strings will contain the
character `a`, so it is possible that this test isn't really testing anything.
I modified the test locally to add:
```
val sql = s"SELECT count(*) FROM t1 WHERE ${field.name} LIKE '%a%'"
spark.sql(sql).show()
```
this produced
```
+--------+
|count(1)|
+--------+
| 0|
+--------+
```
This is a general problem with just comparing results between Spark and
Comet and it would be nice to have a way to also ensure that we see expected
results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
