andygrove commented on PR #340:
URL: https://github.com/apache/datafusion-comet/pull/340#issuecomment-2093735555
@ganeshkumar269 here is my suggestion:
```scala
if (CometSparkSessionExtensions.isSpark34Plus) {
// for Spark 3.4 we expect to reproduce the error message
exactly
assert(cometMessage == sparkMessage)
} else if (CometSparkSessionExtensions.isSpark33Plus) {
// for Spark 3.3 we just need to strip the prefix from the
Comet message
// before comparing
val cometMessageModified = cometMessage
.replace("[CAST_INVALID_INPUT] ", "")
.replace("[CAST_OVERFLOW] ", "")
assert(cometMessageModified == sparkMessage)
} else if (CometSparkSessionExtensions.isSpark32) {
// for Spark 3.2 we just make sure we are seeing a similar
type of error
if (sparkMessage.contains("causes overflow")) {
assert(cometMessage.contains("due to an overflow"))
} else {
// assume that this is an invalid input message in the form:
// `invalid input syntax for type numeric:
-9223372036854775809`
// we just check that the Comet message contains the same
literal value
val i = sparkMessage.indexOf(':') + 2
assert(cometMessage.contains(sparkMessage.substring(i)))
}
}
```
I tested this from the command line for all Spark versions using:
```
mvn test -Pspark-3.2 -DwildcardSuites=org.apache.comet.CometCastSuite
-Dspotless.check.skip=true
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
