richox commented on code in PR #1317:
URL: https://github.com/apache/auron/pull/1317#discussion_r2365592967
##########
spark-extension/src/main/scala/org/apache/spark/sql/auron/NativeConverters.scala:
##########
@@ -447,11 +448,19 @@ object NativeConverters extends Logging {
case cast: Cast
if !Seq(cast.dataType, cast.child.dataType).exists(t =>
t.isInstanceOf[TimestampType] || t.isInstanceOf[DateType]) =>
+ val castChild =
+ if (cast.child.dataType == StringType &&
cast.dataType.isInstanceOf[NumericType] &&
Review Comment:
and `BooleanType`
##########
spark-extension/src/main/java/org/apache/spark/sql/auron/AuronConf.java:
##########
@@ -52,6 +52,9 @@ public enum AuronConf {
// TypedImperativeAggregate one row mem use size
UDAF_FALLBACK_ESTIM_ROW_SIZE("spark.auron.udafFallback.typedImperativeEstimatedRowSize",
256),
+ /// enable trimming string inputs before casting to numeric types
+ CAST_STRING_TRIM_ENABLE("spark.auron.cast.stringTrimBeforeNumeric", true),
Review Comment:
i suggest a better name `spark.auron.cast.trimString` because casting string
to boolean also needs to be supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
