yew1eb opened a new issue, #1686:
URL: https://github.com/apache/auron/issues/1686
**Describe the bug**
when `spark.auron.datetime.extract.enabled=true`, queries using
hour()/minute()/second() function unexpectedly fallback to spark, due to an
IllegalArgumentException thrown in NativeConverters.buildTimePartExt. The error
message is:
“Literal must have a corresponding value to string, but class String found.”
**To Reproduce**
Run the test:
```
test("test filter with hour function") {
withEnvConf("spark.auron.datetime.extract.enabled" -> "true") {
withTable("t_hour") {
sql("""
|create table t_hour using parquet as
|select to_timestamp('2024-12-18 01:23:45') as event_time union
all
|select to_timestamp('2024-12-18 08:00:00') union all
|select to_timestamp('2024-12-18 08:59:59')
|""".stripMargin)
// Keep rows where HOUR >= 8, then group by hour
checkSparkAnswerAndOperator(
"""
|select h, count(*)
|from (select hour(event_time) as h from t_hour) t
|where h >= 8
|group by h
|order by h
|""".stripMargin)
}
}
}
```
- The test fails because the plan falls back to spark execution.
- Log shows fallback due to IllegalArgumentException in buildTimePartExt:
```
DEBUG AuronLogUtils: Converting Project: Project (unknown)
projectExprs: [hour(event_time#5, Some(America/Los_Angeles)) AS h#15]
WARN AuronConverters: Falling back exec: ProjectExec: requirement failed:
Literal must have a corresponding value to string, but class String found.
java.lang.IllegalArgumentException: requirement failed: Literal must have a
corresponding value to string, but class String found.
at scala.Predef$.require(Predef.scala:281)
at
org.apache.spark.sql.catalyst.expressions.Literal$.validateLiteralValue(literals.scala:246)
at
org.apache.spark.sql.catalyst.expressions.Literal.<init>(literals.scala:355)
at
org.apache.spark.sql.auron.NativeConverters$.buildTimePartExt(NativeConverters.scala:1339)
at
org.apache.spark.sql.auron.NativeConverters$.convertExprWithFallback(NativeConverters.scala:917)
at
org.apache.spark.sql.auron.NativeConverters$.convertExpr(NativeConverters.scala:305)
at
org.apache.spark.sql.execution.auron.plan.NativeProjectBase$.addNamedExpression$1(NativeProjectBase.scala:117)
at
org.apache.spark.sql.execution.auron.plan.NativeProjectBase$.$anonfun$getNativeProjectBuilder$1(NativeProjectBase.scala:128)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
