heyihong commented on code in PR #52188:
URL: https://github.com/apache/spark/pull/52188#discussion_r2321731563
##########
sql/connect/common/src/main/scala/org/apache/spark/sql/connect/common/LiteralValueProtoConverter.scala:
##########
@@ -434,11 +434,16 @@ object LiteralValueProtoConverter {
case proto.DataType.KindCase.BOOLEAN => v => v.getBoolean
case proto.DataType.KindCase.STRING => v => v.getString
case proto.DataType.KindCase.BINARY => v => v.getBinary.toByteArray
- case proto.DataType.KindCase.DATE => v => v.getDate
- case proto.DataType.KindCase.TIMESTAMP => v => v.getTimestamp
- case proto.DataType.KindCase.TIMESTAMP_NTZ => v => v.getTimestampNtz
- case proto.DataType.KindCase.DAY_TIME_INTERVAL => v =>
v.getDayTimeInterval
- case proto.DataType.KindCase.YEAR_MONTH_INTERVAL => v =>
v.getYearMonthInterval
+ case proto.DataType.KindCase.DATE =>
+ v => SparkDateTimeUtils.toJavaDate(v.getDate)
Review Comment:
My understanding is that the contract should return a Scala value instead of
a Catalyst value, since LiteralValueProtoConverter.toCatalystValue is called
elsewhere, such as in
[MLUtils](https://github.com/apache/spark/blob/336ca8c12a163d31519a491a28d26ff091626986/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ml/MLUtils.scala#L143),
[ObservedMetrics](https://github.com/apache/spark/blob/336ca8c12a163d31519a491a28d26ff091626986/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/SparkResult.scala#L385),
and
[NAFill](https://github.com/apache/spark/blob/336ca8c12a163d31519a491a28d26ff091626986/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala#L427).
However, the name toCatalystValue is confusing, so we should consider renaming
it.
There may be some additional conversion involved, but if that is a concern,
we could provide an extra implementation that directly converts a Proto value
to a Catalyst value in LiteralExpressionProtoConverter. Still, I feel that this
may not be achievable within LiteralValueProtoConverter due to the missing
`sql/catalyst` dependency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
