dtenedor commented on a change in pull request #35735:
URL: https://github.com/apache/spark/pull/35735#discussion_r822307487
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/HistogramNumeric.scala
##########
@@ -124,8 +128,33 @@ case class HistogramNumeric(
null
} else {
val result = (0 until buffer.getUsedBins).map { index =>
+ // Note that the 'coord.x' and 'coord.y' have double-precision
floating point type here.
val coord = buffer.getBin(index)
- InternalRow.apply(coord.x, coord.y)
+ if (propagateInputType) {
+ // If the
SQLConf.spark.sql.legacy.histogramNumericPropagateInputType is set to true,
+ // we need to internally convert the 'coord.x' value to the expected
result type, for
+ // cases like integer types, timestamps, and intervals which are
valid inputs to the
+ // numeric histogram aggregate function. For example, in this case:
+ // 'SELECT histogram_numeric(val, 3) FROM VALUES (0L), (1L), (2L),
(10L) AS tab(col)'
+ // returns an array of structs where the first field has LongType.
+ val result: Any = left.dataType match {
+ case ByteType => coord.x.toByte
+ case IntegerType | DateType | _: YearMonthIntervalType =>
+ coord.x.toInt
+ case FloatType => coord.x.toFloat
+ case ShortType => coord.x.toShort
+ case _: DayTimeIntervalType | LongType | TimestampType |
TimestampNTZType =>
+ coord.x.toLong
+ case _ => coord.x
+ }
+ InternalRow.apply(result, coord.y)
Review comment:
I tried this but sadly the `Cast.canCast` method returns exceptions for
certain combinations of source and destination types. It looks like we have to
force it with these to* methods instead.
<img width="902" alt="image"
src="https://user-images.githubusercontent.com/99207096/157380664-ec824c2f-29e6-4ced-8512-7881b0863e59.png";>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/99207096/157380664-ec824c2f-29e6-4ced-8512-7881b0863e59.png"](https://user-images.githubusercontent.com/99207096/157380664-ec824c2f-29e6-4ced-8512-7881b0863e59.png"){kind=link}