cloud-fan commented on a change in pull request #32945:
URL: https://github.com/apache/spark/pull/32945#discussion_r654111649
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGeneratorSuite.scala
##########
@@ -154,7 +156,15 @@ class RandomDataGeneratorSuite extends SparkFunSuite with
SQLHelper {
val data = generator.apply()
val catalyst = toCatalyst(data)
val convertedBack = toScala(catalyst)
- assert(data == convertedBack)
+ dt match {
Review comment:
shall we update the `RandomDataGenerator`? to make sure we generate
correct data for `interval year`.
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
##########
@@ -284,7 +285,10 @@ object RandomDataGenerator {
new CalendarInterval(months, days, ns)
})
case _: DayTimeIntervalType => Some(() => Duration.of(rand.nextLong(),
ChronoUnit.MICROS))
- case _: YearMonthIntervalType => Some(() =>
Period.ofMonths(rand.nextInt()).normalized())
+ case YearMonthIntervalType(_, YEAR) =>
+ val period = Period.ofMonths(rand.nextInt()).normalized()
+ Some(() => Period.ofYears(period.getYears).normalized())
Review comment:
can we just do `Period.ofYears(rand.nextInt() / 12).normalized())`?
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
##########
@@ -284,7 +285,10 @@ object RandomDataGenerator {
new CalendarInterval(months, days, ns)
})
case _: DayTimeIntervalType => Some(() => Duration.of(rand.nextLong(),
ChronoUnit.MICROS))
- case _: YearMonthIntervalType => Some(() =>
Period.ofMonths(rand.nextInt()).normalized())
+ case YearMonthIntervalType(_, YEAR) =>
+ val period = Period.ofMonths(rand.nextInt()).normalized()
+ Some(() => Period.ofYears(period.getYears).normalized())
Review comment:
did you push the commit?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
