MaxGekk commented on issue #26314: [SPARK-29371][SQL] Support interval field values with fractional parts URL: https://github.com/apache/spark/pull/26314#issuecomment-548291510 I modified slightly the benchmark and append fractions: ```diff diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/IntervalBenchmark.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/IntervalBenchmark.scala index d75cb1040f..ec32340b2b 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/IntervalBenchmark.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/IntervalBenchmark.scala @@ -83,8 +83,8 @@ object IntervalBenchmark extends SqlBasedBenchmark { override def runBenchmarkSuite(mainArgs: Array[String]): Unit = { val N = 1000000 val timeUnits = Seq( - "13 months", "100 weeks", "9 days", "12 hours", - "5 minutes", "45 seconds", "123 milliseconds", "567 microseconds") + "13.123 months", "100.123 weeks", "9.123 days", "12.123 hours", + "5.123 minutes", "45.123 seconds", "123.456 milliseconds", "567 microseconds") val intervalToTest = ListBuffer[String]() ``` and the results look not so nice: ``` Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15 Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz cast strings to intervals: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ prepare string w/ interval 440 497 51 2.3 440.3 1.0X prepare string w/o interval 370 392 21 2.7 370.3 1.2X 1 units w/ interval 5322 5414 142 0.2 5322.4 0.1X 1 units w/o interval 5002 5038 35 0.2 5002.2 0.1X 2 units w/ interval 18172 18285 189 0.1 18172.0 0.0X 2 units w/o interval 17899 18278 511 0.1 17899.4 0.0X 3 units w/ interval 30881 31481 665 0.0 30880.5 0.0X 3 units w/o interval 30839 31600 664 0.0 30838.5 0.0X 4 units w/ interval 44694 45302 547 0.0 44693.9 0.0X 4 units w/o interval 42490 42599 99 0.0 42489.5 0.0X 5 units w/ interval 54424 54469 67 0.0 54424.5 0.0X 5 units w/o interval 54879 55015 161 0.0 54879.1 0.0X 6 units w/ interval 69155 69281 140 0.0 69154.8 0.0X 6 units w/o interval 68877 69018 156 0.0 68876.7 0.0X 7 units w/ interval 82116 82204 112 0.0 82116.4 0.0X 7 units w/o interval 81495 81641 144 0.0 81495.4 0.0X 8 units w/ interval 94609 94630 22 0.0 94608.7 0.0X 8 units w/o interval 93915 94099 180 0.0 93914.8 0.0X 9 units w/ interval 93132 95566 2126 0.0 93132.2 0.0X 9 units w/o interval 91703 94179 NaN 0.0 91703.1 0.0X ``` The throughput is roughly ~10 interval strings with 8-9 units per second.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
