Github user ssonker commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r194319491
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,23 @@ object DateTimeUtils {
computedTimeZones.computeIfAbsent(timeZoneId, computeTimeZone)
}
+ private val threadLocalComputedCalendarsMap =
+ new ThreadLocal[mutable.Map[TimeZone, Calendar]] {
--- End diff --
I ran a benchmark and following is the output:
```string to timestamp calendar caching: Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
------------------------------------------------------------------------------------------------
with map 8 / 9 12.9
77.7 1.0X
without map 11 / 12 9.4
106.3 0.7X```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
