Maxim Gekk created SPARK-32006: ---------------------------------- Summary: Create date/timestamp formatters once before collect in `hiveResultString()` Key: SPARK-32006 URL: https://issues.apache.org/jira/browse/SPARK-32006 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.0.1, 3.1.0 Reporter: Maxim Gekk
Spark 2.4 re-uses one instance of SimpleDateFormat while formatting timestamps in toHiveString. Currently, toHiveString() creates timestampFormatter per each value. Even w/ caching, it causes additional overhead comparing to Spark 2.4. The ticket aims to create an instance of TimestampFormatter before collect in hiveResultString() -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org