marmbrus commented on issue #24195: [SPARK-25496][SQL] Deprecate from_utc_timestamp and to_utc_timestamp URL: https://github.com/apache/spark/pull/24195#issuecomment-578352360 @MaxGekk thanks for showing the examples on parsing timestamps with timezones! I think I will change my code to use this mechanism instead. That said, I remain strongly -1 on the idea that we should require users to set a flag to allow their code that was working to continue to work. I do not think that this is not how the Spark project operates. Look at the RDD API. There is a lot of cruft in there. That, however, is better than the alternative: breaking users programs or forcing them to stay on old versions. I'm less strongly -1 on the deprecation, but I do think that it *must* include pointers to the correct way to accomplish what the user is trying to do. IMO, this string concatenation suggestion is more of a hack than `from_utc_timestamp`. Given this, and the closeness to Spark 3.0. I still think the correct course of action is to back out this change and come back with a holistic plan for better timestamp handling in Spark. Separate but relatedly, I'm not sure how you figured that magic incantation for `to_timestamp`. Looking at the docs for `to_timestamp` I cannot find anything that tells me what is allowed in the format field. I think fixing that would be a very high impact activity.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
