marmbrus commented on issue #24195: [SPARK-25496][SQL] Deprecate from_utc_timestamp and to_utc_timestamp URL: https://github.com/apache/spark/pull/24195#issuecomment-577445510 Hey everyone, I'm actually building a data pipeline right now and I just hit this. I think the solution we went with here is pretty actively hostile to users of Spark. I'm worried that failing this function behind a flag (that has been available since Spark 1.5) just because it has confusing semantics is really going to hinder the adoption of Spark 3.0. I would consider two different types of users here: - Someone building something new - Here, you give them no advice. We drop the ability work with timezones with no pointer on what they should do instead. It was also surprising to me that the function was both deprecated and threw and error from Scala. Everything I googled points back to this function. I'm now very frustrated. - Someone with an existing pipelines - Here, it probably doesn't matter that the function is "confusing", its doing what they want. Again, you don't give them a pointer to something better, you just annoy them by forcing them to change their code to specify this flag (and I assure you the vast majority of them will just set the flag without finding and reading this whole discussion). Given the above, I think this PR should be reverted until you can at a minimum point the users to something better.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
