adaitche commented on issue #24195: [SPARK-25496][SQL] Deprecate from_utc_timestamp and to_utc_timestamp URL: https://github.com/apache/spark/pull/24195#issuecomment-578050373 @MaxGekk I just tried it in PySpark (Spark version 2.4.4). Weirdly it doesn't work for `America/Los_Angeles`. Here is my code: ```python from pyspark.sql import SparkSession from pyspark.sql.functions import to_timestamp, concat_ws spark = (SparkSession .builder .config("spark.sql.session.timeZone", "UTC") .getOrCreate() ) df = spark.createDataFrame( [ ("2019-01-24 11:30:00.123", "America/Los_Angeles"), ("2020-01-01 01:30:00.123", "PST"), ], ["ts_str", "tz_name"] ) df = df.withColumn("ts_str_tz", F.concat_ws(" ", "ts_str", "tz_name")) df = df.withColumn("dt", F.to_timestamp("ts_str_tz", "yyyy-MM-dd HH:mm:ss.SSS z")) df.show(10, False) ``` This prints ``` +-----------------------+-------------------+-------------------------------------------+-------------------+ |ts_str |tz_name |ts_str_tz |dt | +-----------------------+-------------------+-------------------------------------------+-------------------+ |2019-01-24 11:30:00.123|America/Los_Angeles|2019-01-24 11:30:00.123 America/Los_Angeles|null | |2020-01-01 01:30:00.123|PST |2020-01-01 01:30:00.123 PST |2020-01-01 09:30:00| +-----------------------+-------------------+-------------------------------------------+-------------------+ ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
