MaxGekk commented on code in PR #56852:
URL: https://github.com/apache/spark/pull/56852#discussion_r3489906892
##########
python/pyspark/sql/functions/builtin.py:
##########
@@ -13179,6 +13179,55 @@ def timestamp_micros(col: "ColumnOrName") -> Column:
return _invoke_function_over_columns("timestamp_micros", col)
+@_try_remote_functions
+def timestamp_nanos(col: "ColumnOrName") -> Column:
+ """
+ Creates a nanosecond-precision timestamp (``TIMESTAMP_LTZ(9)``) from the
number of
+ nanoseconds since the UTC epoch.
+
+ .. versionadded:: 4.3.0
+
+ Parameters
+ ----------
+ col : :class:`~pyspark.sql.Column` or column name
+ a column of ``BIGINT`` or ``DECIMAL`` nanosecond values since the UTC
epoch.
+
+ Returns
+ -------
+ :class:`~pyspark.sql.Column`
+ a ``TIMESTAMP_LTZ(9)`` column representing the corresponding point in
time.
+
+ See Also
+ --------
+ :meth:`pyspark.sql.functions.timestamp_seconds`
+ :meth:`pyspark.sql.functions.timestamp_millis`
+ :meth:`pyspark.sql.functions.timestamp_micros`
+ :meth:`pyspark.sql.functions.unix_nanos`
+
+ Examples
+ --------
+ >>> import pyspark.sql.functions as sf
+ >>> spark.conf.set("spark.sql.timestampNanosTypes.enabled", "true")
Review Comment:
`timestamp_nanos` returns `TIMESTAMP_LTZ(9)`, whose `.show()` rendering
follows `spark.sql.session.timeZone`. This block sets only the nanos flag and
asserts the UTC wall-clock `2020-01-01 13:24:35.123456789`, but never pins the
timezone — and the doctest harness (`_test()`) sets none, so it uses the JVM/OS
default. Under a non-UTC default (e.g. `America/Los_Angeles` → `2020-01-01
05:24:35.123456789`) this doctest fails.
Every sibling `TIMESTAMP_LTZ` doctest pins UTC for this reason —
`timestamp_micros`, `timestamp_seconds`, `timestamp_millis`. Pin it here too:
```suggestion
>>> spark.conf.set("spark.sql.session.timeZone", "UTC")
>>> spark.conf.set("spark.sql.timestampNanosTypes.enabled", "true")
```
and add the matching `>>> spark.conf.unset("spark.sql.session.timeZone")`
next to the closing `unset` at the end of the block.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
