HyukjinKwon commented on a change in pull request #33877:
URL: https://github.com/apache/spark/pull/33877#discussion_r703562782
##########
File path: python/pyspark/pandas/data_type_ops/datetime_ops.py
##########
@@ -58,15 +66,18 @@ def sub(self, left: IndexOpsLike, right: Any) ->
SeriesOrIndex:
"The timestamp subtraction returns an integer in seconds, "
"whereas pandas returns 'timedelta64[ns]'."
)
- if isinstance(right, IndexOpsMixin) and
isinstance(right.spark.data_type, TimestampType):
+ if isinstance(right, IndexOpsMixin) and isinstance(
+ right.spark.data_type, (TimestampType, TimestampNTZType)
+ ):
warnings.warn(msg, UserWarning)
return left.astype("long") - right.astype("long")
Review comment:
This is not related to the change. It will hit the code path when:
pd.Timestamp - pd.Timestamp which will result in interval types.
Once we implement ones in PySpark, we should switch it to return interval
types. Currently, we return long values instead of interval types due to
missing interval in PySpark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
