gaogaotiantian commented on code in PR #52980:
URL: https://github.com/apache/spark/pull/52980#discussion_r2516432892
##########
python/pyspark/worker.py:
##########
@@ -3307,6 +3309,13 @@ def main(infile, outfile):
if tracebackDumpIntervalSeconds is not None and
int(tracebackDumpIntervalSeconds) > 0:
faulthandler.dump_traceback_later(int(tracebackDumpIntervalSeconds),
repeat=True)
+ tzname = os.environ.get("SPARK_SESSION_LOCAL_TIMEZONE", None)
+ if tzname is not None:
+ tz = zoneinfo.ZoneInfo(tzname)
Review Comment:
So we need to consider whether this should be treated as a bug. I believe
Pandas UDF now uses this conf directly:
https://github.com/apache/spark/blob/f3f1449a9aac6d87c235f710bdbf9ffbbd715b9c/python/pyspark/worker.py#L2691
It would be weird if we have different behaviors between pandas UDF and
Python UDF right? If we consider this as a bug as mentioned in 33863, this PR
is just a bug fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
