zhengruifeng commented on code in PR #52980:
URL: https://github.com/apache/spark/pull/52980#discussion_r2522007767
##########
python/pyspark/worker.py:
##########
@@ -3307,6 +3309,13 @@ def main(infile, outfile):
if tracebackDumpIntervalSeconds is not None and
int(tracebackDumpIntervalSeconds) > 0:
faulthandler.dump_traceback_later(int(tracebackDumpIntervalSeconds),
repeat=True)
+ tzname = os.environ.get("SPARK_SESSION_LOCAL_TIMEZONE", None)
+ if tzname is not None:
+ tz = zoneinfo.ZoneInfo(tzname)
Review Comment:
+1 for a flag to control this.
even if this is a bug comparing with pandas udf, existing workflows might
already depend on it. We need a flag to restore old behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
