This is an automated email from the ASF dual-hosted git repository.
ueshin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new e2cd90bb3e1a [SPARK-55730][PYTHON] Not make timezone lower case
e2cd90bb3e1a is described below
commit e2cd90bb3e1a17021a580ae6d755e9acad8afd62
Author: Takuya Ueshin <[email protected]>
AuthorDate: Fri Feb 27 11:47:32 2026 -0800
[SPARK-55730][PYTHON] Not make timezone lower case
### What changes were proposed in this pull request?
Not make timezone lower case.
### Why are the changes needed?
The resolution of timezone has been changed in pandas 3:
-
https://pandas.pydata.org/pandas-docs/version/3.0/whatsnew/v3.0.0.html#pytz-now-an-optional-dependency
which caused the issue that a lowercase timezone is not recognized by
pandas' search in some environment.
```
...
File "/.../pandas/_libs/tslibs/timezones.pyx", line 133, in
pandas._libs.tslibs.timezones.maybe_get_tz
File "/.../pandas/_libs/tslibs/timezones.pyx", line 158, in
pandas._libs.tslibs.timezones.maybe_get_tz
File "/.../zoneinfo/_common.py", line 24, in load_tzdata
raise ZoneInfoNotFoundError(f"No time zone found with key {key}")
zoneinfo._common.ZoneInfoNotFoundError: 'No time zone found with key
etc/utc'
```
If it's `Etc/UTC`, it works.
### Does this PR introduce _any_ user-facing change?
Yes, it will behave more like pandas 3.
### How was this patch tested?
Manually.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #54528 from ueshin/issues/SPARK-55730/timezone.
Authored-by: Takuya Ueshin <[email protected]>
Signed-off-by: Takuya Ueshin <[email protected]>
---
python/pyspark/worker.py | 2 +-
python/pyspark/worker_util.py | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/python/pyspark/worker.py b/python/pyspark/worker.py
index a6ee3f25e486..380cfb96db48 100644
--- a/python/pyspark/worker.py
+++ b/python/pyspark/worker.py
@@ -158,7 +158,7 @@ class RunnerConf(Conf):
@property
def timezone(self) -> Optional[str]:
- return self.get("spark.sql.session.timeZone", None)
+ return self.get("spark.sql.session.timeZone", None, lower_str=False)
@property
def arrow_max_records_per_batch(self) -> int:
diff --git a/python/pyspark/worker_util.py b/python/pyspark/worker_util.py
index ccf937b5e734..74835afa60dd 100644
--- a/python/pyspark/worker_util.py
+++ b/python/pyspark/worker_util.py
@@ -246,8 +246,8 @@ class Conf:
v = utf8_deserializer.loads(infile)
self._conf[k] = v
- def get(self, key: str, default: Any = "") -> Any:
+ def get(self, key: str, default: Any = "", *, lower_str: bool = True) ->
Any:
val = self._conf.get(key, default)
- if isinstance(val, str):
+ if isinstance(val, str) and lower_str:
return val.lower()
return val
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
