This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-connect-swift.git
The following commit(s) were added to refs/heads/main by this push:
new 7682a68 [SPARK-52747] Test timezone variations
7682a68 is described below
commit 7682a68df6081f5fc2f2be89c00e5539d95269d2
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sat Aug 8 20:25:32 2026 -0700
[SPARK-52747] Test timezone variations
### What changes were proposed in this pull request?
This PR aims to test timezone variations in the `timestamp` test of
`DataFrameTests` by removing `TODO(SPARK-52747)` and verifying the behavior
explicitly under multiple session timezones.
- A timestamp literal with a timezone offset (`Z`) is verified to denote a
fixed instant independent of `spark.sql.session.timeZone` (`UTC` and
`America/Los_Angeles`).
- A timestamp literal without a timezone offset is verified to be
interpreted in the session timezone.
- The original `spark.sql.session.timeZone` setting is restored at the end
of the test.
### Why are the changes needed?
`TODO(SPARK-52747)` was added by SPARK-52744 as a workaround when the
`MacOS` integration test started to run a Spark Connect server directly on the
host whose timezone is not `UTC`. This is not a server behavior change but the
documented Spark semantics: a timestamp literal without a timezone offset is
interpreted in the session timezone. Since the Swift client converts Arrow
timestamp values into absolute instants (`Date(timeIntervalSince1970:)`), it
already handles timezone variati [...]
### Does this PR introduce _any_ user-facing change?
No. This is a test-only change.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Fable 5
Closes #493 from dongjoon-hyun/dongjoon/suspicious-bohr-3d780f.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
Tests/SparkConnectTests/DataFrameTests.swift | 26 ++++++++++++++++++--------
1 file changed, 18 insertions(+), 8 deletions(-)
diff --git a/Tests/SparkConnectTests/DataFrameTests.swift
b/Tests/SparkConnectTests/DataFrameTests.swift
index 2ebd0db..5df8109 100644
--- a/Tests/SparkConnectTests/DataFrameTests.swift
+++ b/Tests/SparkConnectTests/DataFrameTests.swift
@@ -1400,14 +1400,24 @@ struct DataFrameTests {
@Test
func timestamp() async throws {
let spark = try await SparkSession.builder.getOrCreate()
- // TODO(SPARK-52747)
- let df = try await spark.sql(
- "SELECT TIMESTAMP '2025-05-01 16:23:40Z', TIMESTAMP '2025-05-01
16:23:40.123456Z'")
- let expected = [
- Row(
- Date(timeIntervalSince1970: 1746116620.0), Date(timeIntervalSince1970:
1746116620.123456))
- ]
- #expect(try await df.collect() == expected)
+ let timeZone = try await spark.conf.get("spark.sql.session.timeZone")
+ // A timestamp literal with a timezone offset denotes a fixed instant
+ // independent of the session timezone.
+ for tz in ["UTC", "America/Los_Angeles"] {
+ try await spark.conf.set("spark.sql.session.timeZone", tz)
+ let df = try await spark.sql(
+ "SELECT TIMESTAMP '2025-05-01 16:23:40Z', TIMESTAMP '2025-05-01
16:23:40.123456Z'")
+ let expected = [
+ Row(
+ Date(timeIntervalSince1970: 1746116620.0),
Date(timeIntervalSince1970: 1746116620.123456))
+ ]
+ #expect(try await df.collect() == expected)
+ }
+ // A timestamp literal without a timezone offset is interpreted in the
session timezone.
+ try await spark.conf.set("spark.sql.session.timeZone",
"America/Los_Angeles")
+ let df = try await spark.sql("SELECT TIMESTAMP '2025-05-01 16:23:40'")
+ #expect(try await df.collect() == [Row(Date(timeIntervalSince1970:
1746141820.0))])
+ try await spark.conf.set("spark.sql.session.timeZone", timeZone)
await spark.stop()
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
