vboo123 opened a new pull request, #56797: URL: https://github.com/apache/spark/pull/56797
### What changes were proposed in this pull request? This PR adds end-to-end, golden-file, and proto round-trip test coverage for the TIME data type (`TimeType`) over Spark Connect. This is a test-only change with no engine or logic modifications. SPARK-57566 is a sub-task of the umbrella SPARK-57550 (extend TIME type support across Spark). **ClientE2ETestSuite** (Scala client, end-to-end against a real in-process Connect server): - `make_time` builtin returns a TIME value over Connect - `hour`, `minute`, `second` extract fields from a TIME value over Connect - `current_time` returns a TIME-typed column over Connect - A TIME column round-trips via `createDataFrame` over Connect - A TIME column round-trips through a parquet datasource over Connect The parquet datasource round-trip test verifies that a TIME column survives a real write-then-read through a file datasource over Connect, not just in-memory. This exercises parquet's TIME logical-type handling end-to-end and was added to satisfy the ticket's "createDataFrame / datasource round-trip" scope. **PlanGenerationTestSuite** (golden plan-generation): - Added `make_time`, `current_time`, and `current_time(precision)` function tests - Added the missing `LocalTime` case to the `function lit array` literal test (it already covered every other temporal type) - New and updated golden files generated via `SPARK_GENERATE_GOLDEN_FILES=1` and manually reviewed **LiteralExpressionProtoConverterSuite** (server-side proto/catalyst round-trip): - Added TIME literal round-trip coverage, including non-default precision propagation **Python note:** No new Python tests were added because Connect Python TIME coverage already exists through the parity suites. The classic PySpark tests `test_make_time`, `test_current_time`, and `test_create_dataframe_from_datetime_time` are re-run over a Connect session via `test_parity_functions` and `test_parity_creation`. The Python side is intentionally covered by existing parity, not omitted. ### Why are the changes needed? The TIME data type was added by SPIP SPARK-51162 and shipped in Spark 4.1.0, but Spark Connect previously had only minimal TIME coverage: a single `SELECT TIME '12:13:14'` literal test in `ClientE2ETestSuite`, `LocalTime` literals in `PlanGenerationTestSuite`, and schema handling in `SparkConnectPlannerSuite`. The client-to-protobuf-to-server round-trip and the TIME builtin functions over Connect were unverified. These tests close that gap. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? This is a test-only change. All scoped suites were run locally and passed: - `connect-client-jvm/testOnly org.apache.spark.sql.PlanGenerationTestSuite` — 727 tests passed (golden files regenerated and manually reviewed) - `connect/testOnly org.apache.spark.sql.connect.planner.LiteralExpressionProtoConverterSuite` — 44 tests passed - `connect-client-jvm/testOnly org.apache.spark.sql.connect.ClientE2ETestSuite -- -z "SPARK-57566"` — 5 tests passed ### Was this patch authored or co-authored using generative AI tooling? Yes. Generated using Kiro (Claude Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
