spark-test commented on code in PR #59583:
URL: https://github.com/apache/spark/pull/59583#discussion_r4310463636
##########
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/V1WriteCommandSuite.scala:
##########
@@ -227,6 +227,25 @@ class V1WriteCommandSuite extends SharedSparkSession with
V1WriteCommandSuiteBas
}
}
+ test("v1 write with already sorted plan - floating-point partition column") {
+ withPlannedWrite { enabled =>
+ withTable("t") {
+ sql(
+ """
+ |CREATE TABLE t(i INT, k STRING) USING PARQUET
+ |PARTITIONED BY (j DOUBLE)
+ |""".stripMargin)
+ // Normalizing the partition values keeps them in order, so they are
not sorted again.
+ executeAndCheckOrderingAndCustomValidate(
+ hasLogicalSort = true, orderingMatched = Some(true)) {
+ sql("INSERT INTO t SELECT i, k, CAST(j AS DOUBLE) AS j FROM t0 ORDER
BY j")
Review Comment:
**Non-blocking (P2):** This fixture only casts integer-derived t0.j values
to DOUBLE, so its ordering and sort-count checks cannot detect loss of
normalization in the already-ordered legacy write branch. The separate
signed-zero/NaN test has no input ordering and reaches the unmatched branch
with planned writes disabled. If the orderingMatched branch used the original
plan instead of normalizedPlan, both tests would remain green even though an
already-sorted single-task input containing signed zeros or distinct NaN
payloads could create separate zero directories or revisit an existing
partition file. The sorted fixture needs exceptional partition values and
assertions on the resulting directories and preserved rows to cover this branch.
**Verification:**
- **Behavior:** For an already-ordered one-task floating dynamic-partition
write with planned writes disabled, assert matching input ordering, canonical
zero/NaN directories and exact row-ID preservation.
- **Regression:** Establish that omitting normalization only when legacy
input ordering matches causes the exceptional-value regression assertions or
write operation to fail, while correct normalization passes.
- **Compatibility:** Retain assertions that existing floating partition
ordering avoids an extra sort and that static negative-zero overwrite still
replaces its literal partition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
