LuciferYang commented on PR #53784: URL: https://github.com/apache/spark/pull/53784#issuecomment-3789663028
@zhengruifeng This patch has caused some strange issues, it resulted in the failure of the Maven daily test, - https://github.com/apache/spark/actions/runs/21249870076 <img width="1880" height="1166" alt="image" src="https://github.com/user-attachments/assets/86266699-536e-4832-b5e2-196f6b4cf067"; /> and I have reproduced it locally as follows: **Before this one:** ``` git reset --hard 7b379b77310c773e3f6eab2fc9a3a6b9496a8028 build/mvn clean install -DskipTests -pl sql/catalyst -am build/mvn test -pl sql/catalyst -DwildcardSuites=org.apache.spark.sql.RowJsonSuite RowJsonSuite: - IntegerType null - FloatType null - ArrayType(DoubleType,true) null - BooleanType true - BooleanType false - ByteType 23 - ByteType -126 - ShortType 20281 - ShortType -8752 - IntegerType 1078231987 - IntegerType -10 - LongType 139289832109874199 - LongType -7873748239973488 - FloatType 1.0232E11 - FloatType 9.7E-13 - DoubleType 3.891E98 - DoubleType -780000.0 - DecimalType(10,2) 1092.88 - DecimalType(7,4) 782.0003 - DecimalType(4,2) -77.89 - StringType hello world - BinaryType - DateType 2019-04-22 - DateType 2018-05-14 - TimestampType 2017-01-06 10:22:03.0 - TimestampType 2017-05-30T17:22:03Z - TimestampNTZType 2018-05-14T12:13 - ArrayType(LongType,true) - ArrayType(IntegerType,false) List(1, -2, 3) - MapType(StringType,StringType,true) Map(a -> b, c -> d, e -> null) - MapType(IntegerType,StringType,true) Map(1 -> b, 2 -> d, 3 -> null) - StructType(StructField(c1,StringType,true),StructField(c2,IntegerType,true)) [1,2] - UDT - no schema - unsupported type Run completed in 2 seconds, 18 milliseconds. Total number of tests run: 35 Suites: completed 2, aborted 0 Tests: succeeded 35, failed 0, canceled 0, ignored 0, pending 0 All tests passed. ``` **After this pr merged** ``` git reset --hard a8358c3b39d0601d7a2329f04b08e41497aafc36 build/mvn clean install -DskipTests -pl sql/catalyst -am build/mvn test -pl sql/catalyst -DwildcardSuites=org.apache.spark.sql.RowJsonSuite RowJsonSuite: - IntegerType null - FloatType null - ArrayType(DoubleType,true) null - BooleanType true - BooleanType false - ByteType 23 - ByteType -126 - ShortType 20281 - ShortType -8752 - IntegerType 1078231987 - IntegerType -10 - LongType 139289832109874199 - LongType -7873748239973488 - FloatType 1.0232E11 - FloatType 9.7E-13 - DoubleType 3.891E98 - DoubleType -780000.0 - DecimalType(10,2) 1092.88 - DecimalType(7,4) 782.0003 - DecimalType(4,2) -77.89 - StringType hello world - BinaryType - DateType 2019-04-22 *** FAILED *** JObject(List(("a", JString("2019-04-21")))) did not equal JObject(List(("a", JString("2019-04-22")))) (RowJsonSuite.scala:41) Analysis: JObject(obj: List(0: ("a", JString("2019-04-21")) -> ("a", JString("2019-04-22")))) - DateType 2018-05-14 - TimestampType 2017-01-06 10:22:03.0 *** FAILED *** JObject(List(("a", JString("2017-01-05 18:22:03")))) did not equal JObject(List(("a", JString("2017-01-06 10:22:03")))) (RowJsonSuite.scala:41) Analysis: JObject(obj: List(0: ("a", JString("2017-01-05 18:22:03")) -> ("a", JString("2017-01-06 10:22:03")))) - TimestampType 2017-05-30T02:22:03Z *** FAILED *** JObject(List(("a", JString("2017-05-29 19:22:03")))) did not equal JObject(List(("a", JString("2017-05-30 10:22:03")))) (RowJsonSuite.scala:41) Analysis: JObject(obj: List(0: ("a", JString("2017-05-29 19:22:03")) -> ("a", JString("2017-05-30 10:22:03")))) - TimestampNTZType 2018-05-14T12:13 - ArrayType(LongType,true) - ArrayType(IntegerType,false) List(1, -2, 3) - MapType(StringType,StringType,true) Map(a -> b, c -> d, e -> null) - MapType(IntegerType,StringType,true) Map(1 -> b, 2 -> d, 3 -> null) - StructType(StructField(c1,StringType,true),StructField(c2,IntegerType,true)) [1,2] - UDT - no schema - unsupported type Run completed in 2 seconds, 97 milliseconds. Total number of tests run: 35 Suites: completed 2, aborted 0 Tests: succeeded 32, failed 3, canceled 0, ignored 0, pending 0 *** 3 TESTS FAILED *** ``` I haven't delved deeply into investigating the cause of the failure yet. Could you take a look at it when you have time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
