This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 97bf1ee9f6f7 [SPARK-47301][SQL][TESTS][FOLLOWUP] Remove workaround for ParquetIOSuite 97bf1ee9f6f7 is described below commit 97bf1ee9f6f76d49df50560bf792135308f289a9 Author: panbingkun <panbing...@baidu.com> AuthorDate: Tue May 14 23:37:47 2024 +0800 [SPARK-47301][SQL][TESTS][FOLLOWUP] Remove workaround for ParquetIOSuite ### What changes were proposed in this pull request? The pr aims to remove workaround for ParquetIOSuite. ### Why are the changes needed? After https://github.com/apache/spark/pull/46562 is completed, the reason why the ut `SPARK-7837 Do not close output writer twice when commitTask() fails` failed due to different event processing time sequence no longer exists, so we remove the previous workaround here. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? - Manually test. - Pass GA. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #46577 from panbingkun/SPARK-47301_FOLLOWUP. Authored-by: panbingkun <panbing...@baidu.com> Signed-off-by: Wenchen Fan <wenc...@databricks.com> --- .../spark/sql/execution/datasources/parquet/ParquetIOSuite.scala | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala index ba8fef0b3a8d..4fb8faa43a39 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala @@ -1589,12 +1589,8 @@ class ParquetIOWithoutOutputCommitCoordinationSuite .coalesce(1) df.write.partitionBy("a").options(extraOptions).parquet(dir.getCanonicalPath) } - if (m2.getErrorClass != null) { - assert(m2.getErrorClass == "TASK_WRITE_FAILED") - assert(m2.getCause.getMessage.contains("Intentional exception for testing purposes")) - } else { - assert(m2.getMessage.contains("TASK_WRITE_FAILED")) - } + assert(m2.getErrorClass == "TASK_WRITE_FAILED") + assert(m2.getCause.getMessage.contains("Intentional exception for testing purposes")) } } } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org