(spark) branch master updated: [MINOR][TEST] Add output/exception to error message when schema not matched in `TPCDSQueryTestSuite`

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new e9697130a1ac [MINOR][TEST] Add output/exception to error message when 
schema not matched in `TPCDSQueryTestSuite`
e9697130a1ac is described below

commit e9697130a1acd2293d52ce72b9bddf6a203e3e8c
Author: Liang-Chi Hsieh <vii...@gmail.com>
AuthorDate: Mon Feb 5 09:43:44 2024 -0800

    [MINOR][TEST] Add output/exception to error message when schema not matched 
in `TPCDSQueryTestSuite`
    
    ### What changes were proposed in this pull request?
    
    This patch adds output/exception string to the error message when output 
schema not match expected schema in `TPCDSQueryTestSuite`.
    
    ### Why are the changes needed?
    
    We have used `TPCDSQueryTestSuite` for testing TPCDS query results. The 
test suite checks output schema and then output result. If any exception 
happens during query execution, it will handle the exception and return an 
empty schema and exception class + message as output. So, when any exception 
happens, the test suite just fails on schema check and never uses/shows the 
exception, e.g.,
    
    ```
    java.lang.Exception: Expected "struct<[count(1):bigint]>", but got 
"struct<[]>" Schema did not match
    ```
    
    We cannot see what exception was happened there from the log. It is somehow 
inconvenient for debugging.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No, test only.
    
    ### How was this patch tested?
    
    Existing tests
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No
    
    Closes #45025 from viirya/minor_ouput_exception.
    
    Authored-by: Liang-Chi Hsieh <vii...@gmail.com>
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
 .../test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala    | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala 
b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala
index ef7bdc2b079e..bde615552987 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala
@@ -139,7 +139,14 @@ class TPCDSQueryTestSuite extends QueryTest with TPCDSBase 
with SQLQueryTestHelp
           (segments(1).trim, segments(2).replaceAll("\\s+$", ""))
         }
 
-        assertResult(expectedSchema, s"Schema did not match\n$queryString") {
+        val notMatchedSchemaOutput = if (schema == emptySchema) {
+          // There might be exception. See `handleExceptions`.
+          s"Schema did not match\n$queryString\nOutput/Exception: 
$outputString"
+        } else {
+          s"Schema did not match\n$queryString"
+        }
+
+        assertResult(expectedSchema, notMatchedSchemaOutput) {
           schema
         }
         if (shouldSortResults) {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org