[spark] branch master updated: [SPARK-41948][SQL] Fix NPE for error classes: CANNOT_PARSE_JSON_FIELD

[maxgekk]

This is an automated email from the ASF dual-hosted git repository.

maxgekk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new cc1674d66ef [SPARK-41948][SQL] Fix NPE for error classes: 
CANNOT_PARSE_JSON_FIELD
cc1674d66ef is described below

commit cc1674d66ef34f540aa7bd5c7e465605e264e040
Author: panbingkun <pbk1...@gmail.com>
AuthorDate: Mon Jan 23 15:15:59 2023 +0300

    [SPARK-41948][SQL] Fix NPE for error classes: CANNOT_PARSE_JSON_FIELD
    
    ### What changes were proposed in this pull request?
    The pr aims to fix NPE for error classes: CANNOT_PARSE_JSON_FIELD.
    
    ### Why are the changes needed?
    1. When I want to delete redundant 'toString()' in code block as follow
    <img width="722" alt="image" 
src="https://user-images.githubusercontent.com/15246973/211269145-0f087bb1-dc93-480c-9f9d-afde5ac1c8de.png";>
    I found the UT("select from_json('[1, \"2\", 3]', 'array<int>')") failed.
    
    #### Why can it succeed before deletion?
    
    `parse.getCurrentName.toString()` => null.toString() => throw NPE, but 
follow logical can cover it,
    
https://github.com/apache/spark/blob/15a0f55246bee7b043bd6081f53744fbf74403eb/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala#L569-L573
    But obviously this is not our original intention.
    
    #### After deletion, The IllegalArgumentException will be thrown.
    `parse.getCurrentName` => throw java.lang.IllegalArgumentException as 
follow:
    `Caused by: java.lang.IllegalArgumentException: Cannot resolve variable 
'fieldName' (enableSubstitutionInVariables=false).
            at 
org.apache.commons.text.StringSubstitutor.substitute(StringSubstitutor.java:1532)
            at 
org.apache.commons.text.StringSubstitutor.substitute(StringSubstitutor.java:1389)
            at 
org.apache.commons.text.StringSubstitutor.replace(StringSubstitutor.java:893)
            at 
org.apache.spark.ErrorClassesJsonReader.getErrorMessage(ErrorClassesJSONReader.scala:51)
            ... 140 more
    `
    Above code can't handle IllegalArgumentException, so the UT failed.
    
    So, we should consider the case where `parse.getCurrentName` is null.
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    Pass GA.
    Existed UT.
    
    Closes #39466 from panbingkun/SPARK-41948.
    
    Authored-by: panbingkun <pbk1...@gmail.com>
    Signed-off-by: Max Gekk <max.g...@gmail.com>
---
 .../main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala | 4 ++--
 sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala | 3 ++-
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
index 8128c460602..9c8c764cf92 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
@@ -1443,8 +1443,8 @@ private[sql] object QueryExecutionErrors extends 
QueryErrorsBase {
     new SparkRuntimeException(
       errorClass = "CANNOT_PARSE_JSON_FIELD",
       messageParameters = Map(
-        "fieldName" -> parser.getCurrentName.toString(),
-        "fieldValue" -> parser.getText.toString(),
+        "fieldName" -> toSQLValue(parser.getCurrentName, StringType),
+        "fieldValue" -> parser.getText,
         "jsonType" -> jsonType.toString(),
         "dataType" -> toSQLType(dataType)))
   }
diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala 
b/sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala
index 6e16533eb30..57c54e88229 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala
@@ -27,6 +27,7 @@ import org.apache.commons.lang3.exception.ExceptionUtils
 import org.apache.spark.{SparkException, SparkRuntimeException}
 import org.apache.spark.sql.catalyst.InternalRow
 import org.apache.spark.sql.catalyst.expressions.{Literal, StructsToJson}
+import org.apache.spark.sql.catalyst.expressions.Cast._
 import org.apache.spark.sql.functions._
 import org.apache.spark.sql.internal.SQLConf
 import org.apache.spark.sql.test.SharedSparkSession
@@ -785,7 +786,7 @@ class JsonFunctionsSuite extends QueryTest with 
SharedSparkSession {
         exception = 
ExceptionUtils.getRootCause(exception).asInstanceOf[SparkRuntimeException],
         errorClass = "CANNOT_PARSE_JSON_FIELD",
         parameters = Map(
-          "fieldName" -> "a",
+          "fieldName" -> toSQLValue("a", StringType),
           "fieldValue" -> "1",
           "jsonType" -> "VALUE_STRING",
           "dataType" -> "\"INT\"")


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org