[jira] [Commented] (SPARK-27671) Fix error when casting from a nested null in a struct
[ https://issues.apache.org/jira/browse/SPARK-27671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839609#comment-16839609 ] Dongjoon Hyun commented on SPARK-27671: --- Thanks, [~viirya]. For now, it looks okay to me. > Fix error when casting from a nested null in a struct > - > > Key: SPARK-27671 > URL: https://issues.apache.org/jira/browse/SPARK-27671 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.0.0, 2.4.3 >Reporter: Liang-Chi Hsieh >Assignee: Liang-Chi Hsieh >Priority: Major > Fix For: 2.4.4, 3.0.0 > > > When a null in a nested field in struct, casting from the struct throws > error, currently. > {code} > scala> sql("select cast(struct(1, null) as struct)").show > scala.MatchError: NullType (of class org.apache.spark.sql.types.NullType$) > > at org.apache.spark.sql.catalyst.expressions.Cast.castToInt(Cast.scala:447) > > at org.apache.spark.sql.catalyst.expressions.Cast.cast(Cast.scala:635) > > at > org.apache.spark.sql.catalyst.expressions.Cast.$anonfun$castStruct$1(Cast.scala:603) > > {code} > {code} > scala> sql("select * FROM VALUES (('a', (10, null))), (('b', (10, 50))), > (('c', null)) AS tab(x, y)").show > org.apache.spark.sql.AnalysisException: failed to evaluate expression > named_struct('col1', 10, 'col2', NULL): NullType (of class > org.apache.spark.sql.t > ypes.NullType$); line 1 pos 14 > > at > org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:47) > > at > org.apache.spark.sql.catalyst.analysis.ResolveInlineTables.$anonfun$convert$6(ResolveInlineTables.scala:106) > > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27671) Fix error when casting from a nested null in a struct
[ https://issues.apache.org/jira/browse/SPARK-27671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839070#comment-16839070 ] Liang-Chi Hsieh commented on SPARK-27671: - [~dongjoon] Thanks for test and updating `Affects Version/s`. Do we need a patch for branch-2.3? If needed, please let me know. > Fix error when casting from a nested null in a struct > - > > Key: SPARK-27671 > URL: https://issues.apache.org/jira/browse/SPARK-27671 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.0.0, 2.4.3 >Reporter: Liang-Chi Hsieh >Assignee: Liang-Chi Hsieh >Priority: Major > Fix For: 2.4.4, 3.0.0 > > > When a null in a nested field in struct, casting from the struct throws > error, currently. > {code} > scala> sql("select cast(struct(1, null) as struct)").show > scala.MatchError: NullType (of class org.apache.spark.sql.types.NullType$) > > at org.apache.spark.sql.catalyst.expressions.Cast.castToInt(Cast.scala:447) > > at org.apache.spark.sql.catalyst.expressions.Cast.cast(Cast.scala:635) > > at > org.apache.spark.sql.catalyst.expressions.Cast.$anonfun$castStruct$1(Cast.scala:603) > > {code} > {code} > scala> sql("select * FROM VALUES (('a', (10, null))), (('b', (10, 50))), > (('c', null)) AS tab(x, y)").show > org.apache.spark.sql.AnalysisException: failed to evaluate expression > named_struct('col1', 10, 'col2', NULL): NullType (of class > org.apache.spark.sql.t > ypes.NullType$); line 1 pos 14 > > at > org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:47) > > at > org.apache.spark.sql.catalyst.analysis.ResolveInlineTables.$anonfun$convert$6(ResolveInlineTables.scala:106) > > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27671) Fix error when casting from a nested null in a struct
[ https://issues.apache.org/jira/browse/SPARK-27671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838795#comment-16838795 ] Dongjoon Hyun commented on SPARK-27671: --- The patch can land on `master` and `branch-2.4`. From branch-2.3, we need more patches. So, it's not feasible to backport. > Fix error when casting from a nested null in a struct > - > > Key: SPARK-27671 > URL: https://issues.apache.org/jira/browse/SPARK-27671 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.0.0, 2.4.3 >Reporter: Liang-Chi Hsieh >Priority: Major > > When a null in a nested field in struct, casting from the struct throws > error, currently. > {code} > scala> sql("select cast(struct(1, null) as struct)").show > scala.MatchError: NullType (of class org.apache.spark.sql.types.NullType$) > > at org.apache.spark.sql.catalyst.expressions.Cast.castToInt(Cast.scala:447) > > at org.apache.spark.sql.catalyst.expressions.Cast.cast(Cast.scala:635) > > at > org.apache.spark.sql.catalyst.expressions.Cast.$anonfun$castStruct$1(Cast.scala:603) > > {code} > {code} > scala> sql("select * FROM VALUES (('a', (10, null))), (('b', (10, 50))), > (('c', null)) AS tab(x, y)").show > org.apache.spark.sql.AnalysisException: failed to evaluate expression > named_struct('col1', 10, 'col2', NULL): NullType (of class > org.apache.spark.sql.t > ypes.NullType$); line 1 pos 14 > > at > org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:47) > > at > org.apache.spark.sql.catalyst.analysis.ResolveInlineTables.$anonfun$convert$6(ResolveInlineTables.scala:106) > > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org