[ https://issues.apache.org/jira/browse/SPARK-25084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan resolved SPARK-25084. --------------------------------- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22066 [https://github.com/apache/spark/pull/22066] > "distribute by" on multiple columns may lead to codegen issue > ------------------------------------------------------------- > > Key: SPARK-25084 > URL: https://issues.apache.org/jira/browse/SPARK-25084 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.3.1 > Reporter: yucai > Assignee: yucai > Priority: Blocker > Fix For: 2.4.0 > > > Test Query: > {code:java} > select * from store_sales distribute by (ss_sold_time_sk, ss_item_sk, > ss_customer_sk, ss_cdemo_sk, ss_addr_sk, ss_promo_sk) limit 1;{code} > Exception: > {code:java} > Caused by: org.codehaus.commons.compiler.CompileException: File > 'generated.java', Line 131, Column 67: failed to compile: > org.codehaus.commons.compiler.CompileException: File 'generated.java', Line > 131, Column 67: One of ', )' expected instead of '[' > at > org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:1435) > at > org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1497) > at > org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1494) > at > org.spark_project.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599) > at > org.spark_project.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379) > at > org.spark_project.guava.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342){code} > Wrong Codegen: > {code:java} > /* 131 */ private int computeHashForStruct_1(InternalRow > mutableStateArray[0], int value1) { > /* 132 */ > /* 133 */ > /* 134 */ if (!mutableStateArray[0].isNullAt(5)) { > /* 135 */ > /* 136 */ final int element5 = mutableStateArray[0].getInt(5); > /* 137 */ value1 = > org.apache.spark.unsafe.hash.Murmur3_x86_32.hashInt(element5, value1); > /* 138 */ > /* 139 */ }{code} > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org