[ https://issues.apache.org/jira/browse/SPARK-25084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576417#comment-16576417 ]
yucai edited comment on SPARK-25084 at 8/10/18 3:17 PM: -------------------------------------------------------- [~smilegator], [~jerryshao] Thanks a lot for marking it blocker. A lot of eBay's tables use "distribute by" or "cluster by", it is important for us to move to Spark 2.3. was (Author: yucai): [~smilegator][~jerryshao] Thanks a lot for marking it blocker. A lot of eBay's tables use "distribute by" or "cluster by", it is important for us to move to Spark 2.3. > "distribute by" on multiple columns may lead to codegen issue > ------------------------------------------------------------- > > Key: SPARK-25084 > URL: https://issues.apache.org/jira/browse/SPARK-25084 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.3.1 > Reporter: yucai > Priority: Blocker > > Test Query: > {code:java} > select * from store_sales distribute by (ss_sold_time_sk, ss_item_sk, > ss_customer_sk, ss_cdemo_sk, ss_addr_sk, ss_promo_sk) limit 1;{code} > Exception: > {code:java} > Caused by: org.codehaus.commons.compiler.CompileException: File > 'generated.java', Line 131, Column 67: failed to compile: > org.codehaus.commons.compiler.CompileException: File 'generated.java', Line > 131, Column 67: One of ', )' expected instead of '[' > at > org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:1435) > at > org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1497) > at > org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1494) > at > org.spark_project.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599) > at > org.spark_project.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379) > at > org.spark_project.guava.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342){code} > Wrong Codegen: > {code:java} > /* 131 */ private int computeHashForStruct_1(InternalRow > mutableStateArray[0], int value1) { > /* 132 */ > /* 133 */ > /* 134 */ if (!mutableStateArray[0].isNullAt(5)) { > /* 135 */ > /* 136 */ final int element5 = mutableStateArray[0].getInt(5); > /* 137 */ value1 = > org.apache.spark.unsafe.hash.Murmur3_x86_32.hashInt(element5, value1); > /* 138 */ > /* 139 */ }{code} > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org