[ 
https://issues.apache.org/jira/browse/SPARK-25084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575860#comment-16575860
 ] 

Lantao Jin commented on SPARK-25084:
------------------------------------

I offer other fix way. https://github.com/apache/spark/pull/22067
It doesn't need "input" as a global variable (If distribute by random)

> "distribute by" on multiple columns may lead to codegen issue
> -------------------------------------------------------------
>
>                 Key: SPARK-25084
>                 URL: https://issues.apache.org/jira/browse/SPARK-25084
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.3.1
>            Reporter: yucai
>            Priority: Blocker
>
> Test Query:
> {code:java}
> select * from store_sales distribute by (ss_sold_time_sk, ss_item_sk, 
> ss_customer_sk, ss_cdemo_sk, ss_addr_sk, ss_promo_sk) limit 1;{code}
> Exception:
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException: File 
> 'generated.java', Line 131, Column 67: failed to compile: 
> org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 
> 131, Column 67: One of ', )' expected instead of '['
> at 
> org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:1435)
> at 
> org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1497)
> at 
> org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1494)
> at 
> org.spark_project.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
> at 
> org.spark_project.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)
> at 
> org.spark_project.guava.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342){code}
> Wrong Codegen:
> {code:java}
> /* 131 */ private int computeHashForStruct_1(InternalRow 
> mutableStateArray[0], int value1) {
> /* 132 */
> /* 133 */
> /* 134 */ if (!mutableStateArray[0].isNullAt(5)) {
> /* 135 */
> /* 136 */ final int element5 = mutableStateArray[0].getInt(5);
> /* 137 */ value1 = 
> org.apache.spark.unsafe.hash.Murmur3_x86_32.hashInt(element5, value1);
> /* 138 */
> /* 139 */ }{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to