[
https://issues.apache.org/jira/browse/SPARK-22600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290423#comment-16290423
]
Liang-Chi Hsieh edited comment on SPARK-22600 at 12/14/17 6:59 AM:
-------------------------------------------------------------------
The current approach proposes a new contract that Spark doesn't promise before:
{{Expression.genCode}} must output something that can be used as parameter name
or literal. If we output a java expression that can produce a value such as
{{var1 + 1}}, because it can't be used as parameter name, this approach will
fail the compilation.
To change the expression with a generated parameter name should be difficult as
we already use the expression in generated code.
If we accept this new contract, we should document it clearly and check if any
places use such expression as codegen output.
The current approach is documented in the design doc. Please give me feedbacks
if you have time to go through it. Thank you.
The design doc is posted at
https://docs.google.com/document/d/1By_V-A2sxCWbP7dZ5EzHIuMSe8K0fQL9lqovGWXnsfs/edit?usp=sharing
was (Author: viirya):
The current approach proposes a new contract that Spark doesn't promise before:
{{Expression.genCode}} must output something that can be used as parameter name
or literal. If we output a java expression that can produce a value such as
{{var1 + 1}}, because it can't be used as parameter name, this approach will
fail the compilation.
To change the expression with a generated parameter name should be difficult as
we already use the expression in generated code.
If we accept this new contract, we should document it clearly and check if any
places use such expression as codegen output.
The current approach is documented in the design doc.
The design doc is posted at
https://docs.google.com/document/d/1By_V-A2sxCWbP7dZ5EzHIuMSe8K0fQL9lqovGWXnsfs/edit?usp=sharing
> Fix 64kb limit for deeply nested expressions under wholestage codegen
> ---------------------------------------------------------------------
>
> Key: SPARK-22600
> URL: https://issues.apache.org/jira/browse/SPARK-22600
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Liang-Chi Hsieh
> Assignee: Liang-Chi Hsieh
> Fix For: 2.3.0
>
>
> This is an extension of SPARK-22543 to fix 64kb compile error for deeply
> nested expressions under wholestage codegen.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
