[
https://issues.apache.org/jira/browse/FLINK-14398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16952684#comment-16952684
]
Timo Walther commented on FLINK-14398:
--------------------------------------
[~jark] I agree, I think for the Blink planner we need a better code generation
framework at some point. This is also the point when we can reimplement this
code in Java.
As a short term solution, [~haodang] feel free to open a PR for further code
splitting. Should I assign this issue to you? The Blink planner is similar to
the Flink planner but has been further developed internally at Alibaba and was
open sourced recently. Many code parts look similar to the old planner. The old
planner is still the default planner for now.
> Codegen produced larger-than-64kb method
> ----------------------------------------
>
> Key: FLINK-14398
> URL: https://issues.apache.org/jira/browse/FLINK-14398
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Legacy Planner, Table SQL / Planner
> Affects Versions: 1.8.0
> Reporter: Hao Dang
> Priority: Major
> Attachments: codegen.example.txt
>
>
> In one of our production pipelines, we have a table with 1200+ columns. At
> runtime, it failed due to a method inside the generated code exceeding 64kb
> when compiled to bytecode.
> After we investigated the generated code, it appeared that the *map* method
> inside a generated *RichMapFunction* was too long. See attached file
> (codegen.example.txt).
> In the problematic *map* method, *result setters* were correctly split into
> individual methods and did not have the largest footprint.
> However, there were also 1000+ input unboxing expressions inside
> *reusableInputUnboxingExprs*, which, individually were not trivial and when
> flattened linearly in the *map* function
> [here|https://github.com/apache/flink/blob/release-1.8.0/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/codegen/FunctionCodeGenerator.scala#L187],
> pushed the method size beyond 64kb in bytecode.
> We think it is worthwhile to split these input unboxing code snippets into
> individual methods. We were able to verify, in our production environment,
> that splitting input unboxing code snippets into individual methods resolves
> the issue. Would love to hear thoughts from the team and find the best path
> to fix it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
