[
https://issues.apache.org/jira/browse/SPARK-16713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qifan Pu updated SPARK-16713:
-----------------------------
Description: Ideally, we would wish codegen methods to be less than 8KB for
bytecode size. Beyond 8K JIT won't compile and can cause performance
degradation. We have seen this for queries with wide schema (30+ fields), where
agg_doAggregateWithKeys() can be more than 8K. This is also a major reason for
performance regression when we enable fash aggregate hashmap (such as using
VectorizedHashMapGenerator.scala). (was: Ideally, we would wish codegen
methods to be less than 8KB for bytecode size. Beyond 8K JIT won't compile and
can cause performance degradation. We have seen this for queries with wide
schema (30+ fields). This is also a major reason for performance regression
when we enable fash aggregate hashmap (such as using
VectorizedHashMapGenerator.scala).)
> Limit codegen method size to 8KB
> --------------------------------
>
> Key: SPARK-16713
> URL: https://issues.apache.org/jira/browse/SPARK-16713
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core, SQL
> Affects Versions: 2.0.0
> Reporter: Qifan Pu
>
> Ideally, we would wish codegen methods to be less than 8KB for bytecode size.
> Beyond 8K JIT won't compile and can cause performance degradation. We have
> seen this for queries with wide schema (30+ fields), where
> agg_doAggregateWithKeys() can be more than 8K. This is also a major reason
> for performance regression when we enable fash aggregate hashmap (such as
> using VectorizedHashMapGenerator.scala).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
