GitHub user kiszk opened a pull request:
https://github.com/apache/spark/pull/19797
[SPARK-22570][SQL] Avoid to create a lot of global variables to reuse an
object in generated code
## What changes were proposed in this pull request?
This PR reduces # of global variables in generated code by reusing an
object that can be held in one global variable. When a lot of global variables
were generated, the generated code may meet 64K constant pool limit.
This PR reduces # of generated global variables in the following three
operations:
* `Cast` with String to primitive byte/short/int/long
* `RegExpReplace`
* `CreateArray`
I intentionally leave [this
part](https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala#L595-L603).
This is because this variable keeps a class that is dynamically generated. In
other word, it is not possible to reuse one class.
## How was this patch tested?
Added test cases
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kiszk/spark SPARK-22570
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19797.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19797
----
commit 84610fb306713dfbe82ca72569fac09257b082bb
Author: Kazuaki Ishizaki <ishiz...@jp.ibm.com>
Date: 2017-11-22T16:47:45Z
initial commit for cast
commit 535160e63be200a722c7efba14a3d0befec47358
Author: Kazuaki Ishizaki <ishiz...@jp.ibm.com>
Date: 2017-11-22T16:48:08Z
initial commit for regexpreplace
commit 8ba2f59e209436c05479918fa7c5c70d6333b4b4
Author: Kazuaki Ishizaki <ishiz...@jp.ibm.com>
Date: 2017-11-22T16:48:30Z
createarray
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
