GitHub user saurfang opened a pull request:
https://github.com/apache/spark/pull/7076
[SPARK-8443][SQL] Split GenerateMutableProjection Codegen due to JVM Code
Size Limits
By grouping projection calls into multiple apply function, we are able to
push the number of projections codegen can handle from ~1k to ~60k. I have set
the unit test to test against 5k as 60k took 15s for the unit test to complete.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/saurfang/spark codegen_size_limit
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7076.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7076
----
commit 590a9f4b3f3ffba8e3a4fe7a6657de8d43cfc148
Author: Forest Fang <[email protected]>
Date: 2015-06-18T15:56:38Z
[SPARK-8443][SQL] split projection code by size limit
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
