GitHub user kapilsingh5050 opened a pull request:
https://github.com/apache/spark/pull/15620
[SPARK-18091] [SQL] Deep if expressions cause Generated
SpecificUnsafeProjection code to exceed JVM code size limit
## What changes were proposed in this pull request?
Fix for SPARK-18091 which is a bug related to large if expressions causing
generated SpecificUnsafeProjection code to exceed JVM code size limit.
This PR changes if expression's code generation to place its predicate,
true value and false value expressions' generated code in separate methods in
context so as to never generate too long combined code.
## How was this patch tested?
Added a unit test and also tested manually with the application (having
transformations similar to the unit test) which caused the issue to be
identified in the first place.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kapilsingh5050/spark SPARK-18091-IfCodegenFix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/15620.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #15620
----
commit 62f7b4e6880681c0ea21b6824e5e8225fcdd8f84
Author: Kapil Singh <[email protected]>
Date: 2016-10-25T09:33:35Z
Fixed If expression code generation to never generate code large enough to
exceed JVM method code size limit
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
