GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/12351
[SPARK-14593][SQL] Make currentVars work with splitExpressions to enable
whole stage codegen for large input columns
## What changes were proposed in this pull request?
JIRA: https://issues.apache.org/jira/browse/SPARK-14593
We now disable whole stage codegen if the input column number are too
large. If we can make `CodeGenContext.currentVars` work with
`CodeGenContext.splitExpressions`. We can enable whole stage codegen for large
input columns.
## How was this patch tested?
Existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 split-expres-current-vars
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12351.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12351
----
commit a5a79ada33a04ee205e4ccc71d42acbd36c008b4
Author: Liang-Chi Hsieh <[email protected]>
Date: 2016-04-13T06:26:50Z
Make currentVars work with splitExpressions to enable whole stage codegen
for large input columns.
commit 78c967360a95c8a8a9850697b4e0a358c3310ee5
Author: Liang-Chi Hsieh <[email protected]>
Date: 2016-04-13T06:27:56Z
Merge remote-tracking branch 'upstream/master' into
split-expres-current-vars
commit ccc72e118a29d5f1ca38d204aebca97ea9b70fcb
Author: Liang-Chi Hsieh <[email protected]>
Date: 2016-04-13T06:31:56Z
Remove debug code.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
