GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/7564
[SPARK-8850] [SQL] [WIP] Enable Unsafe mode by default
The purpose of this WIP pull request is to figure out which changes will be
necessary to enable Unsafe mode by default in Spark SQL.
This is currently rebased on top of #7560. In addition to that issue, I
need to make some default buffer sizes configurable so that
HiveCompatibilitySuite can run properly; I'll do this tomorrow.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark unsafe-by-default
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7564.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7564
----
commit 1b075567ee4f5a4e3bfece94b301f4a3ceb299cb
Author: navis.ryu <[email protected]>
Date: 2015-06-14T03:10:03Z
[SPARK-8357] [SQL] Memory leakage on unsafe aggregation path with empty
input
commit d396589c8669c2bbb38a1d025261d486e15f0c0a
Author: navis.ryu <[email protected]>
Date: 2015-06-14T05:14:13Z
added comments
commit 15c5afc1a1dcc052e248484f937ef0566aa6e6d4
Author: navis.ryu <[email protected]>
Date: 2015-06-22T08:29:18Z
added a test as suggested by JoshRosen
commit 4d326b908952e1c67eb0abc1c8a5e87375e5f519
Author: navis.ryu <[email protected]>
Date: 2015-06-22T13:38:36Z
fixed test fails
commit 51178e829e6321c9bd06424c81655abc1e90ed19
Author: navis.ryu <[email protected]>
Date: 2015-06-23T01:45:26Z
addressed comments
commit 1a02a5550fd579c704eb1ea282344f96479c3613
Author: navis.ryu <[email protected]>
Date: 2015-06-23T07:29:53Z
Rolled-back test-conf cleanup & fixed possible CCE & added more tests
commit 735972f8d894ddc6f302b9a1aaa3cd225b53a6cd
Author: navis.ryu <[email protected]>
Date: 2015-06-23T07:38:53Z
used new conf apis
commit 143e1ef8a50ca4752b2b129cbd0097ee22c3caaa
Author: navis.ryu <[email protected]>
Date: 2015-06-26T08:52:39Z
fixed format & added test for CCE case
commit c5419b37f2ece4842d8c2bd7463a50b61245fbcf
Author: navis.ryu <[email protected]>
Date: 2015-06-30T02:33:15Z
addressed comments
commit adc8239fae2c38eee492ed8dad15e0ae491986d3
Author: Josh Rosen <[email protected]>
Date: 2015-07-21T02:50:20Z
Back out Projection changes.
commit 3c7db0f21747aed8e563dc79c00c15a423cbf369
Author: Josh Rosen <[email protected]>
Date: 2015-07-21T02:51:37Z
Merge remote-tracking branch 'origin/master' into SPARK-8357
commit c649310bebfe2922383fb14c7c5f3344566eac85
Author: Josh Rosen <[email protected]>
Date: 2015-07-21T02:52:20Z
Revert SparkPlan change:
commit 3486ce4b04a99df8a81846dccbad1c2b97c41d31
Author: Josh Rosen <[email protected]>
Date: 2015-07-21T06:33:52Z
Some minor cleanup
commit ce55b32bbe8d762bd6d33e2026cb63d5044d69a5
Author: Josh Rosen <[email protected]>
Date: 2015-07-21T01:55:45Z
Enable Unsafe by default
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
