GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/12479
[DO NOT MERGE] just for testing
like https://github.com/apache/spark/pull/12369 but with the default
changed to always spill after reading -- just a way to easily run tests in this
configuration.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark default_spill_after_read
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12479.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12479
----
commit 6d50ef94170a90db23a49feeb1f95621826539a3
Author: Imran Rashid <[email protected]>
Date: 2016-04-13T17:50:33Z
SpillableMemoryConsumer, just for clearer reporting
commit 14da2f7204e5d7ea04a466033330e6ceb75cad4b
Author: Imran Rashid <[email protected]>
Date: 2016-04-13T17:57:35Z
failing test cases
commit b74f215ceeabcd5e401f18478a7fd3954cad48ea
Author: Imran Rashid <[email protected]>
Date: 2016-04-13T18:55:43Z
add conf to force spilling after inserting all records into Spillable
commit 241f71208dac8dad7eba6aa13d065504861f21d7
Author: Imran Rashid <[email protected]>
Date: 2016-04-13T21:32:50Z
style
commit a3c1c12b5f9bc60baae7258956821f757754db75
Author: Imran Rashid <[email protected]>
Date: 2016-04-18T18:02:34Z
fix spill for empty collection, and tests
commit 162e899527a0cc5fa2d1eb52c74ced4536bf0ec4
Author: Imran Rashid <[email protected]>
Date: 2016-04-18T19:35:09Z
for testing only, change default to always spill after read
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
