[
https://issues.apache.org/jira/browse/SPARK-9699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin resolved SPARK-9699.
--------------------------------
Resolution: Fixed
Assignee: Reynold Xin
I tested this locally and it is unclear when it should be turned on or off,
because in some cases it is still faster with it on. Safer to just keep it on
for now.
> Test whether we should turn compression on or off by default for external
> sorting
> ---------------------------------------------------------------------------------
>
> Key: SPARK-9699
> URL: https://issues.apache.org/jira/browse/SPARK-9699
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Reynold Xin
> Assignee: Reynold Xin
>
> As of Spark 1.4, compression is on by default for spilling, trading off CPU
> cycles for less I/O throughput usage. It is unclear whether it is better to
> have compression on or off when we spill data. We should test that and adjust
> the default setting accordingly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
