[
https://issues.apache.org/jira/browse/SPARK-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14137698#comment-14137698
]
Michael Armbrust commented on SPARK-2902:
-----------------------------------------
Things we might consider changing: parquet auto conversion, increase batch
size, turn on compression.
As part of this it would be great if we read from spark defaults at startup so
these could be turned off without recompiling.
> Change default options to be more agressive
> -------------------------------------------
>
> Key: SPARK-2902
> URL: https://issues.apache.org/jira/browse/SPARK-2902
> Project: Spark
> Issue Type: Task
> Components: SQL
> Affects Versions: 1.0.1, 1.0.2
> Reporter: Cheng Lian
> Assignee: Cheng Lian
>
> Compression for in-memory columnar storage is disabled by default, it's time
> to enable it. Also, it help alleviating OOM mentioned in SPARK-2650
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
