[
https://issues.apache.org/jira/browse/SPARK-9700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin updated SPARK-9700:
-------------------------------
Description:
Previously, we use 64MB as the default page size, which was way too big for a
lot of Spark applications (especially for single node).
This patch changes it so that the default page size, if unset by the user, is
determined by the number of cores available and the total execution memory
available.
was:
The default 64M does not work well with other default configs, it's easy to out
of memory, for example, 512M heap in local mode.
We should change this to 4M/8M/16M.
> Pick default page size more intelligently
> -----------------------------------------
>
> Key: SPARK-9700
> URL: https://issues.apache.org/jira/browse/SPARK-9700
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Davies Liu
> Assignee: Reynold Xin
> Priority: Blocker
>
> Previously, we use 64MB as the default page size, which was way too big for a
> lot of Spark applications (especially for single node).
> This patch changes it so that the default page size, if unset by the user, is
> determined by the number of cores available and the total execution memory
> available.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
