[jira] [Commented] (SPARK-23029) Setting spark.shuffle.file.buffer will make the shuffle fail

2018-01-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323872#comment-16323872 ] Saisai Shao commented on SPARK-23029: - There should be no issue unless you want to fix doc issue...

[jira] [Commented] (SPARK-23029) Setting spark.shuffle.file.buffer will make the shuffle fail

2018-01-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323871#comment-16323871 ] Saisai Shao commented on SPARK-23029: - Total buffer size for one executor is {{32MB * C (core) * R

[jira] [Commented] (SPARK-23029) Setting spark.shuffle.file.buffer will make the shuffle fail

2018-01-12 Thread Fernando Pereira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323783#comment-16323783 ] Fernando Pereira commented on SPARK-23029: -- There is some bug here anyway. With buffer = 32768

[jira] [Commented] (SPARK-23029) Setting spark.shuffle.file.buffer will make the shuffle fail

2018-01-12 Thread Fernando Pereira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323777#comment-16323777 ] Fernando Pereira commented on SPARK-23029: -- IMHO it's a misleading syntax to accept 32 = 32k, as

[jira] [Commented] (SPARK-23029) Setting spark.shuffle.file.buffer will make the shuffle fail

2018-01-11 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323496#comment-16323496 ] Saisai Shao commented on SPARK-23029: - Please see the comments in the code in