[jira] [Comment Edited] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15773594#comment-15773594 ] Chen He edited comment on SPARK-18988 at 12/23/16 8:06 PM: --- The "automatically"

[jira] [Commented] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15773594#comment-15773594 ] Chen He commented on SPARK-18988: - The "automatically" means HistoryServer will create it during startup.

[jira] [Comment Edited] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15773186#comment-15773186 ] Chen He edited comment on SPARK-18988 at 12/23/16 4:18 PM: --- OK, if this is

[jira] [Comment Edited] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15773186#comment-15773186 ] Chen He edited comment on SPARK-18988 at 12/23/16 4:16 PM: --- OK, if this is

[jira] [Comment Edited] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15773186#comment-15773186 ] Chen He edited comment on SPARK-18988 at 12/23/16 4:10 PM: --- OK, if this is

[jira] [Reopened] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He reopened SPARK-18988: - > Spark "spark.eventLog.dir" dir should create the directory if it is different > from

[jira] [Commented] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15773186#comment-15773186 ] Chen He commented on SPARK-18988: - OK, if this is reasonable, why "spark.history.fs.logDirectory" can be

[jira] [Updated] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He updated SPARK-18988: Affects Version/s: 2.1.0 > Spark "spark.eventLog.dir" dir should create the directory if it is different

[jira] [Commented] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15772317#comment-15772317 ] Chen He commented on SPARK-18988: - It should be like: {code} if (!fileSystem.getFileStatus(new

[jira] [Updated] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He updated SPARK-18988: Description: When set "spark.history.fs.logDirectory" to be hdfs:///spark-history but set

[jira] [Updated] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

[ https://issues.apache.org/jira/browse/SPARK-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He updated SPARK-18988: Description: When set "spark.history.fs.logDirectory" to be hdfs:///spark-history but set

[jira] [Created] (SPARK-18988) Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory"

Chen He created SPARK-18988: --- Summary: Spark "spark.eventLog.dir" dir should create the directory if it is different from "spark.history.fs.logDirectory" Key: SPARK-18988 URL:

[jira] [Created] (SPARK-18968) .sparkStaging quickly fill up HDFS

Chen He created SPARK-18968: --- Summary: .sparkStaging quickly fill up HDFS Key: SPARK-18968 URL: https://issues.apache.org/jira/browse/SPARK-18968 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13628) Temporary intermediate output file should be renamed before copying to destination filesystem

[ https://issues.apache.org/jira/browse/SPARK-13628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187419#comment-15187419 ] Chen He commented on SPARK-13628: - Really appreciate your reply, Mr. Sean Owen. I am new to Spark and

[jira] [Created] (SPARK-13628) Temporary intermediate output file should be renamed before copying to destination filesystem

Chen He created SPARK-13628: --- Summary: Temporary intermediate output file should be renamed before copying to destination filesystem Key: SPARK-13628 URL: https://issues.apache.org/jira/browse/SPARK-13628

[jira] [Commented] (SPARK-2277) Make TaskScheduler track whether there's host on a rack

[ https://issues.apache.org/jira/browse/SPARK-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050381#comment-14050381 ] Chen He commented on SPARK-2277: This is interesting. I will take a look. Make