[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when tasks still to run

2017-08-07 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117241#comment-16117241 ] Thomas Graves commented on SPARK-21656: --- As a said above it DOES help the applicati

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when tasks still to run

2017-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117217#comment-16117217 ] Sean Owen commented on SPARK-21656: --- I do not understand what the bug is. Configuration

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when tasks still to run

2017-08-07 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117204#comment-16117204 ] Thomas Graves commented on SPARK-21656: --- Another option would be just to add logic

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when tasks still to run

2017-08-07 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117200#comment-16117200 ] Thomas Graves commented on SPARK-21656: --- why not fix the bug in dynamic allocation?

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when tasks still to run

2017-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117167#comment-16117167 ] Sean Owen commented on SPARK-21656: --- If the issue is "given more time" then increase th

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when tasks still to run

2017-08-07 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117159#comment-16117159 ] Thomas Graves commented on SPARK-21656: --- If given more time the scheduler would hav

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when tasks still to run

2017-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117122#comment-16117122 ] Sean Owen commented on SPARK-21656: --- Good point. In that case, what's wrong with killin

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when tasks still to run

2017-08-07 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117086#comment-16117086 ] Thomas Graves commented on SPARK-21656: --- The executor can be idle if the scheduler

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when tasks still to run

2017-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16116975#comment-16116975 ] Sean Owen commented on SPARK-21656: --- I don't see how an executor would be idle if there