[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-08-02 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-126997730 Do you mind closing this PR? I don't think we're making progress in discussing it and I don't think this is a correct change --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-08-02 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-127095058 ok. i close this. and will find the better way --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-08-02 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei closed the pull request at: https://github.com/apache/spark/pull/7559 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-27 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-125106918 I don't think it makes sense to add yet another piece of bookkeeping to patch over another problem. Update the existing data structures in the listener class. In any

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-27 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-125112620 @srowen In my picture, i just want to say the type of executor id is digital or character, eg: the executor id 5 and driver in picture. Now, do you

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-27 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-125115185 No, that's not what I'm referring to. It is hacky to filter executors based on numeric-or-not name but happens to work now. The problem is that most lines of code in

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-26 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-125069024 @srowen I am sorry not to make clear. My problem is : a executor is removed successfully but it exists in ExecutorsPage of the web ui. From code, i find

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-26 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on a diff in the pull request: https://github.com/apache/spark/pull/7559#discussion_r35503997 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsPage.scala --- @@ -58,7 +58,11 @@ private[ui] class ExecutorsPage( val

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-25 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-124814662 @KaiXinXiaoLei I think there are still some issues with your code, mostly that you're not explaining the change. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-124371932 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-124372643 [Test build #38329 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38329/consoleFull) for PR 7559 at commit

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-124372120 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-124448536 [Test build #38329 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38329/console) for PR 7559 at commit

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-124448628 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-24 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-124780545 I think my change code is not the reason of this failed test , And in my machine, this test passed. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-123397022 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-123399124 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-123399849 [Test build #37960 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37960/consoleFull) for PR 7559 at commit

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-123399173 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-123400328 [Test build #37960 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37960/console) for PR 7559 at commit

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-123400334 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-123183603 The log about this executor : 2015-07-17 11:48:20,762 | ERROR | [SIGTERM handler] | RECEIVED SIGNAL 15: SIGTERM 2015-07-17 11:48:20,993 | WARN |

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread KaiXinXiaoLei
GitHub user KaiXinXiaoLei opened a pull request: https://github.com/apache/spark/pull/7559 [SPARK-9209] Using executor allocation, a executor is removed but it exists in ExecutorsPage of the web ui I set spark.dynamicAllocation.enabled = true”, and run a big job. After some

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-123183920 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7559#issuecomment-123184251 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/7559#discussion_r35078099 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsPage.scala --- @@ -58,7 +58,11 @@ private[ui] class ExecutorsPage( val diskUsed =

[GitHub] spark pull request: [SPARK-9209] Using executor allocation, a exec...

2015-07-21 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/7559#discussion_r35078149 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -68,6 +72,7 @@ class ExecutorsListener(storageStatusListener: