[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-64979607 Yeah the issue is that kill -9 risks causing legitimate shutdown behaviors to fail, and it's hard to reason about every case where that might be affected by this. Also, it will mean that we won't catch shutdown behavior bugs in the future because we have this brute-force mechanism. So as a first step it might be good to just diagnose specific shutdown problems. If we find a case where we have a buggy third party library or something where there is no alternative to killing, then we could justify this a bit better. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user WangTaoTheTonic closed the pull request at: https://github.com/apache/spark/pull/3316 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-64981884 OK, I understand it is better for catching bugs in future to keep not forcing to kill daemons. Thanks for comment and i will close this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-64966312 Okay - then I have the same question for the Thriftserver. Have you ever run jstack on it when it was hanging? I just think it might be good to fix the root cause instead of using kill -9. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-6497 Fixing the root cause is a good idea, but I think many reasons(probably simply because shutdown progress is slow or some other reasons) will lead daemons not being stopped and `spark-daemon.sh` is also used by Master and Slave. I will analyse what happened when Thrift Server not being stopped but it is harder for me to analyse Master or Slave while my cluster is using on yarn mode. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-64938358 Have not encountered that history server cann't not be stopped. It is always thrift server who is in that situation. I commit this way because we use the same script to stop them. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-64406710 [Test build #23840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23840/consoleFull) for PR 3316 at commit [`88bd312`](https://github.com/apache/spark/commit/88bd312a52efc539719afb4221e469a495305ce0). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-64421403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23840/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-64421394 [Test build #23840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23840/consoleFull) for PR 3316 at commit [`88bd312`](https://github.com/apache/spark/commit/88bd312a52efc539719afb4221e469a495305ce0). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
GitHub user WangTaoTheTonic opened a pull request: https://github.com/apache/spark/pull/3316 [SPARK-4451]force to kill process after 5 seconds When we restart history server or thrift server, sometimes the processes will not be launched as they were not totally stopping. So I think we'd better wait some seconds and kill it by force after that waiting. I do this referring to Hadoop. https://issues.apache.org/jira/browse/SPARK-4451 You can merge this pull request into a Git repository by running: $ git pull https://github.com/WangTaoTheTonic/spark master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3316.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3316 commit a7dff8c94e741e0f0ab9bc3d38c7f435b0e471b2 Author: WangTao barneystin...@aliyun.com Date: 2014-11-17T17:22:52Z force to kill process after 5 seconds --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-63343192 [Test build #23479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23479/consoleFull) for PR 3316 at commit [`12b3718`](https://github.com/apache/spark/commit/12b37180c590ee297563f16f3e66aa31872e49d2). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-63356639 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23479/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4451]force to kill process after 5 seco...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3316#issuecomment-63356629 [Test build #23479 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23479/consoleFull) for PR 3316 at commit [`12b3718`](https://github.com/apache/spark/commit/12b37180c590ee297563f16f3e66aa31872e49d2). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
