[GitHub] spark pull request #12054: [SPARK-14262] correct app's state after master le...

2017-03-02 Thread GavinGavinNo1
Github user GavinGavinNo1 closed the pull request at: https://github.com/apache/spark/pull/12054 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #16855: [SPARK-13931] Stage can hang if an executor fails while ...

2017-02-28 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the issue: https://github.com/apache/spark/pull/16855 @kayousterhout OK --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16855: [SPARK-13931] Stage can hang if an executor fails while ...

2017-02-28 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the issue: https://github.com/apache/spark/pull/16855 @kayousterhout OK, I have updated it --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16855: [SPARK-13931] Stage can hang if an executor fails...

2017-02-19 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r101940439 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,55 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16855: [SPARK-13931] Stage can hang if an executor fails...

2017-02-13 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100813496 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,55 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16855: [SPARK-13931] Stage can hang if an executor fails...

2017-02-09 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100356980 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,55 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16855: [SPARK-13931] Stage can hang if an executor fails...

2017-02-09 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100346247 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -874,7 +874,8 @@ private[spark] class TaskSetManager

[GitHub] spark pull request #11760: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread GavinGavinNo1
Github user GavinGavinNo1 closed the pull request at: https://github.com/apache/spark/pull/11760 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #11760: [SPARK-13931] Resolve stage hanging up problem in a part...

2017-02-08 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the issue: https://github.com/apache/spark/pull/11760 @kayousterhout I got some problem with git conflict. So I create a new branch and a new pull request. You may refer to https://github.com/apache/spark/pull/16855. And I close this pull

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread GavinGavinNo1
GitHub user GavinGavinNo1 reopened a pull request: https://github.com/apache/spark/pull/16855 [SPARK-13931] Resolve stage hanging up problem in a particular case ## What changes were proposed in this pull request? When function 'executorLost' is invoked in class 'TaskSetManager

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread GavinGavinNo1
Github user GavinGavinNo1 closed the pull request at: https://github.com/apache/spark/pull/16855 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread GavinGavinNo1
GitHub user GavinGavinNo1 opened a pull request: https://github.com/apache/spark/pull/16855 [SPARK-13931] Resolve stage hanging up problem in a particular case ## What changes were proposed in this pull request? When function 'executorLost' is invoked in class 'TaskSetManager

[GitHub] spark issue #11760: [SPARK-13931] Resolve stage hanging up problem in a part...

2017-02-06 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the issue: https://github.com/apache/spark/pull/11760 @kayousterhout Sorry that I tried before but internet connection is poor in my company and then I forget. I'll work on it back home today or tomorrow. Thank you for caring for this PR

[GitHub] spark pull request: [SPARK-13931] Resolve stage hanging up problem...

2016-04-11 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the pull request: https://github.com/apache/spark/pull/11760#issuecomment-208658733 @kayousterhout Sorry to bother you again. What should I do then? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-12552][Core]Correctly count the driver ...

2016-03-30 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the pull request: https://github.com/apache/spark/pull/10506#issuecomment-203702270 Thank you for your comment for PR #12054. I think changing app state from WAITING to RUNNING in function completeRecovery. Suppose that some app is WAITING

[GitHub] spark pull request: [SPARK-14262] correct appStatus after master l...

2016-03-29 Thread GavinGavinNo1
GitHub user GavinGavinNo1 opened a pull request: https://github.com/apache/spark/pull/12054 [SPARK-14262] correct appStatus after master leader changed ## What changes were proposed in this pull request? Suppose master leadership changes, even if recovery is completed

[GitHub] spark pull request: [SPARK-13931] Resolve stage hanging up problem...

2016-03-24 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the pull request: https://github.com/apache/spark/pull/11760#issuecomment-200860006 Is the unit test ok? Thanks. @mccheah @kayousterhout --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-13931] Resolve stage hanging up problem...

2016-03-23 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on a diff in the pull request: https://github.com/apache/spark/pull/11760#discussion_r57269057 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -776,6 +776,7 @@ private[spark] class TaskSetManager

[GitHub] spark pull request: [SPARK-13931] Resolve stage hanging up problem...

2016-03-23 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the pull request: https://github.com/apache/spark/pull/11760#issuecomment-200604346 @mccheah @kayousterhout OK, I'll add a logging line. But it's quite difficult to construct a reproduction of the case. I can add a test symbolically

[GitHub] spark pull request: [SPARK-13931] Resolve stage hanging up problem...

2016-03-23 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the pull request: https://github.com/apache/spark/pull/11760#issuecomment-200599739 @mccheah @kayousterhout Could you please take a look at this for me? Thank you in advance. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: Resolve stage hanging up problem in a particul...

2016-03-16 Thread GavinGavinNo1
GitHub user GavinGavinNo1 opened a pull request: https://github.com/apache/spark/pull/11760 Resolve stage hanging up problem in a particular case ## What changes were proposed in this pull request? When function 'executorLost' is invoked in class 'TaskSetManager', it's

[GitHub] spark pull request: [SPARK-10529][SQL]When creating multiple HiveC...

2015-09-18 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the pull request: https://github.com/apache/spark/pull/8713#issuecomment-141414230 @marmbrus Well, we both know that we can have multiple contexts. The difference is that it can't support continuous creating contexts. No matter how much size my

[GitHub] spark pull request: [SPARK-10529][SQL]When creating multiple HiveC...

2015-09-17 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the pull request: https://github.com/apache/spark/pull/8713#issuecomment-141073796 @marmbrus Thanks a lot. I'm so sorry I didn't make myself clear. I mean I'm not familiar with submitting an issue or contributing to spark. What you suggest I

[GitHub] spark pull request: [SPARK-10529][SQL]When creating multiple HiveC...

2015-09-15 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the pull request: https://github.com/apache/spark/pull/8713#issuecomment-140621464 @marmbrus Sorry to disturb again. Could you please give me a reply? It's my first try. Maybe I need some advice. --- If your project is set up for it, you can

[GitHub] spark pull request: Update IsolatedClientLoader.scala

2015-09-11 Thread GavinGavinNo1
GitHub user GavinGavinNo1 opened a pull request: https://github.com/apache/spark/pull/8713 Update IsolatedClientLoader.scala To resolve problem mentioned in SPARK-10529, I add an attribute of ThreadLocal type in object IsolatedClientLoader. So, no matter how many HiveContext

[GitHub] spark pull request: [SPARK-10529][SQL]When creating multiple HiveC...

2015-09-11 Thread GavinGavinNo1
Github user GavinGavinNo1 commented on the pull request: https://github.com/apache/spark/pull/8713#issuecomment-139702061 Thank you much for your comment. I think I haven't got what you mean for the ability to connect to multiple metastores.One HiveContext can only connect to one