[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-19 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/611#issuecomment-35570472 ah, so the sleep removed ;) and the synchronization block is already there, is it ok? --- If your project is set up for it, you can reply to this email and

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/611#discussion_r9855914 yes, seems preStart will be called by postRestart, and so it seems to me the flow will be prerestart -> postStop , postRestart -> preStart, So I think it

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/611#issuecomment-35471622 code updated. Btw, I add the sleep 1000 there, while I am wondering, there are zookeeper session time out, connection time etc. Seems to me then there actually

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/611#discussion_r9855368 prerestart will call into poststop later. so it should be ok. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/611#issuecomment-35461508 @aarondav , I don't run it with FaultToleranceTest, It seems to me it will need a docker installation? I did not play with docker before, I will try to s

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/611#discussion_r9852483 Actually, I thnk the LeaderLatch already synchronized on it's setLeadership method. And it will call isLeader/notLeader with sameThreadExecutor by execute m

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/611#discussion_r9852391 Yes, I think we could if no other possible LeaderElectionAgent will need it. And it will need to change the program flow say create some method to replace

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-17 Thread colorant
GitHub user colorant opened a pull request: https://github.com/apache/incubator-spark/pull/611 For SPARK-1082, Use Curator for ZK interaction in standalone cluster You can merge this pull request into a Git repository by running: $ git pull https://github.com/colorant

[GitHub] incubator-spark pull request: Minor fix for ZooKeeperPersistenceEn...

2014-02-11 Thread colorant
Github user colorant closed the pull request at: https://github.com/apache/incubator-spark/pull/583

[GitHub] incubator-spark pull request: Minor fix for ZooKeeperPersistenceEn...

2014-02-11 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/583#issuecomment-34838369 oops, catch me ;) updated

[GitHub] incubator-spark pull request: Minor fix for ZooKeeperPersistenceEn...

2014-02-11 Thread colorant
GitHub user colorant opened a pull request: https://github.com/apache/incubator-spark/pull/583 Minor fix for ZooKeeperPersistenceEngine to use configured working dir You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache

[GitHub] incubator-spark pull request: SPARK-1053. Don't require SPARK_YARN...

2014-02-11 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/553#issuecomment-34831254 I agree that we do not need a hard check on it :) This patch looks good to me, I just wonder we might need a few more explanation to clarify why this one is

[GitHub] incubator-spark pull request: SPARK-1053. Don't require SPARK_YARN...

2014-02-11 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/553#issuecomment-34830448 @sryza I think this depends on what you mean to do with the yarn-client mode, for spark-shell, since it is already packaged in the assembly jar. you don't