[jira] [Commented] (YARN-5709) Cleanup Curator-based leader election code
[
https://issues.apache.org/jira/browse/YARN-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726301#comment-15726301
]
Karthik Kambatla commented on YARN-5709:
[~djp] - I was really hoping to clean up the configs in 2.8.0. The current
naming is very confusing, at least to me. Has been a while since I looked at it.
Will follow up on this with more clear explanations and likely a patch this
week. [~templedf] - okay if I pick this up?
> Cleanup Curator-based leader election code
> --
>
> Key: YARN-5709
> URL: https://issues.apache.org/jira/browse/YARN-5709
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.8.0
>Reporter: Karthik Kambatla
>Assignee: Daniel Templeton
>Priority: Critical
>
> While reviewing YARN-5677 and YARN-5694, I noticed we could make the
> curator-based election code cleaner. It is nicer to get this fixed in 2.8
> before we ship it, but this can be done at a later time as well.
> # By EmbeddedElector, we meant it was running as part of the RM daemon. Since
> the Curator-based elector is also running embedded, I feel the code should be
> checking for {{!curatorBased}} instead of {{isEmbeddedElector}}
> # {{LeaderElectorService}} should probably be named
> {{CuratorBasedEmbeddedElectorService}} or some such.
> # The code that initializes the elector should be at the same place
> irrespective of whether it is curator-based or not.
> # We seem to be caching the CuratorFramework instance in RM. It makes more
> sense for it to be in RMContext. If others are okay with it, we might even be
> better of having {{RMContext#getCurator()}} method to lazily create the
> curator framework and then cache it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5709) Cleanup Curator-based leader election code
[
https://issues.apache.org/jira/browse/YARN-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15709834#comment-15709834
]
Jian He commented on YARN-5709:
---
When I had offline discussion with [~xgong], the ultimate goal is to remove the
old EmbeddedElectorService, as we saw the hadoop common's elector
implementation kept on having issues one after another in our internal stress
testing. We wanted to just replace that with curator implementation. After
all, there's no point maintaining two.
bq. I feel the code should be checking for !curatorBased instead of
isEmbeddedElector
which code is this ?
bq. The code that initializes the elector should be at the same place
irrespective of whether it is curator-based or not.
Does this mean CuratorLeaderElector initialization should be inside the
AdminService ? I don't think it needs to be the case even for the old
EmbeddedElectorService too, if you look at the implementation, there's no
dependency between the EmbeddedElectorService and AdminService at all.
> Cleanup Curator-based leader election code
> --
>
> Key: YARN-5709
> URL: https://issues.apache.org/jira/browse/YARN-5709
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.8.0
>Reporter: Karthik Kambatla
>Assignee: Daniel Templeton
>Priority: Critical
>
> While reviewing YARN-5677 and YARN-5694, I noticed we could make the
> curator-based election code cleaner. It is nicer to get this fixed in 2.8
> before we ship it, but this can be done at a later time as well.
> # By EmbeddedElector, we meant it was running as part of the RM daemon. Since
> the Curator-based elector is also running embedded, I feel the code should be
> checking for {{!curatorBased}} instead of {{isEmbeddedElector}}
> # {{LeaderElectorService}} should probably be named
> {{CuratorBasedEmbeddedElectorService}} or some such.
> # The code that initializes the elector should be at the same place
> irrespective of whether it is curator-based or not.
> # We seem to be caching the CuratorFramework instance in RM. It makes more
> sense for it to be in RMContext. If others are okay with it, we might even be
> better of having {{RMContext#getCurator()}} method to lazily create the
> curator framework and then cache it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5709) Cleanup Curator-based leader election code
[
https://issues.apache.org/jira/browse/YARN-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15709739#comment-15709739
]
Junping Du commented on YARN-5709:
--
Sure. Thanks for update, Daniel. CC [~jianhe] in case he want to review this
code also.
> Cleanup Curator-based leader election code
> --
>
> Key: YARN-5709
> URL: https://issues.apache.org/jira/browse/YARN-5709
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.8.0
>Reporter: Karthik Kambatla
>Assignee: Daniel Templeton
>Priority: Critical
>
> While reviewing YARN-5677 and YARN-5694, I noticed we could make the
> curator-based election code cleaner. It is nicer to get this fixed in 2.8
> before we ship it, but this can be done at a later time as well.
> # By EmbeddedElector, we meant it was running as part of the RM daemon. Since
> the Curator-based elector is also running embedded, I feel the code should be
> checking for {{!curatorBased}} instead of {{isEmbeddedElector}}
> # {{LeaderElectorService}} should probably be named
> {{CuratorBasedEmbeddedElectorService}} or some such.
> # The code that initializes the elector should be at the same place
> irrespective of whether it is curator-based or not.
> # We seem to be caching the CuratorFramework instance in RM. It makes more
> sense for it to be in RMContext. If others are okay with it, we might even be
> better of having {{RMContext#getCurator()}} method to lazily create the
> curator framework and then cache it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5709) Cleanup Curator-based leader election code
[
https://issues.apache.org/jira/browse/YARN-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15709482#comment-15709482
]
Daniel Templeton commented on YARN-5709:
I was actually just about to start work on it today. It's something that
[~kasha] really wanted to get in before the confusing naming gets any more
entrenched.
> Cleanup Curator-based leader election code
> --
>
> Key: YARN-5709
> URL: https://issues.apache.org/jira/browse/YARN-5709
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.8.0
>Reporter: Karthik Kambatla
>Assignee: Daniel Templeton
>Priority: Critical
>
> While reviewing YARN-5677 and YARN-5694, I noticed we could make the
> curator-based election code cleaner. It is nicer to get this fixed in 2.8
> before we ship it, but this can be done at a later time as well.
> # By EmbeddedElector, we meant it was running as part of the RM daemon. Since
> the Curator-based elector is also running embedded, I feel the code should be
> checking for {{!curatorBased}} instead of {{isEmbeddedElector}}
> # {{LeaderElectorService}} should probably be named
> {{CuratorBasedEmbeddedElectorService}} or some such.
> # The code that initializes the elector should be at the same place
> irrespective of whether it is curator-based or not.
> # We seem to be caching the CuratorFramework instance in RM. It makes more
> sense for it to be in RMContext. If others are okay with it, we might even be
> better of having {{RMContext#getCurator()}} method to lazily create the
> curator framework and then cache it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5709) Cleanup Curator-based leader election code
[
https://issues.apache.org/jira/browse/YARN-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15709474#comment-15709474
]
Junping Du commented on YARN-5709:
--
Hi [~templedf], any progress on this issue? If not, shall we downgrade it to
major and defer to 2.9 given this is just refactor effort?
> Cleanup Curator-based leader election code
> --
>
> Key: YARN-5709
> URL: https://issues.apache.org/jira/browse/YARN-5709
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.8.0
>Reporter: Karthik Kambatla
>Assignee: Daniel Templeton
>Priority: Critical
>
> While reviewing YARN-5677 and YARN-5694, I noticed we could make the
> curator-based election code cleaner. It is nicer to get this fixed in 2.8
> before we ship it, but this can be done at a later time as well.
> # By EmbeddedElector, we meant it was running as part of the RM daemon. Since
> the Curator-based elector is also running embedded, I feel the code should be
> checking for {{!curatorBased}} instead of {{isEmbeddedElector}}
> # {{LeaderElectorService}} should probably be named
> {{CuratorBasedEmbeddedElectorService}} or some such.
> # The code that initializes the elector should be at the same place
> irrespective of whether it is curator-based or not.
> # We seem to be caching the CuratorFramework instance in RM. It makes more
> sense for it to be in RMContext. If others are okay with it, we might even be
> better of having {{RMContext#getCurator()}} method to lazily create the
> curator framework and then cache it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
