[jira] [Commented] (YARN-5597) YARN Federation improvements
[
https://issues.apache.org/jira/browse/YARN-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606719#comment-16606719
]
Bibin A Chundatt commented on YARN-5597:
[~subru]/[~elgoiri]
{quote}
We use the same ZK ensemble and connection string so we are not having issues
here.
{quote}
In HA subcluster with same Zk for, leader election,RM store and Federation
Store + kerberos shouldn't have any issue as per understanding.
Above topology could be load on Zk, since all subclusters will write Store to
single ZK ensemble.
Why not have separate conf for FederaionStore connection string?
Mysql seems best fit now if zk security is required.
What is the clean up strategy for metadata ?.
In Federation store its not required to keep the apps list(router mapping to
app) once the apps is flushed out from RM memory/store rt ??
> YARN Federation improvements
>
>
> Key: YARN-5597
> URL: https://issues.apache.org/jira/browse/YARN-5597
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Subru Krishnan
>Assignee: Subru Krishnan
>Priority: Major
>
> This umbrella JIRA tracks set of improvements over the YARN Federation MVP
> (YARN-2915)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5597) YARN Federation improvements
[ https://issues.apache.org/jira/browse/YARN-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606535#comment-16606535 ] Íñigo Goiri commented on YARN-5597: --- We are currently testing federation using ZK for both the FederationStateStore and the RMStateStore. We use the same ZK ensemble and connection string so we are not having issues here. However, we haven't tested it with Kerberos yet. > YARN Federation improvements > > > Key: YARN-5597 > URL: https://issues.apache.org/jira/browse/YARN-5597 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Subru Krishnan >Assignee: Subru Krishnan >Priority: Major > > This umbrella JIRA tracks set of improvements over the YARN Federation MVP > (YARN-2915) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5597) YARN Federation improvements
[ https://issues.apache.org/jira/browse/YARN-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605094#comment-16605094 ] Subru Krishnan commented on YARN-5597: -- [~bibinchundatt], we use a RDBMS (SQL) for the Federation store and ZK for RM store as 1) there's no leader election in Federation and 2) We only store metadata for which a DB performs great and not what ZK is intended for (IMHO Zk has abused/misused a lot). That said, [~elgoiri] has a deployment with ZK for both Federation and RM stores, so he should be able to guide you. > YARN Federation improvements > > > Key: YARN-5597 > URL: https://issues.apache.org/jira/browse/YARN-5597 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Subru Krishnan >Assignee: Subru Krishnan >Priority: Major > > This umbrella JIRA tracks set of improvements over the YARN Federation MVP > (YARN-2915) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5597) YARN Federation improvements
[ https://issues.apache.org/jira/browse/YARN-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16577824#comment-16577824 ] Bibin A Chundatt commented on YARN-5597: [~subru] IIUC we have option of using Zookeeper as federation store and zookeeper as Resourcemanager State Store. In deployment are we suggesting to use single zookeeper instance for all the RM subclusters (as RMStateStore) and also for FederationStateStore ? While evaluating the store implementation of statestore + kerberos for zk connection if find following limitation Currently in zookeeper 3.4.6 doesn't support multiple secure zk client with different configuration on same jvm instance. ZOOKEEPER-2139 is included in 3.5.2 only. > YARN Federation improvements > > > Key: YARN-5597 > URL: https://issues.apache.org/jira/browse/YARN-5597 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Subru Krishnan >Assignee: Subru Krishnan >Priority: Major > > This umbrella JIRA tracks set of improvements over the YARN Federation MVP > (YARN-2915) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
