[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406672#comment-16406672 ] Íñigo Goiri commented on HDFS-13215: Everything looks good; I tried turnk and branch-2.9 and build properly. Thanks [~ywskycn] for taking this effort. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-2.9.003.patch, > HDFS-13215_branch-2.9.004.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.0.002.patch, HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405846#comment-16405846 ] Yiqun Lin commented on HDFS-13215: -- LGTM, +1. Thanks [~ywskycn]. Please go ahead, :). > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-2.9.003.patch, > HDFS-13215_branch-2.9.004.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.0.002.patch, HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405841#comment-16405841 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color} | {color:red} HDFS-13215 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HDFS-13215 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12915261/HDFS-13215_branch-2.9.004.patch | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/23554/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-2.9.003.patch, > HDFS-13215_branch-2.9.004.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.0.002.patch, HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405842#comment-16405842 ] Wei Yan commented on HDFS-13215: [~linyiqun] any other comments? I just pushed the trunk patch, and the integration looks ok. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-2.9.003.patch, > HDFS-13215_branch-2.9.004.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.0.002.patch, HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405836#comment-16405836 ] Wei Yan commented on HDFS-13215: Thanks [~linyiqun]. There are some files generated by intellij when switching branch... let me remove them. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-2.9.003.patch, > HDFS-13215_branch-3.0.001.patch, HDFS-13215_branch-3.0.002.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405830#comment-16405830 ] Hudson commented on HDFS-13215: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13858 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13858/]) HDFS-13215. RBF: Move Router to its own module. Contributed by Wei Yan (weiy: rev 6e2b5fa493ff8e8c2bb28e6f6f4c19347bc9b99d) * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/site/resources/css/site.css * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/resolver/TestMultipleDestinationResolver.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterAdminCLI.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/GetNamespaceInfoRequest.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/GetRouterRegistrationsRequest.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/RemoveMountTableEntryRequest.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/impl/pb/EnterSafeModeRequestPBImpl.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RemoteParam.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/federation/store/driver/TestStateStoreDriverBase.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/EnterSafeModeRequest.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/federation/resolver/order/TestLocalResolver.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/ConnectionPool.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/store/records/impl/pb/PBRecord.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/driver/TestStateStoreFile.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterHeartbeatService.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/FederationNamenodeContext.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterMountTable.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterAdminServer.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/resolver/TestNamenodeResolver.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/store/CachedRecordStore.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/Router.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/impl/pb/GetRouterRegistrationRequestPBImpl.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/order/HashFirstResolver.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/records/Query.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRpcMultiDestination.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/impl/pb/FederationProtocolPBTranslator.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/driver/TestStateStoreFileBase.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/impl/pb/NamenodeHeartbeatRequestPBImpl.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/GetRouterRegistrationRequest.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileBaseImpl.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterSafemode.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/driver/TestStateStoreZK.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protoco
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405828#comment-16405828 ] Yiqun Lin commented on HDFS-13215: -- Thanks [~ywskycn] for updating the patch. Seems there are some unrelated files added in [^HDFS-13215_branch-2.9.003.patch]. Would you have a check? > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-2.9.003.patch, > HDFS-13215_branch-3.0.001.patch, HDFS-13215_branch-3.0.002.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405785#comment-16405785 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color} | {color:red} HDFS-13215 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HDFS-13215 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12915251/HDFS-13215_branch-3.0.002.patch | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/23551/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-2.9.003.patch, > HDFS-13215_branch-3.0.001.patch, HDFS-13215_branch-3.0.002.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405776#comment-16405776 ] Wei Yan commented on HDFS-13215: Thanks [~linyiqun]. Uploaded [^HDFS-13215_branch-2.9.003.patch] [^HDFS-13215_branch-3.0.002.patch] to remove quota-related fields. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-2.9.003.patch, > HDFS-13215_branch-3.0.001.patch, HDFS-13215_branch-3.0.002.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405734#comment-16405734 ] Yiqun Lin commented on HDFS-13215: -- Thanks [~ywskycn] for the hard work! Just one comment for the patch HDFS-13215_branch-2.9.002.patch and HDFS-13215_branch-3.0.001.patch. In branch-2.9 and branch-3.0, we didn't merge global quota implementation. So the following change for this two branch should be removed: *RBFConfigKeys.java* {noformat} + // HDFS Router-based federation quota + public static final String DFS_ROUTER_QUOTA_ENABLE = + FEDERATION_ROUTER_PREFIX + "quota.enable"; + public static final boolean DFS_ROUTER_QUOTA_ENABLED_DEFAULT = false; + public static final String DFS_ROUTER_QUOTA_CACHE_UPATE_INTERVAL = + FEDERATION_ROUTER_PREFIX + "quota-cache.update.interval"; + public static final long DFS_ROUTER_QUOTA_CACHE_UPATE_INTERVAL_DEFAULT = + 6; {noformat} *hdfs-rbf-default.xml* {noformat} + +dfs.federation.router.quota.enable +false + + Set to true to enable quota system in Router. + + + + +dfs.federation.router.quota-cache.update.interval +60s + + Interval time for updating quota usage cache in Router. + This property is used only if the value of + dfs.federation.router.quota.enable is true. + This setting supports multiple time unit suffixes as described + in dfs.heartbeat.interval. If no suffix is specified then milliseconds + is assumed. + + {noformat} Others looks good to me. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405698#comment-16405698 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 28s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 79 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 30s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 5s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 50s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 11m 9s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 8m 55s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 13s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 33s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 30s{color} | {color:red} hadoop-hdfs-rbf in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m 56s{color} | {color:red} hadoop-hdfs-project in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 17m 42s{color} | {color:red} root in the patch failed. {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 13m 36s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 13m 36s{color} | {color:red} root generated 159 new + 1113 unchanged - 159 fixed = 1272 total (was 1272) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 3m 0s{color} | {color:orange} root: The patch generated 3 new + 431 unchanged - 3 fixed = 434 total (was 434) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 12m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 0s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 8s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 16s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 48s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m 53s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405501#comment-16405501 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 79 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 32s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m 34s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 54s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 14m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 34s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m 28s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 6s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 11m 46s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 11m 46s{color} | {color:red} root generated 161 new + unchanged - 161 fixed = 1272 total (was 1272) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 2m 45s{color} | {color:orange} root: The patch generated 3 new + 431 unchanged - 3 fixed = 434 total (was 434) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 0s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 12s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 13s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 8m 42s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 45s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}126m 9s{color} | {color:red} root in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 43s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405493#comment-16405493 ] Wei Yan commented on HDFS-13215: Thanks so much for the reviews.. I'll wait until tonight to check if [~linyiqun] has more comments before commiting. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405492#comment-16405492 ] Íñigo Goiri commented on HDFS-13215: [^HDFS-13215_branch-2.9.002.patch] and [^HDFS-13215_branch-2.002.patch] work now. +1 on the next 5: * [^HDFS-13215_branch-2.002.patch] * [^HDFS-13215_branch-2.9.002.patch] * [^HDFS-13215_branch-3.0.001.patch] * [^HDFS-13215_branch-3.1.001.patch] * [^HDFS-13215.009.patch] > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405490#comment-16405490 ] Wei Yan commented on HDFS-13215: Uploaded [^HDFS-13215_branch-2.002.patch], [^HDFS-13215_branch-2.9.002.patch] to fix the dist issue for branch-2 > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.002.patch, HDFS-13215_branch-2.9.001.patch, > HDFS-13215_branch-2.9.002.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405474#comment-16405474 ] Wei Yan commented on HDFS-13215: I mixed a change in hadoop-dist/pom.xml, where it has the list of "cp" cmds. Adding it > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.9.001.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405466#comment-16405466 ] Íñigo Goiri commented on HDFS-13215: For [^HDFS-13215_branch-2.001.patch] and [^HDFS-13215_branch-2.9.001.patch], the tgz didn't contain the RBF jars. I think is missing the equivalent of the dev-support/bin/dist-layout-stitching for branch-2. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.9.001.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405415#comment-16405415 ] Íñigo Goiri commented on HDFS-13215: +1 on [^HDFS-13215.009.patch]. Checking the others now. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.9.001.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405366#comment-16405366 ] Wei Yan commented on HDFS-13215: Uploaded 4 patches for the other 4 branches. Verified build and testcases locally. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch, HDFS-13215_branch-2.001.patch, > HDFS-13215_branch-2.9.001.patch, HDFS-13215_branch-3.0.001.patch, > HDFS-13215_branch-3.1.001.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405146#comment-16405146 ] Wei Yan commented on HDFS-13215: Upload [^HDFS-13215.009.patch] for fixing the missing findbugs. Patches for other branches will be uploaded soon. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch, HDFS-13215.009.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405128#comment-16405128 ] Íñigo Goiri commented on HDFS-13215: I tested fully and it looks good. I checked the tgz and it has all the files. The documentation seems ok too. The javac are not related. The checkstyles are fine (number of arguments). Just pending the {{findbugsExcludeFile.xml}} issue that [~linyiqun] mentioned. Thanks [~ywskycn] for taking care of this. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405086#comment-16405086 ] Wei Yan commented on HDFS-13215: Good catch, [~linyiqun]. Looks like findbugs didn't report any issue even we missed this. I'll add it back to hadoop-hdfs-rbf in the next patch. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16404606#comment-16404606 ] Yiqun Lin commented on HDFS-13215: -- Just did a quick look, caught one thing: {noformat} --- hadoop-hdfs-project/hadoop-hdfs/dev-support/findbugsExcludeFile.xml +++ hadoop-hdfs-project/hadoop-hdfs/dev-support/findbugsExcludeFile.xml @@ -14,9 +14,6 @@ - - - {noformat} We removed this in hadoop-hdfs, but missing to add in {{hadoop-hdfs-project/hadoop-hdfs-rbf/dev-support/findbugsExcludeFile.xml}} > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16404400#comment-16404400 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 37s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 79 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 21s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 12s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m 50s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 8m 11s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 48s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 3s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 29s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m 29s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 12m 29s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 12m 29s{color} | {color:red} root generated 161 new + unchanged - 161 fixed = 1272 total (was 1272) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 2m 38s{color} | {color:orange} root: The patch generated 3 new + 431 unchanged - 3 fixed = 434 total (was 434) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 11m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 1s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 7s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 13s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 8m 19s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 46s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 39s{color} | {color:red} root in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 29s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16404378#comment-16404378 ] Wei Yan commented on HDFS-13215: Thanks [~elgoiri] for the detailed review. Meanwhile, I'll prepare patches for other branches. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16404367#comment-16404367 ] Íñigo Goiri commented on HDFS-13215: Thanks [~ywskycn], after [^HDFS-13215.008.patch] the unit tests pass. Having the log4j.properties file I think is also good practice so we are good. This LGTM but let me do a full test checking the dist, deploying, etc. I should be able to do a full pass tomorrow. [~linyiqun], do you mind reviewing it and check if we left something? > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16404338#comment-16404338 ] Wei Yan commented on HDFS-13215: Thanks, [~elgoiri]. Upload [^HDFS-13215.008.patch] to address the comments. All rbf site and .xml files moved to hadoop-hdfs-rbf module. Regarding the test failure, I think it is because of log4j configuration. By default, the log4j debug is disable, so in my env doesn't have the failure; but in your running, the debug mode is enabled, which triggers "FSEditLogAsync.java:128". I add the log4j.properties there, which may fix the issue. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch, > HDFS-13215.008.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16404141#comment-16404141 ] Íñigo Goiri commented on HDFS-13215: Thanks [~ywskycn] for [^HDFS-13215.007.patch]. I would also move the RBF documentation to the new package. I'm not sure we should have {{hdfs-rbf-default.xml}} in {{hadoop-hdfs}}, I think it should go to {{hadoop-hdfs-rbf}}. Then {{hdfs-rbf-default.xml}} should not point to {{hdfs-site.xml}}, right? Regarding the unit tests, the tweak with hdfs-site.xml fixed most of the issues. Now I get: {code} [ERROR] Errors: [ERROR] TestRouterAdminCLI.testCreateInvalidEntry:444 » NullPointerException [ERROR] TestRouterRpc.testErasureCoding:985 » Remote org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException [ERROR] TestRouterRpcMultiDestination>TestRouterRpc.testErasureCoding:985 » Remote org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException {code} The testErasureCoding comes from: {code} [ERROR] testErasureCoding(org.apache.hadoop.hdfs.server.federation.router.TestRouterRpc) Time elapsed: 0.754 s <<< ERROR! org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RemoteException): org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.NullPointerException at org.apache.hadoop.hdfs.server.namenode.FSEditLogOp$AddErasureCodingPolicyOp.toString(FSEditLogOp.java:4482) at java.lang.String.valueOf(String.java:2994) at java.lang.StringBuilder.append(StringBuilder.java:131) at org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync$RpcEdit.toString(FSEditLogAsync.java:319) at java.lang.String.valueOf(String.java:2994) at java.lang.StringBuilder.append(StringBuilder.java:131) at org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.logSync(FSEditLogAsync.java:128) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.addErasureCodingPolicies(FSNamesystem.java:7470) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addErasureCodingPolicies(NameNodeRpcServer.java:2418) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addErasureCodingPolicies(ClientNamenodeProtocolServerSideTranslatorPB.java:1773) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523) {code} This test runs fine in {{trunk}}. The one for the CLI: {code} [ERROR] testCreateInvalidEntry(org.apache.hadoop.hdfs.server.federation.router.TestRouterAdminCLI) Time elapsed: 0.008 s <<< ERROR! java.lang.NullPointerException at org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:529) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:568) at org.apache.hadoop.util.GenericOptionsParser.(GenericOptionsParser.java:174) at org.apache.hadoop.util.GenericOptionsParser.(GenericOptionsParser.java:156) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90) at org.apache.hadoop.hdfs.server.federation.router.TestRouterAdminCLI.testCreateInvalidEntry(TestRouterAdminCLI.java:444) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) {code} However, this also fails on trunk for me so we can leave this one. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403272#comment-16403272 ] Wei Yan commented on HDFS-13215: {quote}I think the error for {{dfs.namenode.fs-limits.min-block-size}} has to do with the fact that {{hdfs-site.xml}} has that value set to 0 which allows any value. {quote} I add a separate hdfs-site.xml to hadoop-hdfs-rbf, like what hadoop-hdfs-httfs does. Upload a new patch [^HDFS-13215.007.patch]. I move webapps to hadoop-hdfs-rbf, and verified it in a dev env, looks ok. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403209#comment-16403209 ] Íñigo Goiri commented on HDFS-13215: I think the error for {{dfs.namenode.fs-limits.min-block-size}} has to do with the fact that {{hdfs-site.xml}} has that value set to 0 which allows any value. We need to figure how YARN takes the yarn-site.xml for all the subprojects. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403169#comment-16403169 ] Íñigo Goiri commented on HDFS-13215: {quote} The three testcases passed in my local env. You miss some patches? {quote} I was testing trunk with [^HDFS-13215.006.patch] on top. {code} [ERROR] Errors: [ERROR] TestRouterAdminCLI.testCreateInvalidEntry:444 » NullPointerException (this is probably something with Windows) [ERROR] TestRouterQuota.testGetQuota:314 » Remote Specified block size is less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 512 < 1048576 [ERROR] TestRouterQuota.testQuotaUpdating:418 » Remote Specified block size is less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 512 < 1048576 [ERROR] TestRouterQuota.testStorageSpaceQuotaaExceed:181 » Remote Specified block size is less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 512 < 1048576 [ERROR] TestRouterRpc.testErasureCoding:985 » Remote org.apache.hadoop.ipc.RemoteException: NPE [ERROR] TestRouterRpc.testProxyGetAdditionalDatanode:854 » Remote Specified block size is less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 512 < 1048576 [ERROR] TestRouterRpcMultiDestination>TestRouterRpc.testErasureCoding:985 » Specified block size is less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 1024 < 1048576 [ERROR] TestRouterRpcMultiDestination>TestRouterRpc.testProxyGetAdditionalDatanode:854 » Specified block size is less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 1024 < 1048576 {code} So it looks like the main issue is this dfs.namenode.fs-limits.min-block-size. Maybe is taking the wrong config files? > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403153#comment-16403153 ] Wei Yan commented on HDFS-13215: The three testcases passed in my local env. You miss some patches? I'll move the webapps out. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403144#comment-16403144 ] Íñigo Goiri commented on HDFS-13215: We probably should move the webapps too. We need to check carefully what does the final dist do with it and if the references to the static HDFs js and css are still correct. There is also things like proto-web.xml. I'm also running the unit tests and I'm getting errors with: * TestRouterQuota * TestRouterRpc * TestRouterRpcMultiDestination > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403141#comment-16403141 ] Wei Yan commented on HDFS-13215: Good catch, [~elgoiri]. There is another dir: hadoop-hdfs/src/main/webapps/router, not sure whether we should move it out or just leave it there. I'm thinking, in the future, if we plan to separate hdfs components just like yarn, we could leave it there and seperate to another module (like hadoop-yarn-ui) which includes all HDFS UI related. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403129#comment-16403129 ] Íñigo Goiri commented on HDFS-13215: We also need to add the hdfs-rb-default.xml to the pom.xml to copy it at the end. Is there anything we need to do for the documentation? > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403123#comment-16403123 ] Íñigo Goiri commented on HDFS-13215: This is not very promising with Yetus. I'll run locally, meanwhile we are missing in findbugsExcludeFile.xml: We need to remove it from the old one too. You may want to do a grep for router or federation. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403111#comment-16403111 ] Wei Yan commented on HDFS-13215: Strange Yetus didn't run any HDFS testcases.. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403107#comment-16403107 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 34s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 77 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m 16s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 33s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 31s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 12m 26s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 12m 26s{color} | {color:red} root generated 161 new + unchanged - 161 fixed = 1272 total (was 1272) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 2m 50s{color} | {color:orange} root: The patch generated 3 new + 431 unchanged - 3 fixed = 434 total (was 434) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 0s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 11s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 8s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 30s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 6s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 6s{color} | {color:red} root in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 38s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}141m 53s{color} | {color:blac
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402584#comment-16402584 ] Wei Yan commented on HDFS-13215: Thanks, [~elgoiri]. I run all RBF tests locally and they're fine. Not sure whether Yetus is ok. Working on branch-2 patch, close to go. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402576#comment-16402576 ] Íñigo Goiri commented on HDFS-13215: Posted message in HDFS-12615 about freezing commits. Do you know if the RBF tests will run now? We need to generate patches for each of the branches: * branch-2 * branch-2.9 * branch-3.0 * branch-3.1 * trunk > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402518#comment-16402518 ] Wei Yan commented on HDFS-13215: Rebase a new patch [^HDFS-13215.006.patch]. [~elgoiri] [~linyiqun] let's freeze the RBF commits to get this in. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402438#comment-16402438 ] Wei Yan commented on HDFS-13215: let me do a quick rebase. and then we can freeze commit. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402432#comment-16402432 ] Íñigo Goiri commented on HDFS-13215: As we committed new stuff, [^HDFS-13215.005.patch] is missing it. We would have to redo the patch. [~ywskycn] let us know when you wanna freeze related commits to avoid this. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402374#comment-16402374 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 73 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 28s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 4s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 11m 12s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 20s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 5s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 16s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 29s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m 11s{color} | {color:red} hadoop-hdfs-project in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 5m 47s{color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m 58s{color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} cc {color} | {color:red} 1m 58s{color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 58s{color} | {color:red} root in the patch failed. {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 3m 9s{color} | {color:orange} root: The patch generated 3 new + 431 unchanged - 3 fixed = 434 total (was 434) {color} | | {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 51s{color} | {color:red} root in the patch failed. {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 0s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 8s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 7s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 2m 51s{color} | {color:red} patch has errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-hdfs-project . {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 30s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 47s{color} | {color:red} root in the patch failed. {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 53m 38s{color} | {color:red} root in the patch failed. {color} | | {color:green}+
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402182#comment-16402182 ] Wei Yan commented on HDFS-13215: Upload [^HDFS-13215.005.patch] to address review comments. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16400845#comment-16400845 ] Wei Yan commented on HDFS-13215: {quote}It looks like Yetus didn't run the unit tests for RBF {quote} Found this following message in [https://builds.apache.org/job/PreCommit-HDFS-Build/23499/artifact/out/patch-unit-root.txt.] {noformat} [INFO] [INFO] Skipping Apache Hadoop HDFS-RBF [INFO] This project has been banned from the build due to previous failures.{noformat} It looks like Yestus skipped some HDFS components due to the test failures in hadoop-hdfs itself. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16400751#comment-16400751 ] Íñigo Goiri commented on HDFS-13215: It looks like Yetus didn't run the unit tests for RBF: [here|https://builds.apache.org/job/PreCommit-HDFS-Build/23499/testReport/]. Not sure what's missing though. I would call it {{RBFConfigKeys}} instead of {{RbfConfigKeys}} (similar for TestRbfConfigFields). I would avoid making changes like the ones in FederationMetrics. We should do a separate JIRA for those. Can we do a separate JIRA for: * hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/protocolPB/package-info.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/FederationMetrics.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/federation/router/Router.java (javadoc comment) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterQuota.java (javadoc comment) Similarly, we should avoid the space changes in {{DFSConfigKeys}} (towards the end of the diff). Thanks for taking this [~ywskycn], this is a lot of work. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16400243#comment-16400243 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 31s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 73 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 5s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 12m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 8s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m 12s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 15m 18s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 15m 18s{color} | {color:red} root generated 162 new + 1123 unchanged - 162 fixed = 1285 total (was 1285) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 3m 24s{color} | {color:orange} root: The patch generated 3 new + 431 unchanged - 3 fixed = 434 total (was 434) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 12m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 1s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 6s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 6s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 8m 55s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-hdfs-project . {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 20s{color} | {color:red} hadoop-hdfs-rbf in the patch failed. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 17s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}161m 20s{color} | {color:red} root in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 53s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}299m 55s{color} |
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399958#comment-16399958 ] Wei Yan commented on HDFS-13215: Thanks, [~maobaolong], [~elgoiri]. The [^HDFS-13215.004.patch] should be good to review. I fixed the built-dist issue mentioned before, and also tried to run dfsrouter and dfsrouteradmin cmd in my local env. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399799#comment-16399799 ] Íñigo Goiri commented on HDFS-13215: At this point, I think the only patches close to completion are HDFS-13204 and HDFS-13224. HDFS-13248 is the other one with a patch available but there is no rush for this one as it will take me a few days to code. The rest are still in discussion. Anyway, we can do a freeze whenever we need. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399793#comment-16399793 ] Íñigo Goiri commented on HDFS-13215: {quote} Given that this Jira will new a module and make a big refactor, i suggest we should finish this Jira as soon as possible, so that the new Jira ticket will base on the new layout. BTW, Not sure if we should suspend a few day of commit for RBF, if not Wei Yan will have to update for getting the newer commit {quote} I agree. In addition, doing this for all the branches is even more convoluted. At this point, I think [~ywskycn] is still trying to figure out what's the best split. Once we have a clear approach, we should freeze RBF commits until done. [~ywskycn], how far you think you are from having a full approach? > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399780#comment-16399780 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 71 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 11m 54s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 8s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 11s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 32s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 49s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 26s{color} | {color:red} hadoop-hdfs-rbf in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 40s{color} | {color:red} hadoop-hdfs-project in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 5m 9s{color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 2m 22s{color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} cc {color} | {color:red} 2m 22s{color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 22s{color} | {color:red} root in the patch failed. {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 3m 12s{color} | {color:orange} root: The patch generated 3 new + 432 unchanged - 3 fixed = 435 total (was 435) {color} | | {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 6m 25s{color} | {color:red} root in the patch failed. {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 0s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 10s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 6s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 2m 57s{color} | {color:red} patch has errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-hdfs-project . {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 33s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 22s{color} | {color:red} hadoop-hdfs-rbf in the patch failed. {color} | | {color:green}+1{color} | {color:gr
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399764#comment-16399764 ] maobaolong commented on HDFS-13215: --- Given that this Jira will new a module and make a big refactor, i suggest we should finish this Jira as soon as possible, so that the new Jira ticket will base on the new layout. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399761#comment-16399761 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 98 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 13s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 13m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 59s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m 29s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m 0s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 34s{color} | {color:red} hadoop-hdfs-rbf in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 52s{color} | {color:red} hadoop-hdfs-project in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 37s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 5m 47s{color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 2m 22s{color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} cc {color} | {color:red} 2m 22s{color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 22s{color} | {color:red} root in the patch failed. {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 3m 41s{color} | {color:orange} root: The patch generated 17 new + 647 unchanged - 14 fixed = 664 total (was 661) {color} | | {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 7m 27s{color} | {color:red} root in the patch failed. {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 1s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 9s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s{color} | {color:red} The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 7s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 2m 55s{color} | {color:red} patch has errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-hdfs-project . {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 38
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398968#comment-16398968 ] genericqa commented on HDFS-13215: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 28s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 71 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 26s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 8s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 13m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 55s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 14s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m 13s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 19m 25s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 19m 25s{color} | {color:red} root generated 160 new + 1125 unchanged - 160 fixed = 1285 total (was 1285) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 3m 40s{color} | {color:orange} root: The patch generated 7 new + 431 unchanged - 3 fixed = 438 total (was 434) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 13m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 2s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 10s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 9s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 8s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project . {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 28s{color} | {color:red} hadoop-hdfs-rbf in the patch failed. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m 30s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}167m 28s{color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 1m 31s{color} | {color:red} The patch generated 1 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}323
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397710#comment-16397710 ] Wei Yan commented on HDFS-13215: {quote}I'm leaning towards having our own hdfs-rbf-site.xml. {quote} I'm ok with this approach. Will put a new diff to move RBF-related configs to a separte hdfs-rbf-default.xml. After we finish this jira, we can start revise hdfs-site.xml and scripts/docs to separate rbf-related configs to hdfs-rbf-site.xml. Sounds good? {quote}We could pretty much just depend on hadoop-hdfs-client. {quote} This may be not easy, given that we tried to reuse as much code as possible from NameNode side... > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397405#comment-16397405 ] Íñigo Goiri commented on HDFS-13215: I'm leaning towards having our own hdfs-rbf-site.xml. We could pretty much just depend on hadoop-hdfs-client. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397396#comment-16397396 ] Wei Yan commented on HDFS-13215: That would need to rebuild the story of router configurations, like we may need to use hdfs-rbf-site.xml, instead of hdfs-site.xml one to avoid confusion. Another option would be: using String to represent class like "DFS_ROUTER_METRICS_CLASS_DEFAULT = FederationRPCPerformanceMonitor.class", as I mentioned above. But I only saw two such cases in current DFSConfigKeys.java, and it requests careful maintain if code refactored... I don't have other idea here.. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397371#comment-16397371 ] Íñigo Goiri commented on HDFS-13215: I guess the only way for us to be completely separate is to have our own {{hdfs-rbf-default.xml}}. That would also tackle my concerns about polluting {{hdfs-default.xml}}. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397323#comment-16397323 ] Wei Yan commented on HDFS-13215: {quote}Can we do our own extension of TestHdfsConfigFields here? {quote} But the old testcase TestHdfsConfigFields will still fail, right? As it still compares *ConfigKeys.java with hdfs-default.xml. If we don't have RouterConfigKeys.java there, the test will fail. But adding RouterConfigKeys.java there would make hadoop-hdfs depends on hadoop-hdfs-rbf, and it will generate cyclic reference between hadoop-hdfs and hadoop-hdfs-rbf modules. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393717#comment-16393717 ] Íñigo Goiri commented on HDFS-13215: Can we do our own extension of TestHdfsConfigFields here? > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393697#comment-16393697 ] Wei Yan commented on HDFS-13215: {noformat} Looks like the change for impl class broken the unit tests. {noformat} Yes, I saw the discussion in HDFS-13099. I move these configs separately as original DFSConfigKeys needs to improve RBF classes. If not splitting out, the hadoop-hdfs component needs a dependency on hadoop-hdfs-rbf. But currently the TestHdfsConfigFields test is broken One possibility is, we change the following config to a string directly "org.apache.hadoop.hdfs.server.federation.metrics.FederationRPCPerformanceMonitor". {code:java} public static final Class DFS_ROUTER_METRICS_CLASS_DEFAULT = FederationRPCPerformanceMonitor.class; {code} {noformat} For backporting we will have to do a couple branch patches as some features are not in 2.9 for example. {noformat} Yes, will also post a branch-2 patch once this one cleared. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393541#comment-16393541 ] Íñigo Goiri commented on HDFS-13215: Thanks for taking this [~ywskycn]. The move looks good. At some point we had some talk with [~linyiqun] about moving the keys to a separate file; I personally like it but there was some drawbacks (I'll try to link the discussion). One of the issues I cna think is the tests for {{hdfs-default.xml}}, do those still apply? For backporting we will have to do a couple branch patches as some features are not in 2.9 for example. Regarding the shaded components, I don't have much experience. Anybody is familiar with this? > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393472#comment-16393472 ] Wei Yan commented on HDFS-13215: Put a patch [^HDFS-13215.000.patch] to separate rbf to a separate module. I also move RBF-related configs to a separate file. Site-related and hdfs cmd related still sit inside hadoop-hdfs module. Having a build problem with "dist", not sure how to make rbf jar included in share/hadoop/hdfs. Tried out different approaches from other components, but none worked there... > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386387#comment-16386387 ] Wei Yan commented on HDFS-13215: [~elgoiri] let me take this one. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Priority: Major > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386372#comment-16386372 ] Íñigo Goiri commented on HDFS-13215: It looks like people is mostly positive about this change. I'd go ahead but I don't have much cycles this week so I'd appreciate if anybody can do the refactor. Anybody willing to take this? > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Priority: Major > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16385870#comment-16385870 ] maobaolong commented on HDFS-13215: --- Any update here? expecting. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Priority: Major > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383206#comment-16383206 ] maobaolong commented on HDFS-13215: --- +1, good idea. 👍 > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Priority: Major > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382888#comment-16382888 ] Chris Douglas commented on HDFS-13215: -- Breaking server-side code into multiple modules is less useful. IIRC, RBF is mostly in its own package, and since we upgraded surefire it's easier to run subsets of tests using its regex syntax. But sure, it's mostly stand-alone, and might as well live in its own package. If tools like the rebalancer take a dependency on this package to customize their behavior in federated environments, those may have to move into a separate package to avoid the circular dependency. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Priority: Major > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382416#comment-16382416 ] Íñigo Goiri commented on HDFS-13215: YARN itself has done a pretty good job at partitioning the project and making the code base more manageable. Recently, in HDFS-12512 we had some issues running tests too; not sure if this would help there. [~chris.douglas], [~aw], do you guys think this is useful at all? Good to have? I should've done this from the start in HDFS-10467 but never thought about it... > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Priority: Major > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382413#comment-16382413 ] Wei Yan commented on HDFS-13215: +1, so we unit tests will be much easier... > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Priority: Major > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org