[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[
https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070385#comment-17070385
]
Hadoop QA commented on HDFS-14385:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 0s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 32s{color}
| {color:red} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.federation.router.TestRouterFaultTolerant |
| | hadoop.hdfs.server.federation.router.TestRouterMultiRack |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.8 Server=19.03.8 Image:yetus/hadoop:4454c6d14b7 |
| JIRA Issue | HDFS-14385 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12998145/HDFS-14385.004.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 79df1a1f67cd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 696a663 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/29056/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/29056/testReport/ |
| Max. process+thread count | 3197 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070358#comment-17070358 ] Xiaoqiao He commented on HDFS-14385: v004 try to fix checkstyle and failed unit tests. > RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster > > > Key: HDFS-14385 > URL: https://issues.apache.org/jira/browse/HDFS-14385 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Attachments: HDFS-14385-HDFS-13891.001.patch, HDFS-14385.002.patch, > HDFS-14385.003.patch, HDFS-14385.004.patch > > > MiniRouterDFSCluster mimic federated HDFS cluster with routers to support RBF > test, In MiniRouterDFSCluster, it starts MiniDFSCluster with complete roles > of HDFS which have significant time cost. As HDFS-14351 discussed, it is > better to provide mock MiniDFSCluster/Namenodes as one option to support some > test case and reduce time cost. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[
https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070325#comment-17070325
]
Hadoop QA commented on HDFS-14385:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
50s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 8s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 15s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs-rbf: The patch
generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 39s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 45s{color}
| {color:red} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 68m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.fs.contract.router.TestRouterHDFSContractRootDirectorySecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractAppendSecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractDelegationToken |
| | hadoop.fs.contract.router.TestRouterHDFSContractSeekSecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractDeleteSecure |
| | hadoop.hdfs.server.federation.router.TestRouterWithSecureStartup |
| | hadoop.fs.contract.router.TestRouterHDFSContractGetFileStatusSecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractRenameSecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractOpenSecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractConcatSecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractMkdirSecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractCreateSecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractSetTimesSecure |
| | hadoop.hdfs.server.federation.router.TestRouterMultiRack |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.8 Server=19.03.8 Image:yetus/hadoop:4454c6d14b7 |
| JIRA Issue | HDFS-14385 |
| JIRA Patch URL |
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070288#comment-17070288 ] Xiaoqiao He commented on HDFS-14385: Thanks [~elgoiri], v003 try to fix checkstyle and bugs. Pending Jenkins. > RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster > > > Key: HDFS-14385 > URL: https://issues.apache.org/jira/browse/HDFS-14385 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Attachments: HDFS-14385-HDFS-13891.001.patch, HDFS-14385.002.patch, > HDFS-14385.003.patch > > > MiniRouterDFSCluster mimic federated HDFS cluster with routers to support RBF > test, In MiniRouterDFSCluster, it starts MiniDFSCluster with complete roles > of HDFS which have significant time cost. As HDFS-14351 discussed, it is > better to provide mock MiniDFSCluster/Namenodes as one option to support some > test case and reduce time cost. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069928#comment-17069928 ] Íñigo Goiri commented on HDFS-14385: In addition to the checkstyles, this breaks something in a lot of tests. Can you take a look? > RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster > > > Key: HDFS-14385 > URL: https://issues.apache.org/jira/browse/HDFS-14385 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Attachments: HDFS-14385-HDFS-13891.001.patch, HDFS-14385.002.patch > > > MiniRouterDFSCluster mimic federated HDFS cluster with routers to support RBF > test, In MiniRouterDFSCluster, it starts MiniDFSCluster with complete roles > of HDFS which have significant time cost. As HDFS-14351 discussed, it is > better to provide mock MiniDFSCluster/Namenodes as one option to support some > test case and reduce time cost. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[
https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069356#comment-17069356
]
Hadoop QA commented on HDFS-14385:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
53s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 17s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs-rbf: The patch
generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 42s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 28s{color}
| {color:red} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 1m
5s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 88m 52s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.fs.contract.router.TestRouterHDFSContractGetFileStatus |
| | hadoop.fs.contract.router.TestRouterHDFSContractOpen |
| | hadoop.hdfs.server.federation.router.TestRouterQuota |
| | hadoop.fs.contract.router.TestRouterHDFSContractRootDirectorySecure |
| | hadoop.hdfs.server.federation.router.TestRouterMountTable |
| | hadoop.fs.contract.router.TestRouterHDFSContractCreate |
| | hadoop.fs.contract.router.TestRouterHDFSContractSetTimes |
| | hadoop.fs.contract.router.TestRouterHDFSContractRename |
| | hadoop.fs.contract.router.TestRouterHDFSContractSeek |
| | hadoop.fs.contract.router.web.TestRouterWebHDFSContractRootDirectory |
| | hadoop.fs.contract.router.TestRouterHDFSContractConcat |
| | hadoop.hdfs.server.federation.router.TestRouterFsck |
| | hadoop.hdfs.server.federation.router.TestRouterRpc |
| | hadoop.fs.contract.router.TestRouterHDFSContractAppendSecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractDelegationToken |
| | hadoop.fs.contract.router.TestRouterHDFSContractSeekSecure |
| | hadoop.fs.contract.router.web.TestRouterWebHDFSContractMkdir |
| | hadoop.fs.contract.router.TestRouterHDFSContractDeleteSecure |
| |
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069301#comment-17069301 ] Xiaoqiao He commented on HDFS-14385: Thanks [~elgoiri] pick this JIRA up, v002 rebase to trunk. PTAL. Thanks. > RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster > > > Key: HDFS-14385 > URL: https://issues.apache.org/jira/browse/HDFS-14385 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Attachments: HDFS-14385-HDFS-13891.001.patch, HDFS-14385.002.patch > > > MiniRouterDFSCluster mimic federated HDFS cluster with routers to support RBF > test, In MiniRouterDFSCluster, it starts MiniDFSCluster with complete roles > of HDFS which have significant time cost. As HDFS-14351 discussed, it is > better to provide mock MiniDFSCluster/Namenodes as one option to support some > test case and reduce time cost. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066288#comment-17066288 ] Íñigo Goiri commented on HDFS-14385: [~hexiaoqiao], do you mind rebasing to trunk and posting a patch for that? HDFS-13891 has already been merged. > RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster > > > Key: HDFS-14385 > URL: https://issues.apache.org/jira/browse/HDFS-14385 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Attachments: HDFS-14385-HDFS-13891.001.patch > > > MiniRouterDFSCluster mimic federated HDFS cluster with routers to support RBF > test, In MiniRouterDFSCluster, it starts MiniDFSCluster with complete roles > of HDFS which have significant time cost. As HDFS-14351 discussed, it is > better to provide mock MiniDFSCluster/Namenodes as one option to support some > test case and reduce time cost. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[
https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066252#comment-17066252
]
Hadoop QA commented on HDFS-14385:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 7m
52s{color} | {color:red} Docker failed to build yetus/hadoop:bdbca0e53b4.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDFS-14385 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12964438/HDFS-14385-HDFS-13891.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/29015/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
>
>
> Key: HDFS-14385
> URL: https://issues.apache.org/jira/browse/HDFS-14385
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: Xiaoqiao He
>Assignee: Xiaoqiao He
>Priority: Major
> Attachments: HDFS-14385-HDFS-13891.001.patch
>
>
> MiniRouterDFSCluster mimic federated HDFS cluster with routers to support RBF
> test, In MiniRouterDFSCluster, it starts MiniDFSCluster with complete roles
> of HDFS which have significant time cost. As HDFS-14351 discussed, it is
> better to provide mock MiniDFSCluster/Namenodes as one option to support some
> test case and reduce time cost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066242#comment-17066242 ] Wei-Chiu Chuang commented on HDFS-14385: Triggering a rebuild: https://builds.apache.org/job/PreCommit-HDFS-Build/29014/ > RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster > > > Key: HDFS-14385 > URL: https://issues.apache.org/jira/browse/HDFS-14385 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Attachments: HDFS-14385-HDFS-13891.001.patch > > > MiniRouterDFSCluster mimic federated HDFS cluster with routers to support RBF > test, In MiniRouterDFSCluster, it starts MiniDFSCluster with complete roles > of HDFS which have significant time cost. As HDFS-14351 discussed, it is > better to provide mock MiniDFSCluster/Namenodes as one option to support some > test case and reduce time cost. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807960#comment-16807960 ] Íñigo Goiri commented on HDFS-14385: There seems to be an issue with HAServiceState. We should also make some test use the mock now. > RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster > > > Key: HDFS-14385 > URL: https://issues.apache.org/jira/browse/HDFS-14385 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf >Reporter: He Xiaoqiao >Assignee: He Xiaoqiao >Priority: Major > Attachments: HDFS-14385-HDFS-13891.001.patch > > > MiniRouterDFSCluster mimic federated HDFS cluster with routers to support RBF > test, In MiniRouterDFSCluster, it starts MiniDFSCluster with complete roles > of HDFS which have significant time cost. As HDFS-14351 discussed, it is > better to provide mock MiniDFSCluster/Namenodes as one option to support some > test case and reduce time cost. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[
https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806779#comment-16806779
]
Hadoop QA commented on HDFS-14385:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-13891 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
12s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} HDFS-13891 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} HDFS-13891 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 16s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs-rbf: The patch
generated 13 new + 0 unchanged - 0 fixed = 13 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 34s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 36s{color}
| {color:red} hadoop-hdfs-rbf in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.fs.contract.router.TestRouterHDFSContractSetTimesSecure |
| | hadoop.hdfs.server.federation.router.TestRouterRpc |
| | hadoop.fs.contract.router.TestRouterHDFSContractSeekSecure |
| | hadoop.fs.contract.router.web.TestRouterWebHDFSContractDelete |
| | hadoop.fs.contract.router.web.TestRouterWebHDFSContractConcat |
| | hadoop.hdfs.server.federation.router.TestRouterMountTable |
| | hadoop.fs.contract.router.TestRouterHDFSContractDelete |
| | hadoop.fs.contract.router.TestRouterHDFSContractSetTimes |
| | hadoop.fs.contract.router.TestRouterHDFSContractRootDirectorySecure |
| | hadoop.fs.contract.router.TestRouterHDFSContractConcatSecure |
| | hadoop.fs.contract.router.web.TestRouterWebHDFSContractRename |
| | hadoop.fs.contract.router.web.TestRouterWebHDFSContractCreate |
| | hadoop.fs.contract.router.TestRouterHDFSContractSeek |
| |
hadoop.hdfs.server.federation.router.TestRouterRPCMultipleDestinationMountTableResolver
|
| | hadoop.hdfs.server.federation.router.TestRouterQuota |
| | hadoop.fs.contract.router.TestRouterHDFSContractDelegationToken |
| |
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800718#comment-16800718 ] He Xiaoqiao commented on HDFS-14385: Thanks [~elgoiri] for more comments, I will work on this jira the next days. > RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster > > > Key: HDFS-14385 > URL: https://issues.apache.org/jira/browse/HDFS-14385 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf >Reporter: He Xiaoqiao >Assignee: He Xiaoqiao >Priority: Major > > MiniRouterDFSCluster mimic federated HDFS cluster with routers to support RBF > test, In MiniRouterDFSCluster, it starts MiniDFSCluster with complete roles > of HDFS which have significant time cost. As HDFS-14351 discussed, it is > better to provide mock MiniDFSCluster/Namenodes as one option to support some > test case and reduce time cost. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14385) RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
[
https://issues.apache.org/jira/browse/HDFS-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797333#comment-16797333
]
Íñigo Goiri commented on HDFS-14385:
>From some of the reports, the targets could potentially be:
* TestRouterQuota takes 146 seconds for 11 tests (~13 per test) but we may need
DNs for this one. Maybe we can reduce some of the times.
* TestRouterClientRejectOverload takes 63 seconds for 4 tests (~16 per test);
this should be very doable.
* TestRouterAllResolver takes 44 seconds for 3 tests (~15 per test) but it
needs a lot of name space stuff, could we mocked though.
* TestRouterRPCClientRetries takes 32 seconds for 3 tests (~10 per tests); this
should be doable.
Another thing here would be to set the Router by default to use ports {{:0}} to
allow running tests in parallel.
> RBF: Optimize MiniRouterDFSCluster with optional light weight MiniDFSCluster
>
>
> Key: HDFS-14385
> URL: https://issues.apache.org/jira/browse/HDFS-14385
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
>
> MiniRouterDFSCluster mimic federated HDFS cluster with routers to support RBF
> test, In MiniRouterDFSCluster, it starts MiniDFSCluster with complete roles
> of HDFS which have significant time cost. As HDFS-14351 discussed, it is
> better to provide mock MiniDFSCluster/Namenodes as one option to support some
> test case and reduce time cost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
