[jira] [Commented] (HDFS-13399) Make Client field AlignmentContext non-static.
[
https://issues.apache.org/jira/browse/HDFS-13399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453528#comment-16453528
]
genericqa commented on HDFS-13399:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-12943 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 6m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m
17s{color} | {color:green} HDFS-12943 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 32m
46s{color} | {color:green} HDFS-12943 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
32s{color} | {color:green} HDFS-12943 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
36s{color} | {color:green} HDFS-12943 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 29s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
45s{color} | {color:green} HDFS-12943 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
26s{color} | {color:green} HDFS-12943 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 26m
36s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 14s{color} | {color:orange} root: The patch generated 3 new + 543 unchanged
- 0 fixed = 546 total (was 543) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 6s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 41s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
35s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 85m 53s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
56s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}246m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.shell.TestCopyPreserveFlag |
| | hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean |
| | hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
| JIRA Issue | HDFS-13399 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920662/HDFS-13399-HDFS-12943.005.patch
|
| Optional Tests | asflicense compile javac
[jira] [Commented] (HDFS-13245) RBF: State store DBMS implementation
[ https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453511#comment-16453511 ] Yiran Wu commented on HDFS-13245: - Ok, I will add it . :D cc: [~linyiqun] > RBF: State store DBMS implementation > > > Key: HDFS-13245 > URL: https://issues.apache.org/jira/browse/HDFS-13245 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: maobaolong >Assignee: Yiran Wu >Priority: Major > Attachments: HDFS-13245.001.patch, HDFS-13245.002.patch, > HDFS-13245.003.patch > > > Add a DBMS implementation for the State Store. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13489) Get base snapshotable path if exists for a given path
[
https://issues.apache.org/jira/browse/HDFS-13489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453477#comment-16453477
]
genericqa commented on HDFS-13489:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 53s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 14m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
49s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 51s{color} | {color:orange} hadoop-hdfs-project: The patch generated 53 new
+ 402 unchanged - 0 fixed = 455 total (was 402) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 51 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 6s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
24s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 75m 28s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
18s{color} | {color:green} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}182m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:b78c94f |
| JIRA Issue | HDFS-13489 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920679/HDFS-13489.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle cc |
| uname | Linux 27bc757509d7
[jira] [Commented] (HDFS-13475) RBF: Admin cannot enforce Router enter SafeMode
[
https://issues.apache.org/jira/browse/HDFS-13475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453468#comment-16453468
]
Wei Yan commented on HDFS-13475:
Thanks [~dibyendu_hadoop] . I think [~csun] is already working on it..
> RBF: Admin cannot enforce Router enter SafeMode
> ---
>
> Key: HDFS-13475
> URL: https://issues.apache.org/jira/browse/HDFS-13475
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Wei Yan
>Priority: Major
>
> To reproduce the issue:
> {code:java}
> $ bin/hdfs dfsrouteradmin -safemode enter
> Successfully enter safe mode.
> $ bin/hdfs dfsrouteradmin -safemode get
> Safe Mode: true{code}
> And then,
> {code:java}
> $ bin/hdfs dfsrouteradmin -safemode get
> Safe Mode: false{code}
> From the code, it looks like the periodicInvoke triggers the leave.
> {code:java}
> public void periodicInvoke() {
> ..
> // Always update to indicate our cache was updated
> if (isCacheStale) {
> if (!rpcServer.isInSafeMode()) {
> enter();
> }
> } else if (rpcServer.isInSafeMode()) {
> // Cache recently updated, leave safe mode
> leave();
> }
> }
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13475) RBF: Admin cannot enforce Router enter SafeMode
[
https://issues.apache.org/jira/browse/HDFS-13475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei Yan reassigned HDFS-13475:
--
Assignee: Chao Sun (was: Wei Yan)
> RBF: Admin cannot enforce Router enter SafeMode
> ---
>
> Key: HDFS-13475
> URL: https://issues.apache.org/jira/browse/HDFS-13475
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Chao Sun
>Priority: Major
>
> To reproduce the issue:
> {code:java}
> $ bin/hdfs dfsrouteradmin -safemode enter
> Successfully enter safe mode.
> $ bin/hdfs dfsrouteradmin -safemode get
> Safe Mode: true{code}
> And then,
> {code:java}
> $ bin/hdfs dfsrouteradmin -safemode get
> Safe Mode: false{code}
> From the code, it looks like the periodicInvoke triggers the leave.
> {code:java}
> public void periodicInvoke() {
> ..
> // Always update to indicate our cache was updated
> if (isCacheStale) {
> if (!rpcServer.isInSafeMode()) {
> enter();
> }
> } else if (rpcServer.isInSafeMode()) {
> // Cache recently updated, leave safe mode
> leave();
> }
> }
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13245) RBF: State store DBMS implementation
[
https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453436#comment-16453436
]
genericqa commented on HDFS-13245:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 14s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-assemblies {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 24m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 24m
56s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 33s{color} | {color:orange} root: The patch generated 21 new + 0 unchanged -
0 fixed = 21 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 48s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-assemblies {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
18s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 37s{color}
| {color:red} hadoop-hdfs-rbf in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}121m 21s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.fs.contract.router.web.TestRouterWebHDFSContractAppend |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:b78c94f |
| JIRA Issue | HDFS-13245 |
| JIRA Patch URL |
[jira] [Commented] (HDFS-13491) [SPS]: Discuss and implement efficient approach to send a copy of a block to another datanode
[ https://issues.apache.org/jira/browse/HDFS-13491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453417#comment-16453417 ] Rakesh R commented on HDFS-13491: - Thanks [~xkrogen], I've updated Jira description with reference link. Please check it. > [SPS]: Discuss and implement efficient approach to send a copy of a block to > another datanode > - > > Key: HDFS-13491 > URL: https://issues.apache.org/jira/browse/HDFS-13491 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Major > > This Jira task is to reach consensus about the block transfer logic to > another data node and implement the same to satisfy block storage policy. > Reference [discussion > thread|https://issues.apache.org/jira/browse/HDFS-13165?focusedCommentId=16441057=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16441057] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13491) [SPS]: Discuss and implement efficient approach to send a copy of a block to another datanode
[ https://issues.apache.org/jira/browse/HDFS-13491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-13491: Description: This Jira task is to reach consensus about the block transfer logic to another data node and implement the same to satisfy block storage policy. Reference [discussion thread|https://issues.apache.org/jira/browse/HDFS-13165?focusedCommentId=16441057=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16441057] was: This Jira task is to reach consensus about the block transfer logic to another data node and implement the same to satisfy block storage policy. Reference discussion thread > [SPS]: Discuss and implement efficient approach to send a copy of a block to > another datanode > - > > Key: HDFS-13491 > URL: https://issues.apache.org/jira/browse/HDFS-13491 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Major > > This Jira task is to reach consensus about the block transfer logic to > another data node and implement the same to satisfy block storage policy. > Reference [discussion > thread|https://issues.apache.org/jira/browse/HDFS-13165?focusedCommentId=16441057=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16441057] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13245) RBF: State store DBMS implementation
[ https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453392#comment-16453392 ] Yiqun Lin commented on HDFS-13245: -- hI [~yiran], haven't taken a deep look, but one comment: It will be good for review to add more javadoc and comments for the private methods/variables defined in the StateStoreSQLImpl. > RBF: State store DBMS implementation > > > Key: HDFS-13245 > URL: https://issues.apache.org/jira/browse/HDFS-13245 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: maobaolong >Assignee: Yiran Wu >Priority: Major > Attachments: HDFS-13245.001.patch, HDFS-13245.002.patch, > HDFS-13245.003.patch > > > Add a DBMS implementation for the State Store. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13432) Ozone: When datanodes register, send NodeReport and ContainerReport
[
https://issues.apache.org/jira/browse/HDFS-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453364#comment-16453364
]
genericqa commented on HDFS-13432:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
17s{color} | {color:red} container-service in trunk failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
16s{color} | {color:red} server-scm in trunk failed. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 19s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
15s{color} | {color:red} container-service in trunk failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
15s{color} | {color:red} server-scm in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
18s{color} | {color:red} container-service in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
19s{color} | {color:red} server-scm in trunk failed. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
11s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
11s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
11s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
10s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 19s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
14s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
14s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
13s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
16s{color} | {color:red} server-scm in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 13s{color}
| {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 13s{color}
| {color:red} server-scm in the patch failed. {color} |
[jira] [Commented] (HDFS-13245) RBF: State store DBMS implementation
[ https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453353#comment-16453353 ] Yiran Wu commented on HDFS-13245: - Thanks to [~elgoiri] and [~maobaolong] suggestions。 I am writing this unit test. I first submitted a 003.patch to modify the directory. I hope everyone can review the directory and the naming. > RBF: State store DBMS implementation > > > Key: HDFS-13245 > URL: https://issues.apache.org/jira/browse/HDFS-13245 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: maobaolong >Assignee: Yiran Wu >Priority: Major > Attachments: HDFS-13245.001.patch, HDFS-13245.002.patch, > HDFS-13245.003.patch > > > Add a DBMS implementation for the State Store. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13503) Fix TestFsck test failures on Windows
[
https://issues.apache.org/jira/browse/HDFS-13503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453349#comment-16453349
]
Xiao Liang commented on HDFS-13503:
---
In my local machine, current status on Windows:
{color:#d04437}[INFO]
---{color}
{color:#d04437}[INFO] T E S T S{color}
{color:#d04437}[INFO]
---{color}
{color:#d04437}[INFO] Running
org.apache.hadoop.hdfs.server.namenode.TestFsck{color}
{color:#d04437}[ERROR] Tests run: 28, Failures: 0, Errors: 1, Skipped: 0, Time
elapsed: 190.863 s <<< FAILURE! - in
org.apache.hadoop.hdfs.server.namenode.TestFsck{color}
{color:#d04437}[ERROR]
testFsckUpgradeDomain(org.apache.hadoop.hdfs.server.namenode.TestFsck) Time
elapsed: 2.283 s <<< ERROR!{color}
{color:#d04437}java.io.IOException: Could not fully delete
D:\Git\Hadoop\hadoop-hdfs-project\hadoop-hdfs\target\test\data\dfs\name1{color}
{color:#d04437} at
org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:1043){color}
{color:#d04437} at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:879){color}
{color:#d04437} at
org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:513){color}
{color:#d04437} at
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:472){color}
{color:#d04437} at
org.apache.hadoop.hdfs.server.namenode.TestFsck.testUpgradeDomain(TestFsck.java:2072){color}
{color:#d04437} at
org.apache.hadoop.hdfs.server.namenode.TestFsck.testFsckUpgradeDomain(TestFsck.java:2047){color}
{color:#d04437} at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method){color}
{color:#d04437} at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62){color}
{color:#d04437} at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){color}
{color:#d04437} at java.lang.reflect.Method.invoke(Method.java:498){color}
{color:#d04437} at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47){color}
{color:#d04437} at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12){color}
{color:#d04437} at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44){color}
{color:#d04437} at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17){color}
{color:#d04437} at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74){color}
{color:#d04437}[INFO]{color}
{color:#d04437}[INFO] Results:{color}
{color:#d04437}[INFO]{color}
{color:#d04437}[ERROR] Errors:{color}
{color:#d04437}[ERROR]
TestFsck.testFsckUpgradeDomain:2047->testUpgradeDomain:2072 » IO Could not
ful...{color}
Wtih the patch:
{color:#14892c}[INFO]
---{color}
{color:#14892c}[INFO] T E S T S{color}
{color:#14892c}[INFO]
---{color}
{color:#14892c}[INFO] Running
org.apache.hadoop.hdfs.server.namenode.TestFsck{color}
{color:#14892c}[INFO] Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time
elapsed: 183.202 s - in org.apache.hadoop.hdfs.server.namenode.TestFsck{color}
{color:#14892c}[INFO]{color}
{color:#14892c}[INFO] Results:{color}
{color:#14892c}[INFO]{color}
{color:#14892c}[INFO] Tests run: 28, Failures: 0, Errors: 0, Skipped: 0{color}
{color:#14892c}[INFO]{color}
{color:#14892c}[INFO]
{color}
{color:#14892c}[INFO] BUILD SUCCESS{color}
{color:#14892c}[INFO]
{color}
{color:#14892c}[INFO] Total time: 03:15 min{color}
{color:#14892c}[INFO] Finished at: 2018-04-25T18:15:37-07:00{color}
{color:#14892c}[INFO] Final Memory: 26M/634M{color}
{color:#14892c}[INFO]
{color}
> Fix TestFsck test failures on Windows
> -
>
> Key: HDFS-13503
> URL: https://issues.apache.org/jira/browse/HDFS-13503
> Project: Hadoop HDFS
> Issue Type: Test
> Components: hdfs
>Reporter: Xiao Liang
>Assignee: Xiao Liang
>Priority: Major
> Labels: windows
> Attachments: HDFS-13503-branch-2.000.patch
>
>
> Test failures on Windows caused by the same reason as HDFS-13336, similar fix
> needed for TestFsck basing on HDFS-13408.
> MiniDFSCluster also needs a small fix for the getStorageDir() interface,
> which should use determineDfsBaseDir() to get the correct path of the data
> directory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail:
[jira] [Commented] (HDFS-13495) RBF: Support Router Admin REST API
[ https://issues.apache.org/jira/browse/HDFS-13495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453327#comment-16453327 ] Íñigo Goiri commented on HDFS-13495: I guess is fine to support admin commands in the Routers through REST then. [~arshad.mohammad] go ahead. > RBF: Support Router Admin REST API > -- > > Key: HDFS-13495 > URL: https://issues.apache.org/jira/browse/HDFS-13495 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Mohammad Arshad >Priority: Major > Labels: RBF > > This JIRA intends to add REST API support for all admin commands. Router > Admin REST APIs can be useful in managing the Routers from a central > management layer tool. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13495) RBF: Support Router Admin REST API
[ https://issues.apache.org/jira/browse/HDFS-13495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453306#comment-16453306 ] Larry McCay commented on HDFS-13495: [~chris.douglas] - I don't know of any particular reason that admin REST APIs should be avoided. It is possible that many admin like things require a restart which would be awkward/impossible to fully do via REST APIs? > RBF: Support Router Admin REST API > -- > > Key: HDFS-13495 > URL: https://issues.apache.org/jira/browse/HDFS-13495 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Mohammad Arshad >Priority: Major > Labels: RBF > > This JIRA intends to add REST API support for all admin commands. Router > Admin REST APIs can be useful in managing the Routers from a central > management layer tool. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13505) Turn on HDFS ACLs by default.
[ https://issues.apache.org/jira/browse/HDFS-13505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453294#comment-16453294 ] Wei-Chiu Chuang commented on HDFS-13505: Isn't this an incompatible change? > Turn on HDFS ACLs by default. > - > > Key: HDFS-13505 > URL: https://issues.apache.org/jira/browse/HDFS-13505 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > Attachments: HDFS-13505.00.patch > > > Turn on HDFS ACLs by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13495) RBF: Support Router Admin REST API
[ https://issues.apache.org/jira/browse/HDFS-13495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453285#comment-16453285 ] Chris Douglas commented on HDFS-13495: -- bq. do you have any insight on why there is no REST API for admin? Not sure. Often REST APIs are exposed through a gateway, like Knox. I don't know if these expose admin operations. IIRC there's also a servlet filter that rejects impersonation of admin/system accounts. [~lmc...@apache.org]? > RBF: Support Router Admin REST API > -- > > Key: HDFS-13495 > URL: https://issues.apache.org/jira/browse/HDFS-13495 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Mohammad Arshad >Priority: Major > Labels: RBF > > This JIRA intends to add REST API support for all admin commands. Router > Admin REST APIs can be useful in managing the Routers from a central > management layer tool. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13505) Turn on HDFS ACLs by default.
[ https://issues.apache.org/jira/browse/HDFS-13505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13505: -- Attachment: (was: HDFS-13505.00.patch) > Turn on HDFS ACLs by default. > - > > Key: HDFS-13505 > URL: https://issues.apache.org/jira/browse/HDFS-13505 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > Attachments: HDFS-13505.00.patch > > > Turn on HDFS ACLs by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13505) Turn on HDFS ACLs by default.
[ https://issues.apache.org/jira/browse/HDFS-13505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13505: -- Attachment: HDFS-13505.00.patch > Turn on HDFS ACLs by default. > - > > Key: HDFS-13505 > URL: https://issues.apache.org/jira/browse/HDFS-13505 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > Attachments: HDFS-13505.00.patch > > > Turn on HDFS ACLs by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13505) Turn on HDFS ACLs by default.
[ https://issues.apache.org/jira/browse/HDFS-13505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13505: -- Attachment: HDFS-13505.00.patch > Turn on HDFS ACLs by default. > - > > Key: HDFS-13505 > URL: https://issues.apache.org/jira/browse/HDFS-13505 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > > Turn on HDFS ACLs by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13489) Get base snapshotable path if exists for a given path
[
https://issues.apache.org/jira/browse/HDFS-13489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453284#comment-16453284
]
genericqa commented on HDFS-13489:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 59s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 16m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
40s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 12s{color} | {color:orange} hadoop-hdfs-project: The patch generated 53 new
+ 403 unchanged - 0 fixed = 456 total (was 403) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 51 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 19s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
39s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 85m 38s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m
55s{color} | {color:green} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}206m 52s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSClientRetries |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:b78c94f |
| JIRA Issue | HDFS-13489 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920679/HDFS-13489.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle cc |
| uname | Linux 0a4d43c8c1ae
[jira] [Created] (HDFS-13505) Turn on HDFS ACLs by default.
Ajay Kumar created HDFS-13505: - Summary: Turn on HDFS ACLs by default. Key: HDFS-13505 URL: https://issues.apache.org/jira/browse/HDFS-13505 Project: Hadoop HDFS Issue Type: Improvement Reporter: Ajay Kumar Assignee: Ajay Kumar Turn on HDFS ACLs by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13399) Make Client field AlignmentContext non-static.
[
https://issues.apache.org/jira/browse/HDFS-13399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453277#comment-16453277
]
genericqa commented on HDFS-13399:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-12943 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m
10s{color} | {color:green} HDFS-12943 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 31m
52s{color} | {color:green} HDFS-12943 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
16s{color} | {color:green} HDFS-12943 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
32s{color} | {color:green} HDFS-12943 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 26s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
43s{color} | {color:green} HDFS-12943 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
23s{color} | {color:green} HDFS-12943 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 29m
24s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 20s{color} | {color:orange} root: The patch generated 3 new + 544 unchanged
- 0 fixed = 547 total (was 544) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 30s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
7s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
35s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}102m 30s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}260m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestStateAlignmentContextWithHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
| JIRA Issue | HDFS-13399 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920662/HDFS-13399-HDFS-12943.005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 03be40d80b4e 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Updated] (HDFS-13503) Fix TestFsck test failures on Windows
[ https://issues.apache.org/jira/browse/HDFS-13503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Liang updated HDFS-13503: -- Status: Patch Available (was: Open) > Fix TestFsck test failures on Windows > - > > Key: HDFS-13503 > URL: https://issues.apache.org/jira/browse/HDFS-13503 > Project: Hadoop HDFS > Issue Type: Test > Components: hdfs >Reporter: Xiao Liang >Assignee: Xiao Liang >Priority: Major > Labels: windows > Attachments: HDFS-13503-branch-2.000.patch > > > Test failures on Windows caused by the same reason as HDFS-13336, similar fix > needed for TestFsck basing on HDFS-13408. > MiniDFSCluster also needs a small fix for the getStorageDir() interface, > which should use determineDfsBaseDir() to get the correct path of the data > directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13503) Fix TestFsck test failures on Windows
[ https://issues.apache.org/jira/browse/HDFS-13503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Liang updated HDFS-13503: -- Attachment: HDFS-13503-branch-2.000.patch > Fix TestFsck test failures on Windows > - > > Key: HDFS-13503 > URL: https://issues.apache.org/jira/browse/HDFS-13503 > Project: Hadoop HDFS > Issue Type: Test > Components: hdfs >Reporter: Xiao Liang >Assignee: Xiao Liang >Priority: Major > Labels: windows > Attachments: HDFS-13503-branch-2.000.patch > > > Test failures on Windows caused by the same reason as HDFS-13336, similar fix > needed for TestFsck basing on HDFS-13408. > MiniDFSCluster also needs a small fix for the getStorageDir() interface, > which should use determineDfsBaseDir() to get the correct path of the data > directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13501) Secure Datanode stop/start from cli does not throw a valid error if HDFS_DATANODE_SECURE_USER is not set
[ https://issues.apache.org/jira/browse/HDFS-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453262#comment-16453262 ] Allen Wittenauer commented on HDFS-13501: - JSVC_HOME is allowed to be configured independently of HDFS_DATANODE_SECURE_USER due to other services that may be using the secure starter code. For example, it's possible to have NFS running in secure mode but not the datanode. OOTB, the only way to tell the shell code if the datanode needs to use the secure daemon starter is via HDFS_DATANODE_SECURE_USER. Since having it set and unset are legal, there's no real way to predict what the user intends without reading through hdfs-site.xml, looking at port numbers, rpc settings, and the like and even then, we might get it wrong. For example, if Hadoop is using authbind or pfexec or any number of other ways to give a process the ability to open reserved ports. They are a little more complicated, but the shell code does support it via hadoop-user-functions. This flexibility is exactly why the current shell doesn't enforce the strict rules that 2.x did. > Secure Datanode stop/start from cli does not throw a valid error if > HDFS_DATANODE_SECURE_USER is not set > > > Key: HDFS-13501 > URL: https://issues.apache.org/jira/browse/HDFS-13501 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > > Secure Datanode start/stop from cli does not throw a valid error if > HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If > HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to > fail (when privilege ports are used) but it should show some valid message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13501) Secure Datanode stop/start from cli does not throw a valid error if HDFS_DATANODE_SECURE_USER is not set
[ https://issues.apache.org/jira/browse/HDFS-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453224#comment-16453224 ] Ajay Kumar commented on HDFS-13501: --- [~aw] true, this is one case when secure DataNode is configured on privileged ports. JSVC_HOME is configured correctly but secure user is left blank. In this case shell script never invokes corresponding secure/unsecure Java class and also prints no error message. I checked hadoop-functions.sh and don't see any easy answer to it. Any suggestions? > Secure Datanode stop/start from cli does not throw a valid error if > HDFS_DATANODE_SECURE_USER is not set > > > Key: HDFS-13501 > URL: https://issues.apache.org/jira/browse/HDFS-13501 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > > Secure Datanode start/stop from cli does not throw a valid error if > HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If > HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to > fail (when privilege ports are used) but it should show some valid message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13432) Ozone: When datanodes register, send NodeReport and ContainerReport
[
https://issues.apache.org/jira/browse/HDFS-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453153#comment-16453153
]
genericqa commented on HDFS-13432:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
17s{color} | {color:red} container-service in trunk failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
17s{color} | {color:red} server-scm in trunk failed. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 11s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
16s{color} | {color:red} container-service in trunk failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
15s{color} | {color:red} server-scm in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
15s{color} | {color:red} container-service in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
16s{color} | {color:red} server-scm in trunk failed. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
11s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
11s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
11s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
11s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 38s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
13s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
14s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
14s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
14s{color} | {color:red} server-scm in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 13s{color}
| {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 14s{color}
| {color:red} server-scm in the patch failed. {color} |
[jira] [Commented] (HDFS-13468) Add erasure coding metrics into ReadStatistics
[ https://issues.apache.org/jira/browse/HDFS-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453111#comment-16453111 ] Hudson commented on HDFS-13468: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14064 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14064/]) HDFS-13468. Add erasure coding metrics into ReadStatistics. (Contributed (lei: rev 96a57c967d6cca32df0534c15da8a573cd155814) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/client/impl/TestBlockReaderLocal.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ReadStatistics.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripeReader.java > Add erasure coding metrics into ReadStatistics > -- > > Key: HDFS-13468 > URL: https://issues.apache.org/jira/browse/HDFS-13468 > Project: Hadoop HDFS > Issue Type: Improvement > Components: erasure-coding >Affects Versions: 3.1.0, 3.0.1 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu >Priority: Major > Fix For: 3.2.0 > > Attachments: HDFS-13468.00.patch, HDFS-13468.01.patch, > HDFS-13468.02.patch > > > Expose Erasure Coding related metrics for InputStream in ReadStatistics. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13501) Secure Datanode stop/start from cli does not throw a valid error if HDFS_DATANODE_SECURE_USER is not set
[ https://issues.apache.org/jira/browse/HDFS-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453085#comment-16453085 ] Allen Wittenauer commented on HDFS-13501: - It's possible to run a secure datanode without using jsvc. Thus why the shell code doesn't check for those values. > Secure Datanode stop/start from cli does not throw a valid error if > HDFS_DATANODE_SECURE_USER is not set > > > Key: HDFS-13501 > URL: https://issues.apache.org/jira/browse/HDFS-13501 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > > Secure Datanode start/stop from cli does not throw a valid error if > HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If > HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to > fail (when privilege ports are used) but it should show some valid message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13468) Add erasure coding metrics into ReadStatistics
[ https://issues.apache.org/jira/browse/HDFS-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-13468: - Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch Available) The test failures were not relevant, and all of them passed on my machine. Thanks [~xiaochen], committed to trunk. > Add erasure coding metrics into ReadStatistics > -- > > Key: HDFS-13468 > URL: https://issues.apache.org/jira/browse/HDFS-13468 > Project: Hadoop HDFS > Issue Type: Improvement > Components: erasure-coding >Affects Versions: 3.1.0, 3.0.1 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu >Priority: Major > Fix For: 3.2.0 > > Attachments: HDFS-13468.00.patch, HDFS-13468.01.patch, > HDFS-13468.02.patch > > > Expose Erasure Coding related metrics for InputStream in ReadStatistics. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13489) Get base snapshotable path if exists for a given path
[ https://issues.apache.org/jira/browse/HDFS-13489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harkrishn Patro updated HDFS-13489: --- Attachment: HDFS-13489.003.patch > Get base snapshotable path if exists for a given path > - > > Key: HDFS-13489 > URL: https://issues.apache.org/jira/browse/HDFS-13489 > Project: Hadoop HDFS > Issue Type: Task > Components: hdfs >Reporter: Harkrishn Patro >Assignee: Harkrishn Patro >Priority: Major > Attachments: HDFS-13489.001.patch, HDFS-13489.002.patch, > HDFS-13489.003.patch > > > Currently, hdfs only lists the snapshotable paths in the filesystem. This > feature would add the functionality of figuring out if a given path is > snapshotable or not. If yes, it would return the base snapshotable path. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13432) Ozone: When datanodes register, send NodeReport and ContainerReport
[ https://issues.apache.org/jira/browse/HDFS-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453051#comment-16453051 ] Bharat Viswanadham commented on HDFS-13432: --- Uploaded the rebased patch as now ozone is merged to trunk. > Ozone: When datanodes register, send NodeReport and ContainerReport > --- > > Key: HDFS-13432 > URL: https://issues.apache.org/jira/browse/HDFS-13432 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Attachments: HDFS-13432-HDFS-7240.00.patch, HDFS-13432.01.patch > > > From chillmode Deisgn Notes: > As part of this Jira, will update register to send NodeReport and > ContaineReport. > Current Datanodes, send one heartbeat per 30 seconds. That means that even if > the datanode is ready it will take around a 1 min or longer before the SCM > sees the datanode container reports. We can address this partially by making > sure that Register call contains both NodeReport and ContainerReport. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13432) Ozone: When datanodes register, send NodeReport and ContainerReport
[ https://issues.apache.org/jira/browse/HDFS-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-13432: -- Attachment: HDFS-13432.01.patch > Ozone: When datanodes register, send NodeReport and ContainerReport > --- > > Key: HDFS-13432 > URL: https://issues.apache.org/jira/browse/HDFS-13432 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Attachments: HDFS-13432-HDFS-7240.00.patch, HDFS-13432.01.patch > > > From chillmode Deisgn Notes: > As part of this Jira, will update register to send NodeReport and > ContaineReport. > Current Datanodes, send one heartbeat per 30 seconds. That means that even if > the datanode is ready it will take around a 1 min or longer before the SCM > sees the datanode container reports. We can address this partially by making > sure that Register call contains both NodeReport and ContainerReport. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13245) RBF: State store DBMS implementation
[
https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452959#comment-16452959
]
genericqa commented on HDFS-13245:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
31s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 31m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 51s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-assemblies {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 30m
27s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 9s{color} | {color:orange} root: The patch generated 21 new + 0 unchanged -
0 fixed = 21 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 8s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-assemblies {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
59s{color} | {color:green} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}146m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:b78c94f |
| JIRA Issue | HDFS-13245 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920645/HDFS-13245.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit
[jira] [Commented] (HDFS-13369) FSCK Report broken with RequestHedgingProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-13369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452943#comment-16452943
]
genericqa commented on HDFS-13369:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
58s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 19s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
25s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m
2s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 17s{color} | {color:orange} root: The patch generated 264 new + 169
unchanged - 5 fixed = 433 total (was 174) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 8 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 257 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 35s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
13s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
39s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 55s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}228m 49s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
| | hadoop.hdfs.server.namenode.ha.TestHAStateTransitions |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.tools.TestDFSZKFailoverController |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.namenode.ha.TestHASafeMode |
| | hadoop.hdfs.TestDFSClientFailover |
| |
[jira] [Commented] (HDFS-13286) Add haadmin commands to transition between standby and observer
[
https://issues.apache.org/jira/browse/HDFS-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452870#comment-16452870
]
Chao Sun commented on HDFS-13286:
-
I think I missed a few places where {{HAServiceStateProto.OBSERVER}} should be
added. Will fix and upload another patch.
> Add haadmin commands to transition between standby and observer
> ---
>
> Key: HDFS-13286
> URL: https://issues.apache.org/jira/browse/HDFS-13286
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-13286-HDFS-12943.000.patch,
> HDFS-13286-HDFS-12943.001.patch, HDFS-13286-HDFS-12943.002.patch,
> HDFS-13286-HDFS-12943.003.patch
>
>
> As discussed in HDFS-12975, we should allow explicit transition between
> standby and observer through haadmin command, such as:
> {code}
> haadmin -transitionToObserver
> {code}
> Initially we should support transition from observer to standby, and standby
> to observer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13399) Make Client field AlignmentContext non-static.
[
https://issues.apache.org/jira/browse/HDFS-13399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452862#comment-16452862
]
Plamen Jeliazkov commented on HDFS-13399:
-
Hey [~shv],
Yes there was some misunderstanding. I was under the impression from the
comment here that you were implying those changes were safe OK to keep in; just
a mistake on my end:
https://issues.apache.org/jira/browse/HDFS-13399?focusedCommentId=16448937=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16448937.
I went ahead and removed nearly all the changes from {{NameNodeProxies}} now.
The only change I had to do was that {{NameNodeProxies.createNonHAProxy}} calls
into {{NameNodeProxiesClient.createNonHAProxyWithClientProtocol}}. Since
{{NameNodeProxiesClient.createNonHAProxyWithClientProtocol}} takes
{{AlignmentContext}} I made the {{NameNodeProxies}} call simply pass null. We
can overload the {{NameNodeProxiesClient.createNonHAProxyWithClientProtocol}}
call if you want to have zero changes to {{NameNodeProxies}}.
The changes within {{NameNodeProxiesClient}} are necessary to keep because of
{{ClientHAProxyFactory}} class calls it, which extends {{HAProxyFactory}}
interface. {{HAProxyFactory}} change caused us to change
{{NameNodeHAProxyFactory}} which is how I ended up modifying
{{NameNodeProxies}}.
I hope this new patch will clear things up now. Please take a look and let me
know.
> Make Client field AlignmentContext non-static.
> --
>
> Key: HDFS-13399
> URL: https://issues.apache.org/jira/browse/HDFS-13399
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Affects Versions: HDFS-12943
>Reporter: Plamen Jeliazkov
>Assignee: Plamen Jeliazkov
>Priority: Major
> Attachments: HDFS-13399-HDFS-12943.000.patch,
> HDFS-13399-HDFS-12943.001.patch, HDFS-13399-HDFS-12943.002.patch,
> HDFS-13399-HDFS-12943.003.patch, HDFS-13399-HDFS-12943.004.patch,
> HDFS-13399-HDFS-12943.005.patch
>
>
> In HDFS-12977, DFSClient's constructor was altered to make use of a new
> static method in Client that allowed one to set an AlignmentContext. This
> work is to remove that static field and make each DFSClient pass it's
> AlignmentContext down to the proxy Call level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13399) Make Client field AlignmentContext non-static.
[ https://issues.apache.org/jira/browse/HDFS-13399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Plamen Jeliazkov updated HDFS-13399: Attachment: HDFS-13399-HDFS-12943.005.patch > Make Client field AlignmentContext non-static. > -- > > Key: HDFS-13399 > URL: https://issues.apache.org/jira/browse/HDFS-13399 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-12943 >Reporter: Plamen Jeliazkov >Assignee: Plamen Jeliazkov >Priority: Major > Attachments: HDFS-13399-HDFS-12943.000.patch, > HDFS-13399-HDFS-12943.001.patch, HDFS-13399-HDFS-12943.002.patch, > HDFS-13399-HDFS-12943.003.patch, HDFS-13399-HDFS-12943.004.patch, > HDFS-13399-HDFS-12943.005.patch > > > In HDFS-12977, DFSClient's constructor was altered to make use of a new > static method in Client that allowed one to set an AlignmentContext. This > work is to remove that static field and make each DFSClient pass it's > AlignmentContext down to the proxy Call level. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13501) Secure Datanode stop/start from cli does not throw a valid error if HDFS_DATANODE_SECURE_USER is not set
[
https://issues.apache.org/jira/browse/HDFS-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452857#comment-16452857
]
genericqa commented on HDFS-13501:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 28s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
12s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 50s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
27s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:b78c94f |
| JIRA Issue | HDFS-13501 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920648/HDFS-13501.00.patch |
| Optional Tests | asflicense mvnsite unit shellcheck shelldocs |
| uname | Linux 25fecff5f8ba 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 6266906 |
| maven | version: Apache Maven 3.3.9 |
| shellcheck | v0.4.6 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/24069/testReport/ |
| Max. process+thread count | 342 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/24069/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Secure Datanode stop/start from cli does not throw a valid error if
> HDFS_DATANODE_SECURE_USER is not set
>
>
> Key: HDFS-13501
> URL: https://issues.apache.org/jira/browse/HDFS-13501
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Ajay Kumar
>Assignee: Ajay Kumar
>Priority: Major
>
> Secure Datanode start/stop from cli does not throw a valid error if
> HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If
> HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to
> fail (when privilege ports are used) but it should show some valid message.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13504) Ozone: Remove SCM Block DB
[
https://issues.apache.org/jira/browse/HDFS-13504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452837#comment-16452837
]
genericqa commented on HDFS-13504:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
10s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
30s{color} | {color:red} server-scm in trunk failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
30s{color} | {color:red} integration-test in trunk failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
27s{color} | {color:red} ozone-manager in trunk failed. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 24s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
55s{color} | {color:red} hadoop-hdds/common in trunk has 1 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
23s{color} | {color:red} server-scm in trunk failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
23s{color} | {color:red} ozone-manager in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
23s{color} | {color:red} server-scm in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
24s{color} | {color:red} integration-test in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
23s{color} | {color:red} ozone-manager in trunk failed. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
12s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
10s{color} | {color:red} integration-test in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
10s{color} | {color:red} ozone-manager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 28m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 28m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
22s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
23s{color} | {color:red} integration-test in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
23s{color} | {color:red} ozone-manager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 27s{color} | {color:green} patch has no errors when building and testing
our client artifacts.
[jira] [Comment Edited] (HDFS-13245) RBF: State store DBMS implementation
[ https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452780#comment-16452780 ] Íñigo Goiri edited comment on HDFS-13245 at 4/25/18 6:00 PM: - Is there any plugin or framework that we can use to at least check the syntax of the SQL scripts? Maven may even have something for that. There must be some library for creating simple queries like SELECT and CREATE instead of just having strings. was (Author: elgoiri): Is there any plugin or framework that we can use to at least check the syntax of the SQL scripts? Maven may even have something for that. > RBF: State store DBMS implementation > > > Key: HDFS-13245 > URL: https://issues.apache.org/jira/browse/HDFS-13245 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: maobaolong >Assignee: Yiran Wu >Priority: Major > Attachments: HDFS-13245.001.patch, HDFS-13245.002.patch, > HDFS-13245.003.patch > > > Add a DBMS implementation for the State Store. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13245) RBF: State store DBMS implementation
[ https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452780#comment-16452780 ] Íñigo Goiri commented on HDFS-13245: Is there any plugin or framework that we can use to at least check the syntax of the SQL scripts? Maven may evne have something for that. > RBF: State store DBMS implementation > > > Key: HDFS-13245 > URL: https://issues.apache.org/jira/browse/HDFS-13245 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: maobaolong >Assignee: Yiran Wu >Priority: Major > Attachments: HDFS-13245.001.patch, HDFS-13245.002.patch, > HDFS-13245.003.patch > > > Add a DBMS implementation for the State Store. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13245) RBF: State store DBMS implementation
[ https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452780#comment-16452780 ] Íñigo Goiri edited comment on HDFS-13245 at 4/25/18 5:58 PM: - Is there any plugin or framework that we can use to at least check the syntax of the SQL scripts? Maven may even have something for that. was (Author: elgoiri): Is there any plugin or framework that we can use to at least check the syntax of the SQL scripts? Maven may evne have something for that. > RBF: State store DBMS implementation > > > Key: HDFS-13245 > URL: https://issues.apache.org/jira/browse/HDFS-13245 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: maobaolong >Assignee: Yiran Wu >Priority: Major > Attachments: HDFS-13245.001.patch, HDFS-13245.002.patch, > HDFS-13245.003.patch > > > Add a DBMS implementation for the State Store. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13495) RBF: Support Router Admin REST API
[ https://issues.apache.org/jira/browse/HDFS-13495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452777#comment-16452777 ] Íñigo Goiri commented on HDFS-13495: That's what I though; I cannot remember REST being used for administration. I'm concerned about diverging from the rest of HDFS but I think is a valid use case. [~chris.douglas], do you have any insight on why there is no REST API for admin? > RBF: Support Router Admin REST API > -- > > Key: HDFS-13495 > URL: https://issues.apache.org/jira/browse/HDFS-13495 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Mohammad Arshad >Priority: Major > Labels: RBF > > This JIRA intends to add REST API support for all admin commands. Router > Admin REST APIs can be useful in managing the Routers from a central > management layer tool. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13501) Secure Datanode stop/start from cli does not throw a valid error if HDFS_DATANODE_SECURE_USER is not set
[ https://issues.apache.org/jira/browse/HDFS-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13501: -- Status: Open (was: Patch Available) > Secure Datanode stop/start from cli does not throw a valid error if > HDFS_DATANODE_SECURE_USER is not set > > > Key: HDFS-13501 > URL: https://issues.apache.org/jira/browse/HDFS-13501 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > > Secure Datanode start/stop from cli does not throw a valid error if > HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If > HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to > fail (when privilege ports are used) but it should show some valid message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13501) Secure Datanode stop/start from cli does not throw a valid error if HDFS_DATANODE_SECURE_USER is not set
[ https://issues.apache.org/jira/browse/HDFS-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13501: -- Attachment: (was: HDFS-13501.00.patch) > Secure Datanode stop/start from cli does not throw a valid error if > HDFS_DATANODE_SECURE_USER is not set > > > Key: HDFS-13501 > URL: https://issues.apache.org/jira/browse/HDFS-13501 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > > Secure Datanode start/stop from cli does not throw a valid error if > HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If > HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to > fail (when privilege ports are used) but it should show some valid message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13443) RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries.
[
https://issues.apache.org/jira/browse/HDFS-13443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452757#comment-16452757
]
Íñigo Goiri commented on HDFS-13443:
[~arshad.mohammad], thanks for [^HDFS-13443.005.patch]; te new naming is
easier to follow.
Sorry for the fascicle review but this is a long one and is taking me a while
to grasp the nuances.
The run for TestRouterAdminWithZKStateStore is
[here|https://builds.apache.org/job/PreCommit-HDFS-Build/24063/testReport/org.apache.hadoop.hdfs.server.federation.router/TestRouterAdminWithZKStateStore/].
Some comments:
* I'm not sure it should be called like that as it's basically testing the
functionality for this JIRA. I would probably call it
TestRemoteRouterMountTableRefresh or similar; no need for the ZK suffix either.
* We have the StateStoreDFSCluster; it may make sense to have the ZK based as a
class for making this reusable in the future and simplify the test.
* I'd like to have some testing for the admin address of the Router. We could
add a check for the Router admin address for all the Routers.
* Typo "do noting".
* Typo "can not" to "cannot".
* Log exceptions instead of printStackTrace().
* You can do the {{MountTableManager mountTableManager =
routerContext.getAdminClient().getMountTableManager();}} in the setup of the
test.
* Use GenericTestUtils.waitFor for waitForRouterRegistration().
* For stopping one Router, just pick one, no need to go through the list.
* I think we need some metric to make sure we run the update operations. Right
now, it might be just luck that we hit the new mount table entries.
* The helper methods should have full names, getNumMountTableEntries(),
addMountTableEntry(), etc. Basically, making them consistent across the test.
Other random comments:
* RouterAdmin, the help message needs ] at the end.
* We need to update the documentation, the federation one and hdfs commands.
* The fields in MountTableRefreshService should all have javadocs and ideally
following the /** */ format.
* MountTableRefreshService#90 to follow the same naming as 94.
* Rename MountTableRefreshService#getRemover() to getClientRemover() as the
creator.
* Message should be something like "Not all router admins updated their cache".
* I would propagate the error of not all Router request completed all the way
to the client
> RBF: Update mount table cache immediately after changing (add/update/remove)
> mount table entries.
> -
>
> Key: HDFS-13443
> URL: https://issues.apache.org/jira/browse/HDFS-13443
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Labels: RBF
> Attachments: HDFS-13443-branch-2.001.patch,
> HDFS-13443-branch-2.002.patch, HDFS-13443.001.patch, HDFS-13443.002.patch,
> HDFS-13443.003.patch, HDFS-13443.004.patch, HDFS-13443.005.patch
>
>
> Currently mount table cache is updated periodically, by default cache is
> updated every minute. After change in mount table, user operations may still
> use old mount table. This is bit wrong.
> To update mount table cache, maybe we can do following
> * *Add refresh API in MountTableManager which will update mount table cache.*
> * *When there is a change in mount table entries, router admin server can
> update its cache and ask other routers to update their cache*. For example if
> there are three routers R1,R2,R3 in a cluster then add mount table entry API,
> at admin server side, will perform following sequence of action
> ## user submit add mount table entry request on R1
> ## R1 adds the mount table entry in state store
> ## R1 call refresh API on R2
> ## R1 calls refresh API on R3
> ## R1 directly freshest its cache
> ## Add mount table entry response send back to user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13286) Add haadmin commands to transition between standby and observer
[
https://issues.apache.org/jira/browse/HDFS-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452736#comment-16452736
]
Chao Sun commented on HDFS-13286:
-
Attached patch v3.
> Add haadmin commands to transition between standby and observer
> ---
>
> Key: HDFS-13286
> URL: https://issues.apache.org/jira/browse/HDFS-13286
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-13286-HDFS-12943.000.patch,
> HDFS-13286-HDFS-12943.001.patch, HDFS-13286-HDFS-12943.002.patch,
> HDFS-13286-HDFS-12943.003.patch
>
>
> As discussed in HDFS-12975, we should allow explicit transition between
> standby and observer through haadmin command, such as:
> {code}
> haadmin -transitionToObserver
> {code}
> Initially we should support transition from observer to standby, and standby
> to observer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13286) Add haadmin commands to transition between standby and observer
[
https://issues.apache.org/jira/browse/HDFS-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HDFS-13286:
Attachment: HDFS-13286-HDFS-12943.003.patch
> Add haadmin commands to transition between standby and observer
> ---
>
> Key: HDFS-13286
> URL: https://issues.apache.org/jira/browse/HDFS-13286
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-13286-HDFS-12943.000.patch,
> HDFS-13286-HDFS-12943.001.patch, HDFS-13286-HDFS-12943.002.patch,
> HDFS-13286-HDFS-12943.003.patch
>
>
> As discussed in HDFS-12975, we should allow explicit transition between
> standby and observer through haadmin command, such as:
> {code}
> haadmin -transitionToObserver
> {code}
> Initially we should support transition from observer to standby, and standby
> to observer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13286) Add haadmin commands to transition between standby and observer
[
https://issues.apache.org/jira/browse/HDFS-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452734#comment-16452734
]
Erik Krogen commented on HDFS-13286:
Seems fine to me, sure.
> Add haadmin commands to transition between standby and observer
> ---
>
> Key: HDFS-13286
> URL: https://issues.apache.org/jira/browse/HDFS-13286
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-13286-HDFS-12943.000.patch,
> HDFS-13286-HDFS-12943.001.patch, HDFS-13286-HDFS-12943.002.patch
>
>
> As discussed in HDFS-12975, we should allow explicit transition between
> standby and observer through haadmin command, such as:
> {code}
> haadmin -transitionToObserver
> {code}
> Initially we should support transition from observer to standby, and standby
> to observer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13492) Limit httpfs binds to certain IP addresses in branch-2
[ https://issues.apache.org/jira/browse/HDFS-13492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452732#comment-16452732 ] Wei-Chiu Chuang commented on HDFS-13492: test failure unrelated. I'll commit rev002 patch in branch-2 later. > Limit httpfs binds to certain IP addresses in branch-2 > -- > > Key: HDFS-13492 > URL: https://issues.apache.org/jira/browse/HDFS-13492 > Project: Hadoop HDFS > Issue Type: Improvement > Components: httpfs >Affects Versions: 2.6.0 >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Attachments: HDFS-13492.branch-2.001.patch, > HDFS-13492.branch-2.002.patch > > > Currently httpfs binds to all IP addresses of the host by default. Some > operators want to limit httpfs to accept only local connections. > We should provide that option, and it's pretty doable in Hadoop 2.x. > Note that httpfs underlying implementation changed in Hadoop 3, and the Jetty > based httpfs implementation already support that I believe. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13286) Add haadmin commands to transition between standby and observer
[
https://issues.apache.org/jira/browse/HDFS-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452731#comment-16452731
]
Chao Sun commented on HDFS-13286:
-
Thanks [~xkrogen]. Will address the issues. In the style issues though:
{code}
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/DummyHAService.java:59:
boolean failToBecomeActive, failToBecomeStandby, failToBecomeObserver,:11:
Variable 'failToBecomeActive' must be private and have accessor methods.
[VisibilityModifier]
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/DummyHAService.java:59:
boolean failToBecomeActive, failToBecomeStandby, failToBecomeObserver,:31:
Variable 'failToBecomeStandby' must be private and have accessor methods.
[VisibilityModifier]
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/DummyHAService.java:59:
boolean failToBecomeActive, failToBecomeStandby, failToBecomeObserver,:52:
Variable 'failToBecomeObserver' must be private and have accessor methods.
[VisibilityModifier]
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/DummyHAService.java:60:
failToFence;:7: Variable 'failToFence' must be private and have accessor
methods. [VisibilityModifier]
{code}
This might be more involved and is not so related to this JIRA. Should we leave
them as it is for now?
> Add haadmin commands to transition between standby and observer
> ---
>
> Key: HDFS-13286
> URL: https://issues.apache.org/jira/browse/HDFS-13286
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-13286-HDFS-12943.000.patch,
> HDFS-13286-HDFS-12943.001.patch, HDFS-13286-HDFS-12943.002.patch
>
>
> As discussed in HDFS-12975, we should allow explicit transition between
> standby and observer through haadmin command, such as:
> {code}
> haadmin -transitionToObserver
> {code}
> Initially we should support transition from observer to standby, and standby
> to observer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13502) Utility to resolve NameServiceId in federated cluster
[ https://issues.apache.org/jira/browse/HDFS-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452725#comment-16452725 ] Arpit Agarwal commented on HDFS-13502: -- Thanks for filing this [~apoorvnaik]. We can potentially expose this mapping via HdfsUtils, which is a public interface. https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java > Utility to resolve NameServiceId in federated cluster > - > > Key: HDFS-13502 > URL: https://issues.apache.org/jira/browse/HDFS-13502 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Apoorv Naik >Assignee: Apoorv Naik >Priority: Major > > A Utility class in HDFS, that would act as a reverse lookup for : > HDFS URLs would be beneficial for deployments having multiple namenodes and > nameservices. > > Consumers would benefit by having a unified namespace across the Federated > cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13501) Secure Datanode stop/start from cli does not throw a valid error if HDFS_DATANODE_SECURE_USER is not set
[ https://issues.apache.org/jira/browse/HDFS-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13501: -- Description: Secure Datanode start/stop from cli does not throw a valid error if HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to fail (when privilege ports are used) but it should show some valid message. (was: Secure Datanode start/stop from cli does not throw a valid error if HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to fail (when privilege ports are used) but it should some valid message.) > Secure Datanode stop/start from cli does not throw a valid error if > HDFS_DATANODE_SECURE_USER is not set > > > Key: HDFS-13501 > URL: https://issues.apache.org/jira/browse/HDFS-13501 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > Attachments: HDFS-13501.00.patch > > > Secure Datanode start/stop from cli does not throw a valid error if > HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If > HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to > fail (when privilege ports are used) but it should show some valid message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13501) Secure Datanode stop/start from cli does not throw a valid error if HDFS_DATANODE_SECURE_USER is not set
[ https://issues.apache.org/jira/browse/HDFS-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13501: -- Status: Patch Available (was: Open) > Secure Datanode stop/start from cli does not throw a valid error if > HDFS_DATANODE_SECURE_USER is not set > > > Key: HDFS-13501 > URL: https://issues.apache.org/jira/browse/HDFS-13501 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > Attachments: HDFS-13501.00.patch > > > Secure Datanode start/stop from cli does not throw a valid error if > HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If > HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to > fail (when privilege ports are used) but it should some valid message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13501) Secure Datanode stop/start from cli does not throw a valid error if HDFS_DATANODE_SECURE_USER is not set
[ https://issues.apache.org/jira/browse/HDFS-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13501: -- Attachment: HDFS-13501.00.patch > Secure Datanode stop/start from cli does not throw a valid error if > HDFS_DATANODE_SECURE_USER is not set > > > Key: HDFS-13501 > URL: https://issues.apache.org/jira/browse/HDFS-13501 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > Attachments: HDFS-13501.00.patch > > > Secure Datanode start/stop from cli does not throw a valid error if > HADOOP_SECURE_DN_USER/HDFS_DATANODE_SECURE_USER is not set. If > HDFS_DATANODE_SECURE_USER and JSVC_HOME is not set start/stop is expected to > fail (when privilege ports are used) but it should some valid message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13491) [SPS]: Discuss and implement efficient approach to send a copy of a block to another datanode
[ https://issues.apache.org/jira/browse/HDFS-13491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452686#comment-16452686 ] Erik Krogen commented on HDFS-13491: [~rakeshr], looks like you meant to include a reference to some discussion, but it does not appear to be linked. Can you update? > [SPS]: Discuss and implement efficient approach to send a copy of a block to > another datanode > - > > Key: HDFS-13491 > URL: https://issues.apache.org/jira/browse/HDFS-13491 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Major > > This Jira task is to reach consensus about the block transfer logic to > another data node and implement the same to satisfy block storage policy. > Reference discussion thread -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13245) RBF: State store DBMS implementation
[ https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452658#comment-16452658 ] Yiran Wu commented on HDFS-13245: - Add new patch. move the sql scripts to the RBF module folder. > RBF: State store DBMS implementation > > > Key: HDFS-13245 > URL: https://issues.apache.org/jira/browse/HDFS-13245 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: maobaolong >Assignee: Yiran Wu >Priority: Major > Attachments: HDFS-13245.001.patch, HDFS-13245.002.patch, > HDFS-13245.003.patch > > > Add a DBMS implementation for the State Store. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13245) RBF: State store DBMS implementation
[ https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiran Wu updated HDFS-13245: Attachment: HDFS-13245.003.patch > RBF: State store DBMS implementation > > > Key: HDFS-13245 > URL: https://issues.apache.org/jira/browse/HDFS-13245 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: maobaolong >Assignee: Yiran Wu >Priority: Major > Attachments: HDFS-13245.001.patch, HDFS-13245.002.patch, > HDFS-13245.003.patch > > > Add a DBMS implementation for the State Store. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13245) RBF: State store DBMS implementation
[ https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiran Wu updated HDFS-13245: Status: Patch Available (was: Open) > RBF: State store DBMS implementation > > > Key: HDFS-13245 > URL: https://issues.apache.org/jira/browse/HDFS-13245 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: maobaolong >Assignee: Yiran Wu >Priority: Major > Attachments: HDFS-13245.001.patch, HDFS-13245.002.patch, > HDFS-13245.003.patch > > > Add a DBMS implementation for the State Store. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13245) RBF: State store DBMS implementation
[ https://issues.apache.org/jira/browse/HDFS-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiran Wu updated HDFS-13245: Status: Open (was: Patch Available) > RBF: State store DBMS implementation > > > Key: HDFS-13245 > URL: https://issues.apache.org/jira/browse/HDFS-13245 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: maobaolong >Assignee: Yiran Wu >Priority: Major > Attachments: HDFS-13245.001.patch, HDFS-13245.002.patch, > HDFS-13245.003.patch > > > Add a DBMS implementation for the State Store. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13495) RBF: Support Router Admin REST API
[ https://issues.apache.org/jira/browse/HDFS-13495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452639#comment-16452639 ] Mohammad Arshad commented on HDFS-13495: NN does not have REST API for admin commands. I have a scenario where I need to perform router admin operations but only HTTPS calls are allowed. This is why I was thinking to add REST for router admin operations. I am not sure if admin operation REST can be useful as a router feature. If it is useful as router feature I can submit a patch. [~elgoiri] what you think? > RBF: Support Router Admin REST API > -- > > Key: HDFS-13495 > URL: https://issues.apache.org/jira/browse/HDFS-13495 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Mohammad Arshad >Priority: Major > Labels: RBF > > This JIRA intends to add REST API support for all admin commands. Router > Admin REST APIs can be useful in managing the Routers from a central > management layer tool. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13499) RBF: Show disabled name services in the UI
[
https://issues.apache.org/jira/browse/HDFS-13499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452631#comment-16452631
]
genericqa commented on HDFS-13499:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
35m 51s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 30s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:b78c94f |
| JIRA Issue | HDFS-13499 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920640/HDFS-13499.000.patch |
| Optional Tests | asflicense shadedclient |
| uname | Linux 6cd819c69cf3 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 6266906 |
| maven | version: Apache Maven 3.3.9 |
| Max. process+thread count | 301 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/24065/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> RBF: Show disabled name services in the UI
> --
>
> Key: HDFS-13499
> URL: https://issues.apache.org/jira/browse/HDFS-13499
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
>Priority: Minor
> Attachments: HDFS-13499.000.patch
>
>
> HDFS-13484 exposes the disabled name services. This JIRA should show them in
> the Web UI.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13502) Utility to resolve NameServiceId in federated cluster
[ https://issues.apache.org/jira/browse/HDFS-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal reassigned HDFS-13502: Assignee: Apoorv Naik > Utility to resolve NameServiceId in federated cluster > - > > Key: HDFS-13502 > URL: https://issues.apache.org/jira/browse/HDFS-13502 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Apoorv Naik >Assignee: Apoorv Naik >Priority: Major > > A Utility class within the hadoop-common, that would act as a reverse lookup > for : HDFS URLs would be beneficial for deployments having > multiple namenodes and nameservices. > > Consumers would benefit by having a unified namespace across the Federated > cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13502) Utility to resolve NameServiceId in federated cluster
[ https://issues.apache.org/jira/browse/HDFS-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-13502: - Description: A Utility class in HDFS, that would act as a reverse lookup for : HDFS URLs would be beneficial for deployments having multiple namenodes and nameservices. Consumers would benefit by having a unified namespace across the Federated cluster. was: A Utility class within the hadoop-common, that would act as a reverse lookup for : HDFS URLs would be beneficial for deployments having multiple namenodes and nameservices. Consumers would benefit by having a unified namespace across the Federated cluster. > Utility to resolve NameServiceId in federated cluster > - > > Key: HDFS-13502 > URL: https://issues.apache.org/jira/browse/HDFS-13502 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Apoorv Naik >Assignee: Apoorv Naik >Priority: Major > > A Utility class in HDFS, that would act as a reverse lookup for : > HDFS URLs would be beneficial for deployments having multiple namenodes and > nameservices. > > Consumers would benefit by having a unified namespace across the Federated > cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12548) HDFS Jenkins build is unstable on branch-2
[
https://issues.apache.org/jira/browse/HDFS-12548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452601#comment-16452601
]
Erik Krogen commented on HDFS-12548:
Changed target version to 2.10.0 since this is a branch-2 issue.
Per [~aw]'s
[comment|https://issues.apache.org/jira/browse/HDFS-13272?focusedCommentId=16452535=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16452535],
branch-2 hasn't managed to run its test properly in a long time. Judging from
looking at a few log outputs, it looks like the HDFS tests always end up
hanging on some test (not always the same one). I am wondering if there is some
deadlock in the testing infra code (hopefully not in the production code).
> HDFS Jenkins build is unstable on branch-2
> --
>
> Key: HDFS-12548
> URL: https://issues.apache.org/jira/browse/HDFS-12548
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build
>Affects Versions: 2.9.0
>Reporter: Rushabh S Shah
>Priority: Critical
>
> Feel free move the ticket to another project (e.g. infra).
> Recently I attached branch-2 patch while working on one jira
> [HDFS-12386|https://issues.apache.org/jira/browse/HDFS-12386?focusedCommentId=16180676=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16180676]
> There were at-least 100 failed and timed out tests. I am sure they are not
> related to my patch.
> Also I came across another jira which was just a javadoc related change and
> there were around 100 failed tests.
> Below are the details for pre-commits that failed in branch-2
> 1 [HDFS-12386 attempt
> 1|https://issues.apache.org/jira/browse/HDFS-12386?focusedCommentId=16180069=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16180069]
> {noformat}
> Ran on slave: asf912.gq1.ygridcore.net/H12
> Failed with following error message:
> Build timed out (after 300 minutes). Marking the build as aborted.
> Build was aborted
> Performing Post build task...
> {noformat}
> 2. [HDFS-12386 attempt
> 2|https://issues.apache.org/jira/browse/HDFS-12386?focusedCommentId=16180676=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16180676]
> {noformat}
> Ran on slave: asf900.gq1.ygridcore.net
> Failed with following error message:
> FATAL: command execution failed
> Command close created at
> at hudson.remoting.Command.(Command.java:60)
> at hudson.remoting.Channel$CloseCommand.(Channel.java:1123)
> at hudson.remoting.Channel$CloseCommand.(Channel.java:1121)
> at hudson.remoting.Channel.close(Channel.java:1281)
> at hudson.remoting.Channel.close(Channel.java:1263)
> at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1128)
> Caused: hudson.remoting.Channel$OrderlyShutdown
> at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1129)
> at hudson.remoting.Channel$1.handle(Channel.java:527)
> at
> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:83)
> Caused: java.io.IOException: Backing channel 'H0' is disconnected.
> at
> hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:192)
> at
> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:257)
> at com.sun.proxy.$Proxy125.isAlive(Unknown Source)
> at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1043)
> at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1035)
> at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:155)
> at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109)
> at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
> at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:735)
> at hudson.model.Build$BuildExecution.build(Build.java:206)
> at hudson.model.Build$BuildExecution.doRun(Build.java:163)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
> at hudson.model.Run.execute(Run.java:1735)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:405)
> {noformat}
> 3. [HDFS-12531 attempt
> 1|https://issues.apache.org/jira/browse/HDFS-12531?focusedCommentId=16176493=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16176493]
> {noformat}
> Ran on slave: asf911.gq1.ygridcore.net
> Failed with following error message:
> FATAL: command execution failed
> Command close created at
> at hudson.remoting.Command.(Command.java:60)
> at
[jira] [Updated] (HDFS-12548) HDFS Jenkins build is unstable on branch-2
[
https://issues.apache.org/jira/browse/HDFS-12548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erik Krogen updated HDFS-12548:
---
Target Version/s: 2.10.0 (was: 3.2.0)
> HDFS Jenkins build is unstable on branch-2
> --
>
> Key: HDFS-12548
> URL: https://issues.apache.org/jira/browse/HDFS-12548
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build
>Affects Versions: 2.9.0
>Reporter: Rushabh S Shah
>Priority: Critical
>
> Feel free move the ticket to another project (e.g. infra).
> Recently I attached branch-2 patch while working on one jira
> [HDFS-12386|https://issues.apache.org/jira/browse/HDFS-12386?focusedCommentId=16180676=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16180676]
> There were at-least 100 failed and timed out tests. I am sure they are not
> related to my patch.
> Also I came across another jira which was just a javadoc related change and
> there were around 100 failed tests.
> Below are the details for pre-commits that failed in branch-2
> 1 [HDFS-12386 attempt
> 1|https://issues.apache.org/jira/browse/HDFS-12386?focusedCommentId=16180069=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16180069]
> {noformat}
> Ran on slave: asf912.gq1.ygridcore.net/H12
> Failed with following error message:
> Build timed out (after 300 minutes). Marking the build as aborted.
> Build was aborted
> Performing Post build task...
> {noformat}
> 2. [HDFS-12386 attempt
> 2|https://issues.apache.org/jira/browse/HDFS-12386?focusedCommentId=16180676=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16180676]
> {noformat}
> Ran on slave: asf900.gq1.ygridcore.net
> Failed with following error message:
> FATAL: command execution failed
> Command close created at
> at hudson.remoting.Command.(Command.java:60)
> at hudson.remoting.Channel$CloseCommand.(Channel.java:1123)
> at hudson.remoting.Channel$CloseCommand.(Channel.java:1121)
> at hudson.remoting.Channel.close(Channel.java:1281)
> at hudson.remoting.Channel.close(Channel.java:1263)
> at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1128)
> Caused: hudson.remoting.Channel$OrderlyShutdown
> at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1129)
> at hudson.remoting.Channel$1.handle(Channel.java:527)
> at
> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:83)
> Caused: java.io.IOException: Backing channel 'H0' is disconnected.
> at
> hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:192)
> at
> hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:257)
> at com.sun.proxy.$Proxy125.isAlive(Unknown Source)
> at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1043)
> at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1035)
> at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:155)
> at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109)
> at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
> at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:735)
> at hudson.model.Build$BuildExecution.build(Build.java:206)
> at hudson.model.Build$BuildExecution.doRun(Build.java:163)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
> at hudson.model.Run.execute(Run.java:1735)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:405)
> {noformat}
> 3. [HDFS-12531 attempt
> 1|https://issues.apache.org/jira/browse/HDFS-12531?focusedCommentId=16176493=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16176493]
> {noformat}
> Ran on slave: asf911.gq1.ygridcore.net
> Failed with following error message:
> FATAL: command execution failed
> Command close created at
> at hudson.remoting.Command.(Command.java:60)
> at hudson.remoting.Channel$CloseCommand.(Channel.java:1123)
> at hudson.remoting.Channel$CloseCommand.(Channel.java:1121)
> at hudson.remoting.Channel.close(Channel.java:1281)
> at hudson.remoting.Channel.close(Channel.java:1263)
> at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1128)
> Caused: hudson.remoting.Channel$OrderlyShutdown
> at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1129)
> at hudson.remoting.Channel$1.handle(Channel.java:527)
> at
>
[jira] [Comment Edited] (HDFS-13286) Add haadmin commands to transition between standby and observer
[
https://issues.apache.org/jira/browse/HDFS-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452588#comment-16452588
]
Erik Krogen edited comment on HDFS-13286 at 4/25/18 4:29 PM:
-
Last comments:
* NameNode#transitionToActive and #transitionToObserver has some hard-coded
strings, e.g. {{"Cannot transition from 'observer' to 'active'"}}, that could
be replaced with calls to {{OBSERVER_STATE.toString}} and
{{ACTIVE_STATE.toString}}
* Revisiting the {{ObserverState}} at a later time sounds fine to me
Also, I managed to find the precommit build; it's timing out (I think because
it's running the unit tests for all of hdfs, common, and yarn) but it completed
the [common unit
tests|https://builds.apache.org/job/PreCommit-HDFS-Build/24061/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt]
(clean) and the [hdfs unit
tests|https://builds.apache.org/job/PreCommit-HDFS-Build/24061/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
(failures but look unrelated). Please address the outstanding
[checkstyle|https://builds.apache.org/job/PreCommit-HDFS-Build/24061/artifact/out/diff-checkstyle-root.txt]
issues as well.
was (Author: xkrogen):
Last comments:
* NameNode#transitionToActive and #transitionToObserver has some hard-coded
strings, e.g. {{"Cannot transition from 'observer' to 'active'"}}, that could
be replaced with calls to {{OBSERVER_STATE.toString}} and
{{ACTIVE_STATE.toString}}
* Revisiting the {{ObserverState}} at a later time sounds fine to me
Also, I managed to find the precommit build; it's timing out (I think because
it's running the unit tests for all of hdfs, common, and yarn) but it completed
the [common unit
tests|https://builds.apache.org/job/PreCommit-HDFS-Build/24061/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt]
(clean) and the [hdfs unit
tests|https://builds.apache.org/job/PreCommit-HDFS-Build/24061/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
(failures but look unrelated). Please address the outstanding
[checkstylehttps://builds.apache.org/job/PreCommit-HDFS-Build/24061/artifact/out/diff-checkstyle-root.txt]
issues as well.
> Add haadmin commands to transition between standby and observer
> ---
>
> Key: HDFS-13286
> URL: https://issues.apache.org/jira/browse/HDFS-13286
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-13286-HDFS-12943.000.patch,
> HDFS-13286-HDFS-12943.001.patch, HDFS-13286-HDFS-12943.002.patch
>
>
> As discussed in HDFS-12975, we should allow explicit transition between
> standby and observer through haadmin command, such as:
> {code}
> haadmin -transitionToObserver
> {code}
> Initially we should support transition from observer to standby, and standby
> to observer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13286) Add haadmin commands to transition between standby and observer
[
https://issues.apache.org/jira/browse/HDFS-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452588#comment-16452588
]
Erik Krogen commented on HDFS-13286:
Last comments:
* NameNode#transitionToActive and #transitionToObserver has some hard-coded
strings, e.g. {{"Cannot transition from 'observer' to 'active'"}}, that could
be replaced with calls to {{OBSERVER_STATE.toString}} and
{{ACTIVE_STATE.toString}}
* Revisiting the {{ObserverState}} at a later time sounds fine to me
Also, I managed to find the precommit build; it's timing out (I think because
it's running the unit tests for all of hdfs, common, and yarn) but it completed
the [common unit
tests|https://builds.apache.org/job/PreCommit-HDFS-Build/24061/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt]
(clean) and the [hdfs unit
tests|https://builds.apache.org/job/PreCommit-HDFS-Build/24061/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
(failures but look unrelated). Please address the outstanding
[checkstylehttps://builds.apache.org/job/PreCommit-HDFS-Build/24061/artifact/out/diff-checkstyle-root.txt]
issues as well.
> Add haadmin commands to transition between standby and observer
> ---
>
> Key: HDFS-13286
> URL: https://issues.apache.org/jira/browse/HDFS-13286
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-13286-HDFS-12943.000.patch,
> HDFS-13286-HDFS-12943.001.patch, HDFS-13286-HDFS-12943.002.patch
>
>
> As discussed in HDFS-12975, we should allow explicit transition between
> standby and observer through haadmin command, such as:
> {code}
> haadmin -transitionToObserver
> {code}
> Initially we should support transition from observer to standby, and standby
> to observer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13504) Ozone: Remove SCM Block DB
[ https://issues.apache.org/jira/browse/HDFS-13504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-13504: -- Attachment: HDFS-13504.001.patch > Ozone: Remove SCM Block DB > -- > > Key: HDFS-13504 > URL: https://issues.apache.org/jira/browse/HDFS-13504 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Major > Attachments: HDFS-13504.001.patch > > > The block/key information is maintained by Ozone Master (a.k.a. KSM). This > ticket is opened to remove the redundant block db at SCM. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13504) Ozone: Remove SCM Block DB
[ https://issues.apache.org/jira/browse/HDFS-13504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-13504: -- Status: Patch Available (was: Open) > Ozone: Remove SCM Block DB > -- > > Key: HDFS-13504 > URL: https://issues.apache.org/jira/browse/HDFS-13504 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Major > Attachments: HDFS-13504.001.patch > > > The block/key information is maintained by Ozone Master (a.k.a. KSM). This > ticket is opened to remove the redundant block db at SCM. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13499) RBF: Show disabled name services in the UI
[ https://issues.apache.org/jira/browse/HDFS-13499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452559#comment-16452559 ] Íñigo Goiri commented on HDFS-13499: I'm working on deploying vanilla OSS 2.9 with this to verify the full disabled functionality. I'll attach a screenshot once I get it up and running. > RBF: Show disabled name services in the UI > -- > > Key: HDFS-13499 > URL: https://issues.apache.org/jira/browse/HDFS-13499 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Minor > Attachments: HDFS-13499.000.patch > > > HDFS-13484 exposes the disabled name services. This JIRA should show them in > the Web UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13499) RBF: Show disabled name services in the UI
[ https://issues.apache.org/jira/browse/HDFS-13499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-13499: --- Attachment: HDFS-13499.000.patch > RBF: Show disabled name services in the UI > -- > > Key: HDFS-13499 > URL: https://issues.apache.org/jira/browse/HDFS-13499 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Minor > Attachments: HDFS-13499.000.patch > > > HDFS-13484 exposes the disabled name services. This JIRA should show them in > the Web UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13499) RBF: Show disabled name services in the UI
[ https://issues.apache.org/jira/browse/HDFS-13499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-13499: --- Status: Patch Available (was: Open) > RBF: Show disabled name services in the UI > -- > > Key: HDFS-13499 > URL: https://issues.apache.org/jira/browse/HDFS-13499 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Minor > Attachments: HDFS-13499.000.patch > > > HDFS-13484 exposes the disabled name services. This JIRA should show them in > the Web UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13504) Ozone: Remove SCM Block DB
Xiaoyu Yao created HDFS-13504: - Summary: Ozone: Remove SCM Block DB Key: HDFS-13504 URL: https://issues.apache.org/jira/browse/HDFS-13504 Project: Hadoop HDFS Issue Type: Improvement Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao The block/key information is maintained by Ozone Master (a.k.a. KSM). This ticket is opened to remove the redundant block db at SCM. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13484) RBF: Disable Nameservices from the federation
[ https://issues.apache.org/jira/browse/HDFS-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452541#comment-16452541 ] Íñigo Goiri commented on HDFS-13484: Thanks [~linyiqun] for reviewing and committing. Moving to HDFS-13499. > RBF: Disable Nameservices from the federation > - > > Key: HDFS-13484 > URL: https://issues.apache.org/jira/browse/HDFS-13484 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 3.0.1 >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Major > Fix For: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.4 > > Attachments: HDFS-13484.000.patch, HDFS-13484.001.patch, > HDFS-13484.002.patch, HDFS-13484.003.patch, HDFS-13484.004.patch, > HDFS-13484.005.patch, HDFS-13484.006.patch, HDFS-13484.007.patch, > HDFS-13484.008.patch, HDFS-13484.009.patch > > > HDFS-13478 introduced the Decommission store. We should disable the access to > decommissioned subclusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13272) DataNodeHttpServer to have configurable HttpServer2 threads
[
https://issues.apache.org/jira/browse/HDFS-13272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452535#comment-16452535
]
Allen Wittenauer commented on HDFS-13272:
-
Increasing timeout won't help. branch-2's hdfs unit tests haven't been able to
complete during the qbt run for a time period measured in months. It's set for
(IIRC) 18 hours. No one really pays much attention to the nightlies, s
The log probably provides some hints as to where to look to fix it:
https://builds.apache.org/job/PreCommit-HDFS-Build/24054/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
> DataNodeHttpServer to have configurable HttpServer2 threads
> ---
>
> Key: HDFS-13272
> URL: https://issues.apache.org/jira/browse/HDFS-13272
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HDFS-13272-branch-2.000.patch,
> HDFS-13272-branch-2.001.patch
>
>
> In HDFS-7279, the Jetty server on the DataNode was hard-coded to use 10
> threads. In addition to the possibility of this being too few threads, it is
> much higher than necessary in resource constrained environments such as
> MiniDFSCluster. To avoid compatibility issues, rather than using
> {{HttpServer2#HTTP_MAX_THREADS}} directly, we can introduce a new
> configuration for the DataNode's thread pool size.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13272) DataNodeHttpServer to have configurable HttpServer2 threads
[
https://issues.apache.org/jira/browse/HDFS-13272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452502#comment-16452502
]
Erik Krogen commented on HDFS-13272:
Looks like the branch-2 precommit tests are just timing out:
https://builds.apache.org/job/PreCommit-HDFS-Build/24054/ ,
https://builds.apache.org/job/PreCommit-HDFS-Build/24053/. I'm not sure what to
do about it. Increase test timeouts? [~aw] do you have any advice?
> DataNodeHttpServer to have configurable HttpServer2 threads
> ---
>
> Key: HDFS-13272
> URL: https://issues.apache.org/jira/browse/HDFS-13272
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HDFS-13272-branch-2.000.patch,
> HDFS-13272-branch-2.001.patch
>
>
> In HDFS-7279, the Jetty server on the DataNode was hard-coded to use 10
> threads. In addition to the possibility of this being too few threads, it is
> much higher than necessary in resource constrained environments such as
> MiniDFSCluster. To avoid compatibility issues, rather than using
> {{HttpServer2#HTTP_MAX_THREADS}} directly, we can introduce a new
> configuration for the DataNode's thread pool size.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13369) FSCK Report broken with RequestHedgingProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-13369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452491#comment-16452491
]
Ranith Sardar commented on HDFS-13369:
--
Thank you [~elgoiri].
I have changed the patch, according to trunk code and your suggestion.
[^HDFS-13369.004.patch]
> FSCK Report broken with RequestHedgingProxyProvider
>
>
> Key: HDFS-13369
> URL: https://issues.apache.org/jira/browse/HDFS-13369
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 2.8.3
>Reporter: Harshakiran Reddy
>Assignee: Ranith Sardar
>Priority: Major
> Attachments: HDFS-13369.001.patch, HDFS-13369.002.patch,
> HDFS-13369.003.patch, HDFS-13369.004.patch
>
>
> Scenario:-
> 1.Configure the RequestHedgingProxy
> 2. write some files in file system
> 3. Take FSCK report for the above files
>
> {noformat}
> bin> hdfs fsck /file1 -locations -files -blocks
> Exception in thread "main" java.lang.ClassCastException:
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler
> cannot be cast to org.apache.hadoop.ipc.RpcInvocationHandler
> at org.apache.hadoop.ipc.RPC.getConnectionIdForProxy(RPC.java:626)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.getConnectionId(RetryInvocationHandler.java:438)
> at org.apache.hadoop.ipc.RPC.getConnectionIdForProxy(RPC.java:628)
> at org.apache.hadoop.ipc.RPC.getServerAddress(RPC.java:611)
> at org.apache.hadoop.hdfs.HAUtil.getAddressOfActive(HAUtil.java:263)
> at
> org.apache.hadoop.hdfs.tools.DFSck.getCurrentNamenodeAddress(DFSck.java:257)
> at org.apache.hadoop.hdfs.tools.DFSck.doWork(DFSck.java:319)
> at org.apache.hadoop.hdfs.tools.DFSck.access$000(DFSck.java:72)
> at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:156)
> at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.hdfs.tools.DFSck.run(DFSck.java:152)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.hdfs.tools.DFSck.main(DFSck.java:385){noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13369) FSCK Report broken with RequestHedgingProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-13369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ranith Sardar updated HDFS-13369:
-
Attachment: HDFS-13369.004.patch
> FSCK Report broken with RequestHedgingProxyProvider
>
>
> Key: HDFS-13369
> URL: https://issues.apache.org/jira/browse/HDFS-13369
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 2.8.3
>Reporter: Harshakiran Reddy
>Assignee: Ranith Sardar
>Priority: Major
> Attachments: HDFS-13369.001.patch, HDFS-13369.002.patch,
> HDFS-13369.003.patch, HDFS-13369.004.patch
>
>
> Scenario:-
> 1.Configure the RequestHedgingProxy
> 2. write some files in file system
> 3. Take FSCK report for the above files
>
> {noformat}
> bin> hdfs fsck /file1 -locations -files -blocks
> Exception in thread "main" java.lang.ClassCastException:
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler
> cannot be cast to org.apache.hadoop.ipc.RpcInvocationHandler
> at org.apache.hadoop.ipc.RPC.getConnectionIdForProxy(RPC.java:626)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.getConnectionId(RetryInvocationHandler.java:438)
> at org.apache.hadoop.ipc.RPC.getConnectionIdForProxy(RPC.java:628)
> at org.apache.hadoop.ipc.RPC.getServerAddress(RPC.java:611)
> at org.apache.hadoop.hdfs.HAUtil.getAddressOfActive(HAUtil.java:263)
> at
> org.apache.hadoop.hdfs.tools.DFSck.getCurrentNamenodeAddress(DFSck.java:257)
> at org.apache.hadoop.hdfs.tools.DFSck.doWork(DFSck.java:319)
> at org.apache.hadoop.hdfs.tools.DFSck.access$000(DFSck.java:72)
> at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:156)
> at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.hdfs.tools.DFSck.run(DFSck.java:152)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.hdfs.tools.DFSck.main(DFSck.java:385){noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13448) HDFS Block Placement - Ignore Locality for First Block Replica
[
https://issues.apache.org/jira/browse/HDFS-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452179#comment-16452179
]
BELUGA BEHR commented on HDFS-13448:
Hello,
Is anyone able to provide additional feedback on this patch? This is an
important one to my customer.
Thanks.
> HDFS Block Placement - Ignore Locality for First Block Replica
> --
>
> Key: HDFS-13448
> URL: https://issues.apache.org/jira/browse/HDFS-13448
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: block placement, hdfs-client
>Affects Versions: 2.9.0, 3.0.1
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HDFS-13448.1.patch, HDFS-13448.2.patch,
> HDFS-13448.3.patch, HDFS-13448.4.patch, HDFS-13448.5.patch
>
>
> According to the HDFS Block Place Rules:
> {quote}
> /**
> * The replica placement strategy is that if the writer is on a datanode,
> * the 1st replica is placed on the local machine,
> * otherwise a random datanode. The 2nd replica is placed on a datanode
> * that is on a different rack. The 3rd replica is placed on a datanode
> * which is on a different node of the rack as the second replica.
> */
> {quote}
> However, there is a hint for the hdfs-client that allows the block placement
> request to not put a block replica on the local datanode _where 'local' means
> the same host as the client is being run on._
> {quote}
> /**
>* Advise that a block replica NOT be written to the local DataNode where
>* 'local' means the same host as the client is being run on.
>*
>* @see CreateFlag#NO_LOCAL_WRITE
>*/
> {quote}
> I propose that we add a new flag that allows the hdfs-client to request that
> the first block replica be placed on a random DataNode in the cluster. The
> subsequent block replicas should follow the normal block placement rules.
> The issue is that when the {{NO_LOCAL_WRITE}} is enabled, the first block
> replica is not placed on the local node, but it is still placed on the local
> rack. Where this comes into play is where you have, for example, a flume
> agent that is loading data into HDFS.
> If the Flume agent is running on a DataNode, then by default, the DataNode
> local to the Flume agent will always get the first block replica and this
> leads to un-even block placements, with the local node always filling up
> faster than any other node in the cluster.
> Modifying this example, if the DataNode is removed from the host where the
> Flume agent is running, or this {{NO_LOCAL_WRITE}} is enabled by Flume, then
> the default block placement policy will still prefer the local rack. This
> remedies the situation only so far as now the first block replica will always
> be distributed to a DataNode on the local rack.
> This new flag would allow a single Flume agent to distribute the blocks
> randomly, evenly, over the entire cluster instead of hot-spotting the local
> node or the local rack.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13489) Get base snapshotable path if exists for a given path
[ https://issues.apache.org/jira/browse/HDFS-13489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451990#comment-16451990 ] Shashikant Banerjee commented on HDFS-13489: Thanks [~hpatro], for updating the patch. Patch v2 looks good to me. Please address the checkStyle issues . > Get base snapshotable path if exists for a given path > - > > Key: HDFS-13489 > URL: https://issues.apache.org/jira/browse/HDFS-13489 > Project: Hadoop HDFS > Issue Type: Task > Components: hdfs >Reporter: Harkrishn Patro >Assignee: Harkrishn Patro >Priority: Major > Attachments: HDFS-13489.001.patch, HDFS-13489.002.patch > > > Currently, hdfs only lists the snapshotable paths in the filesystem. This > feature would add the functionality of figuring out if a given path is > snapshotable or not. If yes, it would return the base snapshotable path. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13499) RBF: Show disabled name services in the UI
[ https://issues.apache.org/jira/browse/HDFS-13499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451827#comment-16451827 ] Yiqun Lin commented on HDFS-13499: -- [~elgoiri], as HDFS-13484 was resolved, we can make the final adjustment for UI and please attach the screen-shot as well. > RBF: Show disabled name services in the UI > -- > > Key: HDFS-13499 > URL: https://issues.apache.org/jira/browse/HDFS-13499 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Minor > > HDFS-13484 exposes the disabled name services. This JIRA should show them in > the Web UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13484) RBF: Disable Nameservices from the federation
[
https://issues.apache.org/jira/browse/HDFS-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451815#comment-16451815
]
Yiqun Lin edited comment on HDFS-13484 at 4/25/18 7:50 AM:
---
LGTM, +1.
Committed this to trunk, branch-3.1, branch-3.0, branch-2 and branch-2.9.
I committed the final patch with redundant ‘|’ removed in following line since
this will cause render page error.
{noformat}
| `-nameservice` `disable` | `enable` *nameservice* | Disable/enable a name
service from the federation. If disabled, requests will not go to that name
service. |
{noformat}
Attach the final patch.
Thanks [~elgoiri] for the contribution.
was (Author: linyiqun):
Committed this to trunk, branch-3.1, branch-3.0, branch-2 and branch-2.9.
I committed the final patch with redundant ‘|’ removed in following line since
this will cause render page error.
{noformat}
| `-nameservice` `disable` | `enable` *nameservice* | Disable/enable a name
service from the federation. If disabled, requests will not go to that name
service. |
{noformat}
Attach the final patch.
Thanks [~elgoiri] for the contribution.
> RBF: Disable Nameservices from the federation
> -
>
> Key: HDFS-13484
> URL: https://issues.apache.org/jira/browse/HDFS-13484
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Affects Versions: 3.0.1
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
>Priority: Major
> Fix For: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.4
>
> Attachments: HDFS-13484.000.patch, HDFS-13484.001.patch,
> HDFS-13484.002.patch, HDFS-13484.003.patch, HDFS-13484.004.patch,
> HDFS-13484.005.patch, HDFS-13484.006.patch, HDFS-13484.007.patch,
> HDFS-13484.008.patch, HDFS-13484.009.patch
>
>
> HDFS-13478 introduced the Decommission store. We should disable the access to
> decommissioned subclusters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13484) RBF: Disable Nameservices from the federation
[ https://issues.apache.org/jira/browse/HDFS-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-13484: - Attachment: HDFS-13484.009.patch > RBF: Disable Nameservices from the federation > - > > Key: HDFS-13484 > URL: https://issues.apache.org/jira/browse/HDFS-13484 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 3.0.1 >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Major > Fix For: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.4 > > Attachments: HDFS-13484.000.patch, HDFS-13484.001.patch, > HDFS-13484.002.patch, HDFS-13484.003.patch, HDFS-13484.004.patch, > HDFS-13484.005.patch, HDFS-13484.006.patch, HDFS-13484.007.patch, > HDFS-13484.008.patch, HDFS-13484.009.patch > > > HDFS-13478 introduced the Decommission store. We should disable the access to > decommissioned subclusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13484) RBF: Disable Nameservices from the federation
[ https://issues.apache.org/jira/browse/HDFS-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-13484: - Affects Version/s: 3.0.1 Target Version/s: 3.2.0, 3.1.1 > RBF: Disable Nameservices from the federation > - > > Key: HDFS-13484 > URL: https://issues.apache.org/jira/browse/HDFS-13484 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 3.0.1 >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Major > Fix For: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.4 > > Attachments: HDFS-13484.000.patch, HDFS-13484.001.patch, > HDFS-13484.002.patch, HDFS-13484.003.patch, HDFS-13484.004.patch, > HDFS-13484.005.patch, HDFS-13484.006.patch, HDFS-13484.007.patch, > HDFS-13484.008.patch > > > HDFS-13478 introduced the Decommission store. We should disable the access to > decommissioned subclusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13484) RBF: Disable Nameservices from the federation
[
https://issues.apache.org/jira/browse/HDFS-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-13484:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.0.4
2.9.2
3.1.1
3.2.0
2.10.0
Status: Resolved (was: Patch Available)
Committed this to trunk, branch-3.1, branch-3.0, branch-2 and branch-2.9.
I committed the final patch with redundant ‘|’ removed in following line since
this will cause render page error.
{noformat}
| `-nameservice` `disable` | `enable` *nameservice* | Disable/enable a name
service from the federation. If disabled, requests will not go to that name
service. |
{noformat}
Attach the final patch.
Thanks [~elgoiri] for the contribution.
> RBF: Disable Nameservices from the federation
> -
>
> Key: HDFS-13484
> URL: https://issues.apache.org/jira/browse/HDFS-13484
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Affects Versions: 3.0.1
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
>Priority: Major
> Fix For: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.4
>
> Attachments: HDFS-13484.000.patch, HDFS-13484.001.patch,
> HDFS-13484.002.patch, HDFS-13484.003.patch, HDFS-13484.004.patch,
> HDFS-13484.005.patch, HDFS-13484.006.patch, HDFS-13484.007.patch,
> HDFS-13484.008.patch
>
>
> HDFS-13478 introduced the Decommission store. We should disable the access to
> decommissioned subclusters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13484) RBF: Disable Nameservices from the federation
[ https://issues.apache.org/jira/browse/HDFS-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451814#comment-16451814 ] Hudson commented on HDFS-13484: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14060 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14060/]) HDFS-13484. RBF: Disable Nameservices from the federation. Contributed (yqlin: rev 30fef0bf1e5c8c0ca073df99ad9b33cb0e4431a5) * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterPermissionChecker.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/FederationTestUtils.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/MiniRouterDFSCluster.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterAdminServer.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcServer.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterAdmin.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/site/markdown/HDFSRouterFederation.md * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/FederationMetrics.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/FederationNamenodeServiceState.java * (add) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestDisableNameservices.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/ActiveNamenodeResolver.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/MockResolver.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterAdminCLI.java * (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/MembershipNamenodeResolver.java > RBF: Disable Nameservices from the federation > - > > Key: HDFS-13484 > URL: https://issues.apache.org/jira/browse/HDFS-13484 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Major > Attachments: HDFS-13484.000.patch, HDFS-13484.001.patch, > HDFS-13484.002.patch, HDFS-13484.003.patch, HDFS-13484.004.patch, > HDFS-13484.005.patch, HDFS-13484.006.patch, HDFS-13484.007.patch, > HDFS-13484.008.patch > > > HDFS-13478 introduced the Decommission store. We should disable the access to > decommissioned subclusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13165) [SPS]: Collects successfully moved block details via IBR
[
https://issues.apache.org/jira/browse/HDFS-13165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451722#comment-16451722
]
Rakesh R commented on HDFS-13165:
-
Based on [~surendrasingh]'s vote and I believe no objection on the IBR related
changes, am planning to commit the latest IBR patch(without datanode block move
changes) to the branch. I'd like to finish the bulk changes amap and
simultaneously will try to reach consensus on the datanode protocol part.
> [SPS]: Collects successfully moved block details via IBR
>
>
> Key: HDFS-13165
> URL: https://issues.apache.org/jira/browse/HDFS-13165
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: Rakesh R
>Priority: Major
> Attachments: HDFS-13165-HDFS-10285-00.patch,
> HDFS-13165-HDFS-10285-01.patch, HDFS-13165-HDFS-10285-02.patch,
> HDFS-13165-HDFS-10285-03.patch, HDFS-13165-HDFS-10285-04.patch,
> HDFS-13165-HDFS-10285-05.patch, HDFS-13165-HDFS-10285-06.patch,
> HDFS-13165-HDFS-10285-07.patch, HDFS-13165-HDFS-10285-08.patch,
> HDFS-13165-HDFS-10285-09.patch
>
>
> This task to make use of the existing IBR to get moved block details and
> remove unwanted future tracking logic exists in BlockStorageMovementTracker
> code, this is no more needed as the file level tracking maintained at NN
> itself.
> Following comments taken from HDFS-10285,
> [here|https://issues.apache.org/jira/browse/HDFS-10285?focusedCommentId=16347472=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16347472]
> Comment-3)
> {quote}BPServiceActor
> Is it actually sending back the moved blocks? Aren’t IBRs sufficient?{quote}
> Comment-21)
> {quote}
> BlockStorageMovementTracker
> Many data structures are riddled with non-threadsafe race conditions and risk
> of CMEs.
> Ex. The moverTaskFutures map. Adding new blocks and/or adding to a block's
> list of futures is synchronized. However the run loop does an unsynchronized
> block get, unsynchronized future remove, unsynchronized isEmpty, possibly
> another unsynchronized get, only then does it do a synchronized remove of the
> block. The whole chunk of code should be synchronized.
> Is the problematic moverTaskFutures even needed? It's aggregating futures
> per-block for seemingly no reason. Why track all the futures at all instead
> of just relying on the completion service? As best I can tell:
> It's only used to determine if a future from the completion service should be
> ignored during shutdown. Shutdown sets the running boolean to false and
> clears the entire datastructure so why not use the running boolean like a
> check just a little further down?
> As synchronization to sleep up to 2 seconds before performing a blocking
> moverCompletionService.take, but only when it thinks there are no active
> futures. I'll ignore the missed notify race that the bounded wait masks, but
> the real question is why not just do the blocking take?
> Why all the complexity? Am I missing something?
> BlocksMovementsStatusHandler
> Suffers same type of thread safety issues as StoragePolicySatisfyWorker. Ex.
> blockIdVsMovementStatus is inconsistent synchronized. Does synchronize to
> return an unmodifiable list which sadly does nothing to protect the caller
> from CME.
> handle is iterating over a non-thread safe list.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
