[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17269570#comment-17269570
]
Jim Brennan commented on HDFS-13723:
[~ahussein], I pulled this back to branch-2.10.
> Occasional "Should be different group" error in
> TestRefreshUserMappings#testGroupMappingRefresh
> ---
>
> Key: HDFS-13723
> URL: https://issues.apache.org/jira/browse/HDFS-13723
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security, test
>Affects Versions: 3.0.0
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Fix For: 3.2.0, 3.1.1, 3.0.4
>
> Attachments: HDFS-13723.001.patch, HDFS-13723.002.patch,
> HDFS-13723.003.patch
>
>
> In some occasions, the user-group mapping refresh timeout test assertion
> would fail due to the mapping didn't refresh in time, reporting "Should be
> different group".
>
> Trace:
> {code:java}
> java.lang.AssertionError: Should be different group
> at
> org.apache.hadoop.security.TestRefreshUserMappings.testGroupMappingRefresh(TestRefreshUserMappings.java:153)
> :
> :
> 2018-07-04 19:35:21,073 [BP-1412052829-172.26.17.254-1530758120647
> heartbeating to localhost/127.0.0.1:39524] INFO datanode.DataNode
> (BPOfferService.java:processCommandFromActive(759)) - Got finalize command
> for block pool BP-1412052829-172.26.17.254-1530758120647
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,090 [IPC Server handler 6 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,092 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> 2018-07-04 19:35:21,095 [IPC Server handler 7 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,096 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> first attempt:
> [jenkins11, jenkins12]
> second attempt, should be same:
> [jenkins11, jenkins12]
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO
> namenode.NameNode (NameNodeRpcServer.java:refreshUserToGroupsMappings(1648))
> - Refreshing all user-to-groups mappings. Requested by user: jenkins
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO security.Groups
> (Groups.java:refresh(401)) - clearing userToGroupsMap cache
> Refreshing groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,102 [IPC Server handler 5 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1
> cmd=refreshUserToGroupsMappings src=nulldst=nullperm=null
> proto=rpc
> Refresh user to groups mapping successful
> third attempt(after refresh command), should be different:
> Getting groups in MockUnixGroupsMapping
> [jenkins21, jenkins22]
> fourth attempt(after timeout), should be different:
> [jenkins21, jenkins22]
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:22,204 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(1965)) - Shutting down the Mini HDFS Cluster
> {code}
>
> Solution:
> Increase the timeout slightly, and place debugging message in load() and
> reload() methods in class GroupCacheLoader.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17269473#comment-17269473
]
Ahmed Hussein commented on HDFS-13723:
--
[~smeng], [~Jim_Brennan] can you please pull this into 2.10? The cherry-pick
[162228e|https://github.com/apache/hadoop/commit/162228e8db937d4bdb9cf61d15ed555f1c96368f]
should be clean.
> Occasional "Should be different group" error in
> TestRefreshUserMappings#testGroupMappingRefresh
> ---
>
> Key: HDFS-13723
> URL: https://issues.apache.org/jira/browse/HDFS-13723
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security, test
>Affects Versions: 3.0.0
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Fix For: 3.2.0, 3.1.1, 3.0.4
>
> Attachments: HDFS-13723.001.patch, HDFS-13723.002.patch,
> HDFS-13723.003.patch
>
>
> In some occasions, the user-group mapping refresh timeout test assertion
> would fail due to the mapping didn't refresh in time, reporting "Should be
> different group".
>
> Trace:
> {code:java}
> java.lang.AssertionError: Should be different group
> at
> org.apache.hadoop.security.TestRefreshUserMappings.testGroupMappingRefresh(TestRefreshUserMappings.java:153)
> :
> :
> 2018-07-04 19:35:21,073 [BP-1412052829-172.26.17.254-1530758120647
> heartbeating to localhost/127.0.0.1:39524] INFO datanode.DataNode
> (BPOfferService.java:processCommandFromActive(759)) - Got finalize command
> for block pool BP-1412052829-172.26.17.254-1530758120647
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,090 [IPC Server handler 6 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,092 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> 2018-07-04 19:35:21,095 [IPC Server handler 7 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,096 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> first attempt:
> [jenkins11, jenkins12]
> second attempt, should be same:
> [jenkins11, jenkins12]
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO
> namenode.NameNode (NameNodeRpcServer.java:refreshUserToGroupsMappings(1648))
> - Refreshing all user-to-groups mappings. Requested by user: jenkins
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO security.Groups
> (Groups.java:refresh(401)) - clearing userToGroupsMap cache
> Refreshing groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,102 [IPC Server handler 5 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1
> cmd=refreshUserToGroupsMappings src=nulldst=nullperm=null
> proto=rpc
> Refresh user to groups mapping successful
> third attempt(after refresh command), should be different:
> Getting groups in MockUnixGroupsMapping
> [jenkins21, jenkins22]
> fourth attempt(after timeout), should be different:
> [jenkins21, jenkins22]
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:22,204 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(1965)) - Shutting down the Mini HDFS Cluster
> {code}
>
> Solution:
> Increase the timeout slightly, and place debugging message in load() and
> reload() methods in class GroupCacheLoader.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540386#comment-16540386
]
Wei-Chiu Chuang commented on HDFS-13723:
and branch-3.1, branch-3.0 as well.
> Occasional "Should be different group" error in
> TestRefreshUserMappings#testGroupMappingRefresh
> ---
>
> Key: HDFS-13723
> URL: https://issues.apache.org/jira/browse/HDFS-13723
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security, test
>Affects Versions: 3.0.0
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Fix For: 3.2.0, 3.1.1, 3.0.4
>
> Attachments: HDFS-13723.001.patch, HDFS-13723.002.patch,
> HDFS-13723.003.patch
>
>
> In some occasions, the user-group mapping refresh timeout test assertion
> would fail due to the mapping didn't refresh in time, reporting "Should be
> different group".
>
> Trace:
> {code:java}
> java.lang.AssertionError: Should be different group
> at
> org.apache.hadoop.security.TestRefreshUserMappings.testGroupMappingRefresh(TestRefreshUserMappings.java:153)
> :
> :
> 2018-07-04 19:35:21,073 [BP-1412052829-172.26.17.254-1530758120647
> heartbeating to localhost/127.0.0.1:39524] INFO datanode.DataNode
> (BPOfferService.java:processCommandFromActive(759)) - Got finalize command
> for block pool BP-1412052829-172.26.17.254-1530758120647
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,090 [IPC Server handler 6 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,092 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> 2018-07-04 19:35:21,095 [IPC Server handler 7 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,096 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> first attempt:
> [jenkins11, jenkins12]
> second attempt, should be same:
> [jenkins11, jenkins12]
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO
> namenode.NameNode (NameNodeRpcServer.java:refreshUserToGroupsMappings(1648))
> - Refreshing all user-to-groups mappings. Requested by user: jenkins
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO security.Groups
> (Groups.java:refresh(401)) - clearing userToGroupsMap cache
> Refreshing groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,102 [IPC Server handler 5 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1
> cmd=refreshUserToGroupsMappings src=nulldst=nullperm=null
> proto=rpc
> Refresh user to groups mapping successful
> third attempt(after refresh command), should be different:
> Getting groups in MockUnixGroupsMapping
> [jenkins21, jenkins22]
> fourth attempt(after timeout), should be different:
> [jenkins21, jenkins22]
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:22,204 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(1965)) - Shutting down the Mini HDFS Cluster
> {code}
>
> Solution:
> Increase the timeout slightly, and place debugging message in load() and
> reload() methods in class GroupCacheLoader.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16536874#comment-16536874
]
Wei-Chiu Chuang commented on HDFS-13723:
+1 failed tests are unrelated.
> Occasional "Should be different group" error in
> TestRefreshUserMappings#testGroupMappingRefresh
> ---
>
> Key: HDFS-13723
> URL: https://issues.apache.org/jira/browse/HDFS-13723
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security, test
>Affects Versions: 3.0.0
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HDFS-13723.001.patch, HDFS-13723.002.patch,
> HDFS-13723.003.patch
>
>
> In some occasions, the user-group mapping refresh timeout test assertion
> would fail due to the mapping didn't refresh in time, reporting "Should be
> different group".
>
> Trace:
> {code:java}
> java.lang.AssertionError: Should be different group
> at
> org.apache.hadoop.security.TestRefreshUserMappings.testGroupMappingRefresh(TestRefreshUserMappings.java:153)
> :
> :
> 2018-07-04 19:35:21,073 [BP-1412052829-172.26.17.254-1530758120647
> heartbeating to localhost/127.0.0.1:39524] INFO datanode.DataNode
> (BPOfferService.java:processCommandFromActive(759)) - Got finalize command
> for block pool BP-1412052829-172.26.17.254-1530758120647
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,090 [IPC Server handler 6 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,092 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> 2018-07-04 19:35:21,095 [IPC Server handler 7 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,096 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> first attempt:
> [jenkins11, jenkins12]
> second attempt, should be same:
> [jenkins11, jenkins12]
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO
> namenode.NameNode (NameNodeRpcServer.java:refreshUserToGroupsMappings(1648))
> - Refreshing all user-to-groups mappings. Requested by user: jenkins
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO security.Groups
> (Groups.java:refresh(401)) - clearing userToGroupsMap cache
> Refreshing groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,102 [IPC Server handler 5 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1
> cmd=refreshUserToGroupsMappings src=nulldst=nullperm=null
> proto=rpc
> Refresh user to groups mapping successful
> third attempt(after refresh command), should be different:
> Getting groups in MockUnixGroupsMapping
> [jenkins21, jenkins22]
> fourth attempt(after timeout), should be different:
> [jenkins21, jenkins22]
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:22,204 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(1965)) - Shutting down the Mini HDFS Cluster
> {code}
>
> Solution:
> Increase the timeout slightly, and place debugging message in load() and
> reload() methods in class GroupCacheLoader.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535572#comment-16535572
]
genericqa commented on HDFS-13723:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 39s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
52s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 26m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 0s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
40s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 83m 53s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
0s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}217m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
| | hadoop.hdfs.client.impl.TestBlockReaderLocal |
| | hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HDFS-13723 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12930608/HDFS-13723.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 7fb7d9251d8e 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk /
[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535440#comment-16535440
]
Siyao Meng commented on HDFS-13723:
---
In [^HDFS-13723.003.patch]:
I have replaced
{code:java}
import org.apache.log4j.Level{code}
with
{code:java}
import org.slf4j.event.Level{code}
to get rid of "deprecation" warning.
> Occasional "Should be different group" error in
> TestRefreshUserMappings#testGroupMappingRefresh
> ---
>
> Key: HDFS-13723
> URL: https://issues.apache.org/jira/browse/HDFS-13723
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security, test
>Affects Versions: 3.0.0
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HDFS-13723.001.patch, HDFS-13723.002.patch,
> HDFS-13723.003.patch
>
>
> In some occasions, the user-group mapping refresh timeout test assertion
> would fail due to the mapping didn't refresh in time, reporting "Should be
> different group".
>
> Trace:
> {code:java}
> java.lang.AssertionError: Should be different group
> at
> org.apache.hadoop.security.TestRefreshUserMappings.testGroupMappingRefresh(TestRefreshUserMappings.java:153)
> :
> :
> 2018-07-04 19:35:21,073 [BP-1412052829-172.26.17.254-1530758120647
> heartbeating to localhost/127.0.0.1:39524] INFO datanode.DataNode
> (BPOfferService.java:processCommandFromActive(759)) - Got finalize command
> for block pool BP-1412052829-172.26.17.254-1530758120647
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,090 [IPC Server handler 6 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,092 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> 2018-07-04 19:35:21,095 [IPC Server handler 7 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,096 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> first attempt:
> [jenkins11, jenkins12]
> second attempt, should be same:
> [jenkins11, jenkins12]
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO
> namenode.NameNode (NameNodeRpcServer.java:refreshUserToGroupsMappings(1648))
> - Refreshing all user-to-groups mappings. Requested by user: jenkins
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO security.Groups
> (Groups.java:refresh(401)) - clearing userToGroupsMap cache
> Refreshing groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,102 [IPC Server handler 5 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1
> cmd=refreshUserToGroupsMappings src=nulldst=nullperm=null
> proto=rpc
> Refresh user to groups mapping successful
> third attempt(after refresh command), should be different:
> Getting groups in MockUnixGroupsMapping
> [jenkins21, jenkins22]
> fourth attempt(after timeout), should be different:
> [jenkins21, jenkins22]
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:22,204 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(1965)) - Shutting down the Mini HDFS Cluster
> {code}
>
> Solution:
> Increase the timeout slightly, and place debugging message in load() and
> reload() methods in class GroupCacheLoader.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535441#comment-16535441
]
Wei-Chiu Chuang commented on HDFS-13723:
LGTM +1 pending Jenkins.
> Occasional "Should be different group" error in
> TestRefreshUserMappings#testGroupMappingRefresh
> ---
>
> Key: HDFS-13723
> URL: https://issues.apache.org/jira/browse/HDFS-13723
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security, test
>Affects Versions: 3.0.0
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HDFS-13723.001.patch, HDFS-13723.002.patch,
> HDFS-13723.003.patch
>
>
> In some occasions, the user-group mapping refresh timeout test assertion
> would fail due to the mapping didn't refresh in time, reporting "Should be
> different group".
>
> Trace:
> {code:java}
> java.lang.AssertionError: Should be different group
> at
> org.apache.hadoop.security.TestRefreshUserMappings.testGroupMappingRefresh(TestRefreshUserMappings.java:153)
> :
> :
> 2018-07-04 19:35:21,073 [BP-1412052829-172.26.17.254-1530758120647
> heartbeating to localhost/127.0.0.1:39524] INFO datanode.DataNode
> (BPOfferService.java:processCommandFromActive(759)) - Got finalize command
> for block pool BP-1412052829-172.26.17.254-1530758120647
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,090 [IPC Server handler 6 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,092 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> 2018-07-04 19:35:21,095 [IPC Server handler 7 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,096 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> first attempt:
> [jenkins11, jenkins12]
> second attempt, should be same:
> [jenkins11, jenkins12]
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO
> namenode.NameNode (NameNodeRpcServer.java:refreshUserToGroupsMappings(1648))
> - Refreshing all user-to-groups mappings. Requested by user: jenkins
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO security.Groups
> (Groups.java:refresh(401)) - clearing userToGroupsMap cache
> Refreshing groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,102 [IPC Server handler 5 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1
> cmd=refreshUserToGroupsMappings src=nulldst=nullperm=null
> proto=rpc
> Refresh user to groups mapping successful
> third attempt(after refresh command), should be different:
> Getting groups in MockUnixGroupsMapping
> [jenkins21, jenkins22]
> fourth attempt(after timeout), should be different:
> [jenkins21, jenkins22]
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:22,204 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(1965)) - Shutting down the Mini HDFS Cluster
> {code}
>
> Solution:
> Increase the timeout slightly, and place debugging message in load() and
> reload() methods in class GroupCacheLoader.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535422#comment-16535422
]
Siyao Meng commented on HDFS-13723:
---
[~jojochuang] Thanks for your comment! I have updated the patch to add the
annotation.
> Occasional "Should be different group" error in
> TestRefreshUserMappings#testGroupMappingRefresh
> ---
>
> Key: HDFS-13723
> URL: https://issues.apache.org/jira/browse/HDFS-13723
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security, test
>Affects Versions: 3.0.0
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HDFS-13723.001.patch, HDFS-13723.002.patch
>
>
> In some occasions, the user-group mapping refresh timeout test assertion
> would fail due to the mapping didn't refresh in time, reporting "Should be
> different group".
>
> Trace:
> {code:java}
> java.lang.AssertionError: Should be different group
> at
> org.apache.hadoop.security.TestRefreshUserMappings.testGroupMappingRefresh(TestRefreshUserMappings.java:153)
> :
> :
> 2018-07-04 19:35:21,073 [BP-1412052829-172.26.17.254-1530758120647
> heartbeating to localhost/127.0.0.1:39524] INFO datanode.DataNode
> (BPOfferService.java:processCommandFromActive(759)) - Got finalize command
> for block pool BP-1412052829-172.26.17.254-1530758120647
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,090 [IPC Server handler 6 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,092 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> 2018-07-04 19:35:21,095 [IPC Server handler 7 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,096 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> first attempt:
> [jenkins11, jenkins12]
> second attempt, should be same:
> [jenkins11, jenkins12]
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO
> namenode.NameNode (NameNodeRpcServer.java:refreshUserToGroupsMappings(1648))
> - Refreshing all user-to-groups mappings. Requested by user: jenkins
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO security.Groups
> (Groups.java:refresh(401)) - clearing userToGroupsMap cache
> Refreshing groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,102 [IPC Server handler 5 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1
> cmd=refreshUserToGroupsMappings src=nulldst=nullperm=null
> proto=rpc
> Refresh user to groups mapping successful
> third attempt(after refresh command), should be different:
> Getting groups in MockUnixGroupsMapping
> [jenkins21, jenkins22]
> fourth attempt(after timeout), should be different:
> [jenkins21, jenkins22]
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:22,204 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(1965)) - Shutting down the Mini HDFS Cluster
> {code}
>
> Solution:
> Increase the timeout slightly, and place debugging message in load() and
> reload() methods in class GroupCacheLoader.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535114#comment-16535114
]
Siyao Meng commented on HDFS-13723:
---
[~genericqa] Unrelated flaky test
+hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints+. Passed locally. Can be
ignored.
> Occasional "Should be different group" error in
> TestRefreshUserMappings#testGroupMappingRefresh
> ---
>
> Key: HDFS-13723
> URL: https://issues.apache.org/jira/browse/HDFS-13723
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security, test
>Affects Versions: 3.0.0
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HDFS-13723.001.patch
>
>
> In some occasions, the user-group mapping refresh timeout test assertion
> would fail due to the mapping didn't refresh in time, reporting "Should be
> different group".
>
> Trace:
> {code:java}
> java.lang.AssertionError: Should be different group
> at
> org.apache.hadoop.security.TestRefreshUserMappings.testGroupMappingRefresh(TestRefreshUserMappings.java:153)
> :
> :
> 2018-07-04 19:35:21,073 [BP-1412052829-172.26.17.254-1530758120647
> heartbeating to localhost/127.0.0.1:39524] INFO datanode.DataNode
> (BPOfferService.java:processCommandFromActive(759)) - Got finalize command
> for block pool BP-1412052829-172.26.17.254-1530758120647
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,090 [IPC Server handler 6 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,092 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> 2018-07-04 19:35:21,095 [IPC Server handler 7 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,096 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> first attempt:
> [jenkins11, jenkins12]
> second attempt, should be same:
> [jenkins11, jenkins12]
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO
> namenode.NameNode (NameNodeRpcServer.java:refreshUserToGroupsMappings(1648))
> - Refreshing all user-to-groups mappings. Requested by user: jenkins
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO security.Groups
> (Groups.java:refresh(401)) - clearing userToGroupsMap cache
> Refreshing groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,102 [IPC Server handler 5 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1
> cmd=refreshUserToGroupsMappings src=nulldst=nullperm=null
> proto=rpc
> Refresh user to groups mapping successful
> third attempt(after refresh command), should be different:
> Getting groups in MockUnixGroupsMapping
> [jenkins21, jenkins22]
> fourth attempt(after timeout), should be different:
> [jenkins21, jenkins22]
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:22,204 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(1965)) - Shutting down the Mini HDFS Cluster
> {code}
>
> Solution:
> Increase the timeout slightly, and place debugging message in load() and
> reload() methods in class GroupCacheLoader.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534404#comment-16534404
]
genericqa commented on HDFS-13723:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 30s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
58s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
52s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 29m 10s{color}
| {color:red} root generated 1 new + 1584 unchanged - 0 fixed = 1585 total (was
1584) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
15s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 9s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
54s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}240m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HDFS-13723 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12930467/HDFS-13723.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux c4ac3d6a4a21 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 39ad989 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| javac |
[jira] [Commented] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh
[
https://issues.apache.org/jira/browse/HDFS-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534299#comment-16534299
]
Wei-Chiu Chuang commented on HDFS-13723:
Thanks [~smeng] for reporting the issue and the patch. Overall makes sense.
Essentially it relaxes the timeout constraint to accommodate potential slowness
in cache reload thread.
Could you add "@VisibleForTesting" annotation above this line?
{code}
static final Logger LOG = LoggerFactory.getLogger(Groups.class);
{code}
This is just to mark that this variable is supposed to be private, and we make
it package private only for testing purposes.
> Occasional "Should be different group" error in
> TestRefreshUserMappings#testGroupMappingRefresh
> ---
>
> Key: HDFS-13723
> URL: https://issues.apache.org/jira/browse/HDFS-13723
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security, test
>Affects Versions: 3.0.0
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HDFS-13723.001.patch
>
>
> In some occasions, the user-group mapping refresh timeout test assertion
> would fail due to the mapping didn't refresh in time, reporting "Should be
> different group".
>
> Trace:
> {code:java}
> java.lang.AssertionError: Should be different group
> at
> org.apache.hadoop.security.TestRefreshUserMappings.testGroupMappingRefresh(TestRefreshUserMappings.java:153)
> :
> :
> 2018-07-04 19:35:21,073 [BP-1412052829-172.26.17.254-1530758120647
> heartbeating to localhost/127.0.0.1:39524] INFO datanode.DataNode
> (BPOfferService.java:processCommandFromActive(759)) - Got finalize command
> for block pool BP-1412052829-172.26.17.254-1530758120647
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,090 [IPC Server handler 6 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,092 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> 2018-07-04 19:35:21,095 [IPC Server handler 7 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1 cmd=datanodeReport
> src=nulldst=nullperm=null proto=rpc
> 2018-07-04 19:35:21,096 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:waitActive(2629)) - Cluster is active
> first attempt:
> [jenkins11, jenkins12]
> second attempt, should be same:
> [jenkins11, jenkins12]
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO
> namenode.NameNode (NameNodeRpcServer.java:refreshUserToGroupsMappings(1648))
> - Refreshing all user-to-groups mappings. Requested by user: jenkins
> 2018-07-04 19:35:21,101 [IPC Server handler 5 on 39524] INFO security.Groups
> (Groups.java:refresh(401)) - clearing userToGroupsMap cache
> Refreshing groups in MockUnixGroupsMapping
> 2018-07-04 19:35:21,102 [IPC Server handler 5 on 39524] INFO
> FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7805)) - allowed=true
>ugi=jenkins (auth:SIMPLE) ip=/127.0.0.1
> cmd=refreshUserToGroupsMappings src=nulldst=nullperm=null
> proto=rpc
> Refresh user to groups mapping successful
> third attempt(after refresh command), should be different:
> Getting groups in MockUnixGroupsMapping
> [jenkins21, jenkins22]
> fourth attempt(after timeout), should be different:
> [jenkins21, jenkins22]
> Getting groups in MockUnixGroupsMapping
> 2018-07-04 19:35:22,204 [main] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(1965)) - Shutting down the Mini HDFS Cluster
> {code}
>
> Solution:
> Increase the timeout slightly, and place debugging message in load() and
> reload() methods in class GroupCacheLoader.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
