wchevreuil edited a comment on pull request #4029:
URL: https://github.com/apache/hbase/pull/4029#issuecomment-1014906494
> Could you please describe your usage more? I do not fully understand wha's
the problem here...
Sure. As mentioned in the jira description, the "if" check below will always
unnecessarily reset static var UserProvider.groups to a newly created instance
of TestingGroups every time `SecureHadoopUser.createUserForTesting` is called.
```
if (!(UserProvider.groups instanceof TestingGroups) ||
conf.getBoolean(TestingGroups.TEST_CONF, false)) {
UserProvider.groups = new TestingGroups(UserProvider.groups);
}
```
That will cause all users but the latest one created in a test to be missed
in the `TestingGroups` cache from `UserProvider.groups`, falling back to the
underlying hadoop `Groups` own group caching. I believe this extra check in
`SecureHadoopUser.createUserForTesting` is redundant. We should only make sure
to wrap the underlying `Groups` implementation once, in the first time we pass
through that code, when testing.
I bumped into this when running
`TestAccessController.testGetUsersPermissions` against a hadoop version that
includes HADOOP-17079. The changes introduced by HADOOP-17079 around several
classes of the hadoop security framework have broken TestAccessController as
below:
```
2022-01-17 17:54:47,648 WARN
[RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=50198]
security.ShellBasedUnixGroupsMapping(218): unable to return groups for user
globalGroupUser1
PartialGroupNameException The user name globalGroupUser1 is not found. id:
globalGroupUser1: no such user
id: globalGroupUser1: no such user
at
org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:291)
at
org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:215)
at
org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroupsSet(ShellBasedUnixGroupsMapping.java:123)
at
org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupSet(Groups.java:413)
at
org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:351)
at
org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:300)
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
at
com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976)
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4960)
at
org.apache.hadoop.security.Groups.getGroupInternal(Groups.java:258)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:216)
at
org.apache.hadoop.hbase.security.User$TestingGroups.getGroups(User.java:420)
at
org.apache.hadoop.hbase.security.access.AccessChecker.getUserGroups(AccessChecker.java:484)
at
org.apache.hadoop.hbase.security.access.PermissionStorage.parsePermissionRecord(PermissionStorage.java:642)
at
org.apache.hadoop.hbase.security.access.PermissionStorage.parsePermissions(PermissionStorage.java:603)
at
org.apache.hadoop.hbase.security.access.PermissionStorage.getPermissions(PermissionStorage.java:540)
at
org.apache.hadoop.hbase.security.access.PermissionStorage.getUserPermissions(PermissionStorage.java:585)
at
org.apache.hadoop.hbase.master.MasterRpcServices.getUserPermissions(MasterRpcServices.java:2845)
```
TestAccessController used to pass before this change only because it
implements its own `ShellBasedUnixGroupsMapping`, overriding `getGroups`
method. But after HADOOP-17079, a new method `getGroupsSet` has been introduced
and is the one that gets called by hadoop security `Groups` class when fetching
the groups cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
