[
https://issues.apache.org/jira/browse/HADOOP-17079?focusedWorklogId=538749&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-538749
]
ASF GitHub Bot logged work on HADOOP-17079:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 20/Jan/21 22:59
Start Date: 20/Jan/21 22:59
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2631:
URL: https://github.com/apache/hadoop/pull/2631#issuecomment-764008495
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 11m 16s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any
@author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include
12 new or modified test files. |
||| _ branch-2.10 Compile Tests _ |
| +0 :ok: | mvndep | 2m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 14m 53s | branch-2.10 passed |
| +1 :green_heart: | compile | 13m 24s | branch-2.10 passed |
| +1 :green_heart: | checkstyle | 2m 21s | branch-2.10 passed |
| +1 :green_heart: | mvnsite | 6m 54s | branch-2.10 passed |
| +1 :green_heart: | javadoc | 5m 18s | branch-2.10 passed |
| +0 :ok: | spotbugs | 2m 3s | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 11m 6s | branch-2.10 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 39s | the patch passed |
| +1 :green_heart: | compile | 12m 55s | the patch passed |
| -1 :x: | javac | 12m 55s | root generated 79 new + 1531 unchanged - 3
fixed = 1610 total (was 1534) |
| +1 :green_heart: | checkstyle | 2m 18s | root: The patch generated 0
new + 1008 unchanged - 11 fixed = 1008 total (was 1019) |
| +1 :green_heart: | mvnsite | 6m 30s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace
issues. |
| +1 :green_heart: | javadoc | 5m 27s | the patch passed |
| -1 :x: | findbugs | 2m 45s | hadoop-common-project/hadoop-common
generated 15 new + 0 unchanged - 0 fixed = 15 total (was 0) |
||| _ Other Tests _ |
| -1 :x: | unit | 9m 18s | hadoop-common in the patch failed. |
| -1 :x: | unit | 63m 3s | hadoop-hdfs in the patch failed. |
| +1 :green_heart: | unit | 5m 0s | hadoop-hdfs-httpfs in the patch
passed. |
| -1 :x: | unit | 17m 55s | hadoop-hdfs-rbf in the patch failed. |
| -1 :x: | unit | 67m 22s | hadoop-yarn-server-resourcemanager in the
patch failed. |
| +1 :green_heart: | unit | 4m 24s | hadoop-mapreduce-client-hs in the
patch passed. |
| +1 :green_heart: | asflicense | 0m 59s | The patch does not generate
ASF License warnings. |
| | | 287m 31s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hadoop-common-project/hadoop-common |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.bindPassword; locked 75% of time
Unsynchronized access at LdapGroupsMapping.java:75% of time Unsynchronized
access at LdapGroupsMapping.java:[line 631] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.bindUser; locked 50% of time
Unsynchronized access at LdapGroupsMapping.java:50% of time Unsynchronized
access at LdapGroupsMapping.java:[line 630] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.conf; locked 50% of time
Unsynchronized access at LdapGroupsMapping.java:50% of time Unsynchronized
access at LdapGroupsMapping.java:[line 633] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.groupHierarchyLevels; locked 50%
of time Unsynchronized access at LdapGroupsMapping.java:50% of time
Unsynchronized access at LdapGroupsMapping.java:[line 340] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.groupNameAttr; locked 50% of time
Unsynchronized access at LdapGroupsMapping.java:50% of time Unsynchronized
access at LdapGroupsMapping.java:[line 534] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.ldapCxtFactoryClass; locked 50% of
time Unsynchronized access at LdapGroupsMapping.java:50% of time
Unsynchronized access at LdapGroupsMapping.java:[line 614] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.ldapUrls; locked 50% of time
Unsynchronized access at LdapGroupsMapping.java:50% of time Unsynchronized
access at LdapGroupsMapping.java:[line 751] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.memberOfAttr; locked 60% of time
Unsynchronized access at LdapGroupsMapping.java:60% of time Unsynchronized
access at LdapGroupsMapping.java:[line 504] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.numAttemptsBeforeFailover; locked
50% of time Unsynchronized access at LdapGroupsMapping.java:50% of time
Unsynchronized access at LdapGroupsMapping.java:[line 346] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.posixGidAttr; locked 50% of time
Unsynchronized access at LdapGroupsMapping.java:50% of time Unsynchronized
access at LdapGroupsMapping.java:[line 398] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.posixUidAttr; locked 60% of time
Unsynchronized access at LdapGroupsMapping.java:60% of time Unsynchronized
access at LdapGroupsMapping.java:[line 399] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.useOneQuery; locked 66% of time
Unsynchronized access at LdapGroupsMapping.java:66% of time Unsynchronized
access at LdapGroupsMapping.java:[line 497] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.useSsl; locked 66% of time
Unsynchronized access at LdapGroupsMapping.java:66% of time Unsynchronized
access at LdapGroupsMapping.java:[line 619] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.userSearchFilter; locked 66% of
time Unsynchronized access at LdapGroupsMapping.java:66% of time
Unsynchronized access at LdapGroupsMapping.java:[line 486] |
| | Inconsistent synchronization of
org.apache.hadoop.security.LdapGroupsMapping.userbaseDN; locked 66% of time
Unsynchronized access at LdapGroupsMapping.java:66% of time Unsynchronized
access at LdapGroupsMapping.java:[line 486] |
| Failed junit tests | hadoop.security.TestLdapGroupsMappingWithOneQuery |
| | hadoop.security.TestLdapGroupsMapping |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys |
| |
hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain |
| | hadoop.hdfs.server.federation.resolver.TestMultipleDestinationResolver
|
| | hadoop.hdfs.server.federation.resolver.order.TestLocalResolver |
| | hadoop.hdfs.server.federation.router.TestRouterNamenodeHeartbeat |
| | hadoop.hdfs.server.federation.router.TestRouterQuota |
| | hadoop.yarn.server.resourcemanager.TestClientRMService |
| | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
|
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2631/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2631 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux d6ad7d8e5ff3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-2.10 / ea3c3d9 |
| Default Java | Oracle Corporation-1.7.0_95-b00 |
| javac |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2631/1/artifact/out/diff-compile-javac-root.txt
|
| findbugs |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2631/1/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html
|
| unit |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2631/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
| unit |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2631/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2631/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| unit |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2631/1/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2631/1/testReport/ |
| Max. process+thread count | 2622 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs
hadoop-hdfs-project/hadoop-hdfs-rbf
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U:
. |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2631/1/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 538749)
Time Spent: 20m (was: 10m)
> Optimize UGI#getGroups by adding UGI#getGroupsSet
> -------------------------------------------------
>
> Key: HADOOP-17079
> URL: https://issues.apache.org/jira/browse/HADOOP-17079
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Xiaoyu Yao
> Assignee: Xiaoyu Yao
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: HADOOP-17079.002.patch, HADOOP-17079.003.patch,
> HADOOP-17079.004.patch, HADOOP-17079.005.patch, HADOOP-17079.006.patch,
> HADOOP-17079.007.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> UGI#getGroups has been optimized with HADOOP-13442 by avoiding the
> List->Set->List conversion. However the returned list is not optimized to
> contains lookup, especially the user's group membership list is huge
> (thousands+) . This ticket is opened to add a UGI#getGroupsSet and use
> Set#contains() instead of List#contains() to speed up large group look up
> while minimize List->Set conversions in Groups#getGroups() call.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
