[
https://issues.apache.org/jira/browse/HADOOP-17346?focusedWorklogId=507464&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-507464
]
ASF GitHub Bot logged work on HADOOP-17346:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 04/Nov/20 02:58
Start Date: 04/Nov/20 02:58
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2431:
URL: https://github.com/apache/hadoop/pull/2431#issuecomment-721484027
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 28m 58s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 25s | | trunk passed |
| +1 :green_heart: | compile | 20m 1s | | trunk passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 |
| +1 :green_heart: | compile | 17m 9s | | trunk passed with JDK
Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 |
| +1 :green_heart: | checkstyle | 1m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 35s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK
Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 |
| +0 :ok: | spotbugs | 2m 23s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 20s | | trunk passed |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 52s | | the patch passed |
| +1 :green_heart: | compile | 19m 18s | | the patch passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 |
| +1 :green_heart: | javac | 19m 18s | | the patch passed |
| +1 :green_heart: | compile | 18m 6s | | the patch passed with JDK
Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 |
| +1 :green_heart: | javac | 18m 6s | | the patch passed |
| +1 :green_heart: | checkstyle | 1m 9s | |
hadoop-common-project/hadoop-common: The patch generated 0 new + 486 unchanged
- 1 fixed = 486 total (was 487) |
| +1 :green_heart: | mvnsite | 1m 38s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 17m 13s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 4s | | the patch passed with JDK
Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 |
| +1 :green_heart: | javadoc | 1m 42s | | the patch passed with JDK
Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 |
| +1 :green_heart: | findbugs | 2m 49s | | the patch passed |
|||| _ Other Tests _ |
| -1 :x: | unit | 10m 30s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2431/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 53s | | The patch does not
generate ASF License warnings. |
| | | 202m 3s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.metrics2.source.TestJvmMetrics |
| | hadoop.crypto.key.TestKeyProviderFactory |
| | hadoop.security.TestLdapGroupsMapping |
| | hadoop.ipc.TestDecayRpcScheduler |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2431/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2431 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 801db6ba89b3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d07dc7afb4a |
| Default Java | Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2431/1/testReport/ |
| Max. process+thread count | 1605 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2431/1/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.1.3 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 507464)
Time Spent: 20m (was: 10m)
> Fair call queue is defeated by abusive service principals
> ---------------------------------------------------------
>
> Key: HADOOP-17346
> URL: https://issues.apache.org/jira/browse/HADOOP-17346
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, ipc
> Reporter: Ahmed Hussein
> Assignee: Ahmed Hussein
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> [~daryn] reported that the FCQ prioritizes based on the full kerberos
> principal (ie. "user/host@realm") rather than short name (ie. "user") to
> prevent service principals like the DNs and NMs being de-prioritized since
> service principals are expected to be well behaved. Notably the DNs
> contribute a significant but important load so the intent is not to
> de-prioritize all DNs because their sum total load is high relative to users.
> This has the unfortunate side effect of allowing misbehaving & non-critical
> service principals to abuse the FCQ. The gstorm/* principals are a prime
> example. Each server is spamming opens as fast as possible which ensures
> that none of the gstorm servers can be de-prioritized because each principal
> is a fraction of the total load from all principals.
> The secondary and more devasting problem is other abusive non-service
> principals cannot be effectively de-prioritized. The sum total of all gstorm
> load prevents other principals from surpassing the priority thresholds.
> Principals stay in the highest priority queues which allows the abusive
> principals to overflow the entire call queue for extended periods of time.
> Notably it prevents the FCQ from moderating the heavy create loads from p_gup
> @ DB which cause significant performance degradation.
> Prioritization should be based on short name with configurable exemptions for
> services like the DN/NM.
> [~daryn] suggested a solution that we applied on our clusters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
