[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[
https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915861#comment-16915861
]
Hadoop QA commented on HADOOP-15602:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} HADOOP-15602 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-15602 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12949271/HADOOP-15602.04.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16505/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Support SASL Rpc request handling in separate Handlers
> ---
>
> Key: HADOOP-15602
> URL: https://issues.apache.org/jira/browse/HADOOP-15602
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Attachments: HADOOP-15602.01.patch, HADOOP-15602.02.patch,
> HADOOP-15602.04.patch
>
>
> Right now, during RPC Connection establishment, all SASL requests are
> considered as OutOfBand requests and handled within the same Reader thread.
> SASL handling involves authentication with Kerberos and SecretManagers(for
> Token validation). During this time, Reader thread would be blocked, hence
> blocking all the incoming RPC requests on other established connections. Some
> secretManager impls require to communicate to external systems (ex: ZK) for
> verification.
> SASL RPC handling in separate dedicated handlers, would enable Reader threads
> to read RPC requests from established connections without blocking.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[ https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915802#comment-16915802 ] Zhankun Tang commented on HADOOP-15602: --- Bulk update: Preparing for 3.1.3 release. moved all 3.1.3 non-blocker issues to 3.1.4, please move back if it is a blocker for you. > Support SASL Rpc request handling in separate Handlers > --- > > Key: HADOOP-15602 > URL: https://issues.apache.org/jira/browse/HADOOP-15602 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > Attachments: HADOOP-15602.01.patch, HADOOP-15602.02.patch, > HADOOP-15602.04.patch > > > Right now, during RPC Connection establishment, all SASL requests are > considered as OutOfBand requests and handled within the same Reader thread. > SASL handling involves authentication with Kerberos and SecretManagers(for > Token validation). During this time, Reader thread would be blocked, hence > blocking all the incoming RPC requests on other established connections. Some > secretManager impls require to communicate to external systems (ex: ZK) for > verification. > SASL RPC handling in separate dedicated handlers, would enable Reader threads > to read RPC requests from established connections without blocking. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[
https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814273#comment-16814273
]
Hadoop QA commented on HADOOP-15602:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} HADOOP-15602 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-15602 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12949271/HADOOP-15602.04.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16138/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Support SASL Rpc request handling in separate Handlers
> ---
>
> Key: HADOOP-15602
> URL: https://issues.apache.org/jira/browse/HADOOP-15602
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Attachments: HADOOP-15602.01.patch, HADOOP-15602.02.patch,
> HADOOP-15602.04.patch
>
>
> Right now, during RPC Connection establishment, all SASL requests are
> considered as OutOfBand requests and handled within the same Reader thread.
> SASL handling involves authentication with Kerberos and SecretManagers(for
> Token validation). During this time, Reader thread would be blocked, hence
> blocking all the incoming RPC requests on other established connections. Some
> secretManager impls require to communicate to external systems (ex: ZK) for
> verification.
> SASL RPC handling in separate dedicated handlers, would enable Reader threads
> to read RPC requests from established connections without blocking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[
https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814271#comment-16814271
]
Brahma Reddy Battula commented on HADOOP-15602:
---
This can be good enchancement when security is enabled.
{quote}I'm not sure there's benefit to using a custom callq and scheduler etc
since those are predicated on weighting users. The user isn't known until the
auth completes so everything is prio 0 and a fair call queue is a no-op.
{quote}
Yes, As of now this custom and fair callQueue might not benfit here. Anyway by
Default it will use linkedBlocking Queue
Changes are lgtm apart from following suggestion.
It's better to log the "namespace" also so that we can diffenraite whether it's
from SASL or normal.
{code:java}
LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
maxQueueSize + " scheduler: " + schedulerClass);
{code}
[~daryn], if you get chance, can you look once..?
> Support SASL Rpc request handling in separate Handlers
> ---
>
> Key: HADOOP-15602
> URL: https://issues.apache.org/jira/browse/HADOOP-15602
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Attachments: HADOOP-15602.01.patch, HADOOP-15602.02.patch,
> HADOOP-15602.04.patch
>
>
> Right now, during RPC Connection establishment, all SASL requests are
> considered as OutOfBand requests and handled within the same Reader thread.
> SASL handling involves authentication with Kerberos and SecretManagers(for
> Token validation). During this time, Reader thread would be blocked, hence
> blocking all the incoming RPC requests on other established connections. Some
> secretManager impls require to communicate to external systems (ex: ZK) for
> verification.
> SASL RPC handling in separate dedicated handlers, would enable Reader threads
> to read RPC requests from established connections without blocking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[ https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762447#comment-16762447 ] Vinayakumar B commented on HADOOP-15602: [~daryn], Please can you take a look at the patch? > Support SASL Rpc request handling in separate Handlers > --- > > Key: HADOOP-15602 > URL: https://issues.apache.org/jira/browse/HADOOP-15602 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > Attachments: HADOOP-15602.01.patch, HADOOP-15602.02.patch, > HADOOP-15602.04.patch > > > Right now, during RPC Connection establishment, all SASL requests are > considered as OutOfBand requests and handled within the same Reader thread. > SASL handling involves authentication with Kerberos and SecretManagers(for > Token validation). During this time, Reader thread would be blocked, hence > blocking all the incoming RPC requests on other established connections. Some > secretManager impls require to communicate to external systems (ex: ZK) for > verification. > SASL RPC handling in separate dedicated handlers, would enable Reader threads > to read RPC requests from established connections without blocking. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[
https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697070#comment-16697070
]
Hadoop QA commented on HADOOP-15602:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 51s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m
5s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 58s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 11 new + 361 unchanged - 10 fixed = 372 total (was 371) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 27s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
20s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}112m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-15602 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12949271/HADOOP-15602.04.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 931785d07eed 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 753f421 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15556/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15556/testReport/ |
| Max. process+thread count | 1515 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15556/console |
| Powered by |
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[ https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696576#comment-16696576 ] Vinayakumar B commented on HADOOP-15602: Updated the correct patch. > Support SASL Rpc request handling in separate Handlers > --- > > Key: HADOOP-15602 > URL: https://issues.apache.org/jira/browse/HADOOP-15602 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > Attachments: HADOOP-15602.01.patch, HADOOP-15602.02.patch, > HADOOP-15602.04.patch > > > Right now, during RPC Connection establishment, all SASL requests are > considered as OutOfBand requests and handled within the same Reader thread. > SASL handling involves authentication with Kerberos and SecretManagers(for > Token validation). During this time, Reader thread would be blocked, hence > blocking all the incoming RPC requests on other established connections. Some > secretManager impls require to communicate to external systems (ex: ZK) for > verification. > SASL RPC handling in separate dedicated handlers, would enable Reader threads > to read RPC requests from established connections without blocking. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[
https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695605#comment-16695605
]
Hadoop QA commented on HADOOP-15602:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HADOOP-15602 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-15602 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12949147/HADOOP-15602.03.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15551/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Support SASL Rpc request handling in separate Handlers
> ---
>
> Key: HADOOP-15602
> URL: https://issues.apache.org/jira/browse/HADOOP-15602
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Attachments: HADOOP-15602.01.patch, HADOOP-15602.02.patch,
> HADOOP-15602.03.patch
>
>
> Right now, during RPC Connection establishment, all SASL requests are
> considered as OutOfBand requests and handled within the same Reader thread.
> SASL handling involves authentication with Kerberos and SecretManagers(for
> Token validation). During this time, Reader thread would be blocked, hence
> blocking all the incoming RPC requests on other established connections. Some
> secretManager impls require to communicate to external systems (ex: ZK) for
> verification.
> SASL RPC handling in separate dedicated handlers, would enable Reader threads
> to read RPC requests from established connections without blocking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[ https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695594#comment-16695594 ] Vinayakumar B commented on HADOOP-15602: Fixed checkstyle errors. > Support SASL Rpc request handling in separate Handlers > --- > > Key: HADOOP-15602 > URL: https://issues.apache.org/jira/browse/HADOOP-15602 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > Attachments: HADOOP-15602.01.patch, HADOOP-15602.02.patch, > HADOOP-15602.03.patch > > > Right now, during RPC Connection establishment, all SASL requests are > considered as OutOfBand requests and handled within the same Reader thread. > SASL handling involves authentication with Kerberos and SecretManagers(for > Token validation). During this time, Reader thread would be blocked, hence > blocking all the incoming RPC requests on other established connections. Some > secretManager impls require to communicate to external systems (ex: ZK) for > verification. > SASL RPC handling in separate dedicated handlers, would enable Reader threads > to read RPC requests from established connections without blocking. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[
https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694546#comment-16694546
]
Hadoop QA commented on HADOOP-15602:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 55s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
43s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 52s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 26 new + 361 unchanged - 10 fixed = 387 total (was 371) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 3s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 20s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 95m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.util.TestDiskCheckerWithDiskIo |
| | hadoop.util.TestReadWriteDiskValidator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-15602 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12949009/HADOOP-15602.02.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 96195cf1e20d 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / c8b3dfa |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15549/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15549/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[ https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694402#comment-16694402 ] Vinayakumar B commented on HADOOP-15602: Rebased and Handled more error cases. For SIMPLE authentications, SASL messages will be processed in the same Reader thread to avoid races. > Support SASL Rpc request handling in separate Handlers > --- > > Key: HADOOP-15602 > URL: https://issues.apache.org/jira/browse/HADOOP-15602 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > Attachments: HADOOP-15602.01.patch, HADOOP-15602.02.patch > > > Right now, during RPC Connection establishment, all SASL requests are > considered as OutOfBand requests and handled within the same Reader thread. > SASL handling involves authentication with Kerberos and SecretManagers(for > Token validation). During this time, Reader thread would be blocked, hence > blocking all the incoming RPC requests on other established connections. Some > secretManager impls require to communicate to external systems (ex: ZK) for > verification. > SASL RPC handling in separate dedicated handlers, would enable Reader threads > to read RPC requests from established connections without blocking. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[
https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689799#comment-16689799
]
Hadoop QA commented on HADOOP-15602:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} HADOOP-15602 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-15602 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12931277/HADOOP-15602.01.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15538/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Support SASL Rpc request handling in separate Handlers
> ---
>
> Key: HADOOP-15602
> URL: https://issues.apache.org/jira/browse/HADOOP-15602
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Attachments: HADOOP-15602.01.patch
>
>
> Right now, during RPC Connection establishment, all SASL requests are
> considered as OutOfBand requests and handled within the same Reader thread.
> SASL handling involves authentication with Kerberos and SecretManagers(for
> Token validation). During this time, Reader thread would be blocked, hence
> blocking all the incoming RPC requests on other established connections. Some
> secretManager impls require to communicate to external systems (ex: ZK) for
> verification.
> SASL RPC handling in separate dedicated handlers, would enable Reader threads
> to read RPC requests from established connections without blocking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[ https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596518#comment-16596518 ] Vinayakumar B commented on HADOOP-15602: Hi [~daryn] , Wondering whether you got chance to look at the changes? > Support SASL Rpc request handling in separate Handlers > --- > > Key: HADOOP-15602 > URL: https://issues.apache.org/jira/browse/HADOOP-15602 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > Attachments: HADOOP-15602.01.patch > > > Right now, during RPC Connection establishment, all SASL requests are > considered as OutOfBand requests and handled within the same Reader thread. > SASL handling involves authentication with Kerberos and SecretManagers(for > Token validation). During this time, Reader thread would be blocked, hence > blocking all the incoming RPC requests on other established connections. Some > secretManager impls require to communicate to external systems (ex: ZK) for > verification. > SASL RPC handling in separate dedicated handlers, would enable Reader threads > to read RPC requests from established connections without blocking. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[
https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542640#comment-16542640
]
Vinayakumar B commented on HADOOP-15602:
Thank you [~daryn].
{quote}I'm not sure there's benefit to using a custom callq and scheduler etc
since those are predicated on weighting users. The user isn't known until the
auth completes so everything is prio 0 and a fair call queue is a no-op.
{quote}
Okay. I just wanted to re-use existing abstraction. May be in future, some
other special call queue impl could help. {{FairCallQueue}} will be used only
on configuration, even for RPC Queue. By default, {{LinkedBlockingQueue}} will
be used for both.
{quote}I'll try to look at the more throughly this week. Did you make sure that
it works correctly when requests are split over multiple sasl packets? Or that
clients are correctly disconnected after errors, etc?
{quote}
Do you mean multiple SASL messages from same connection? If yes, - This should
not happen as client waits for previous SASL message's response before sending
next sasl message.
If you are talking about {{RPC_CONTINUATION_PACKET}}, I could not find any
implementation yet about this, for normal RPC request as well. All RPCs and
SASL are sent as {{RPC_FINAL_PACKET}} in the header, right now.
{code:java}
/**
* RPC_CONTINUATION_PACKET = 1;
*
*
* not implemented yet
*
*/
RPC_CONTINUATION_PACKET(1, 1),
{code}
> Support SASL Rpc request handling in separate Handlers
> ---
>
> Key: HADOOP-15602
> URL: https://issues.apache.org/jira/browse/HADOOP-15602
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Attachments: HADOOP-15602.01.patch
>
>
> Right now, during RPC Connection establishment, all SASL requests are
> considered as OutOfBand requests and handled within the same Reader thread.
> SASL handling involves authentication with Kerberos and SecretManagers(for
> Token validation). During this time, Reader thread would be blocked, hence
> blocking all the incoming RPC requests on other established connections. Some
> secretManager impls require to communicate to external systems (ex: ZK) for
> verification.
> SASL RPC handling in separate dedicated handlers, would enable Reader threads
> to read RPC requests from established connections without blocking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[ https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541840#comment-16541840 ] Daryn Sharp commented on HADOOP-15602: -- Nice. This is something I've long pondered doing (even just last week) since clients rapidly reconnecting (ex. overwhelmedthe call queue) may degrade reader performance by repeatedly authenticating just to be disconnected again. I'm not sure there's benefit to using a custom callq and scheduler etc since those are predicated on weighting users. The user isn't known until the auth completes so everything is prio 0 and a fair call queue is a no-op. I'll try to look at the more throughly this week. Did you make sure that it works correctly when requests are split over multiple sasl packets? Or that clients are correctly disconnected after errors, etc? > Support SASL Rpc request handling in separate Handlers > --- > > Key: HADOOP-15602 > URL: https://issues.apache.org/jira/browse/HADOOP-15602 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > Attachments: HADOOP-15602.01.patch > > > Right now, during RPC Connection establishment, all SASL requests are > considered as OutOfBand requests and handled within the same Reader thread. > SASL handling involves authentication with Kerberos and SecretManagers(for > Token validation). During this time, Reader thread would be blocked, hence > blocking all the incoming RPC requests on other established connections. Some > secretManager impls require to communicate to external systems (ex: ZK) for > verification. > SASL RPC handling in separate dedicated handlers, would enable Reader threads > to read RPC requests from established connections without blocking. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers
[
https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541315#comment-16541315
]
genericqa commented on HADOOP-15602:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 49s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
55s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}126m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-15602 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12931277/HADOOP-15602.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 7f704723ca27 3.13.0-144-generic #193-Ubuntu SMP Thu Mar 15
17:03:53 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / b37074b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14883/testReport/ |
| Max. process+thread count | 1718 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14883/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Support SASL Rpc request handling in separate Handlers
> ---
>
> Key: HADOOP-15602
>
