[jira] [Commented] (HDFS-13716) hdfs.DFSclient should log KMS DT acquisition at INFO level

2018-07-09 Thread Xiao Chen (JIRA) 


[ 
https://issues.apache.org/jira/browse/HDFS-13716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537410#comment-16537410
 ] 

Xiao Chen commented on HDFS-13716:
--

Thanks [~knanasi] for reporting the issue and provide the patch. Also thanks 
Wei-Chiu / Zsolt for reviews.

This will be helpful for troubleshooting, and more consistent with the behavior 
in HDFS. No tests needed as this is log-only changes. +1

> hdfs.DFSclient should log KMS DT acquisition at INFO level
> --
>
> Key: HDFS-13716
> URL: https://issues.apache.org/jira/browse/HDFS-13716
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs, hdfs-client, kms
>Affects Versions: 3.1.0
>Reporter: Kitti Nanasi
>Assignee: Kitti Nanasi
>Priority: Minor
> Attachments: HDFS-13716.001.patch
>
>
> We can see HDFS and Hive delegation token (DT) creation as INFO messages in 
> Spark application logs but not for KMS DTs:
> 18/06/07 10:02:35 INFO hdfs.DFSClient: Created token for admin: 
> HDFS_DELEGATION_TOKEN owner=ad...@example.net, renewer=yarn, realUser=, 
> issueDate=1528390955760, maxDate=1528995755760, sequenceNumber=125659, 
> masterKeyId=795 on ha-hdfs:dev
> 18/06/07 10:02:37 INFO hive.metastore: Trying to connect to metastore with 
> URI thrift://hostnam.example.net:9083
> 18/06/07 10:02:37 INFO hive.metastore: Opened a connection to metastore, 
> current connections: 1
> 18/06/07 10:02:37 INFO hive.metastore: Connected to metastore.
> 18/06/07 10:02:37 INFO security.HiveCredentialProvider: Get Token from hive 
> metastore: Kind: HIVE_DELEGATION_TOKEN, Service: , Ident: 00 1b 61 6e 69 73 
> 68 2d 61 64 6d 69 6e 40 43 4f 52 50 2e 49 4e 54 55 49 54 2e 4e 45 54 04 68 69 
> 76 65 00 8a 01 63 db 33 3a 83 8a 01 63 ff 3f be 83 8e 17 8d 8e 06 96
> Please implement KMS DT acquisition events at INFO level as it will help 
> supportability of encrypted HDSF filesystems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13716) hdfs.DFSclient should log KMS DT acquisition at INFO level

2018-07-09 Thread Zsolt Venczel (JIRA) 


[ 
https://issues.apache.org/jira/browse/HDFS-13716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536915#comment-16536915
 ] 

Zsolt Venczel commented on HDFS-13716:
--

Thanks for reporting this [~knanasi] it's a valid issue to have components 
logging consistently.

The patch provided looks good.

+1 (non-binding)

> hdfs.DFSclient should log KMS DT acquisition at INFO level
> --
>
> Key: HDFS-13716
> URL: https://issues.apache.org/jira/browse/HDFS-13716
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs, hdfs-client, kms
>Affects Versions: 3.1.0
>Reporter: Kitti Nanasi
>Assignee: Kitti Nanasi
>Priority: Minor
> Attachments: HDFS-13716.001.patch
>
>
> We can see HDFS and Hive delegation token (DT) creation as INFO messages in 
> Spark application logs but not for KMS DTs:
> 18/06/07 10:02:35 INFO hdfs.DFSClient: Created token for admin: 
> HDFS_DELEGATION_TOKEN owner=ad...@example.net, renewer=yarn, realUser=, 
> issueDate=1528390955760, maxDate=1528995755760, sequenceNumber=125659, 
> masterKeyId=795 on ha-hdfs:dev
> 18/06/07 10:02:37 INFO hive.metastore: Trying to connect to metastore with 
> URI thrift://hostnam.example.net:9083
> 18/06/07 10:02:37 INFO hive.metastore: Opened a connection to metastore, 
> current connections: 1
> 18/06/07 10:02:37 INFO hive.metastore: Connected to metastore.
> 18/06/07 10:02:37 INFO security.HiveCredentialProvider: Get Token from hive 
> metastore: Kind: HIVE_DELEGATION_TOKEN, Service: , Ident: 00 1b 61 6e 69 73 
> 68 2d 61 64 6d 69 6e 40 43 4f 52 50 2e 49 4e 54 55 49 54 2e 4e 45 54 04 68 69 
> 76 65 00 8a 01 63 db 33 3a 83 8a 01 63 ff 3f be 83 8e 17 8d 8e 06 96
> Please implement KMS DT acquisition events at INFO level as it will help 
> supportability of encrypted HDSF filesystems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13716) hdfs.DFSclient should log KMS DT acquisition at INFO level

2018-07-06 Thread Wei-Chiu Chuang (JIRA) 


[ 
https://issues.apache.org/jira/browse/HDFS-13716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535348#comment-16535348
 ] 

Wei-Chiu Chuang commented on HDFS-13716:


Make sense to me, thanks for reporting the issue. CC: [~xiaochen] 

> hdfs.DFSclient should log KMS DT acquisition at INFO level
> --
>
> Key: HDFS-13716
> URL: https://issues.apache.org/jira/browse/HDFS-13716
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs, hdfs-client, kms
>Affects Versions: 3.1.0
>Reporter: Kitti Nanasi
>Assignee: Kitti Nanasi
>Priority: Minor
> Attachments: HDFS-13716.001.patch
>
>
> We can see HDFS and Hive delegation token (DT) creation as INFO messages in 
> Spark application logs but not for KMS DTs:
> 18/06/07 10:02:35 INFO hdfs.DFSClient: Created token for admin: 
> HDFS_DELEGATION_TOKEN owner=ad...@example.net, renewer=yarn, realUser=, 
> issueDate=1528390955760, maxDate=1528995755760, sequenceNumber=125659, 
> masterKeyId=795 on ha-hdfs:dev
> 18/06/07 10:02:37 INFO hive.metastore: Trying to connect to metastore with 
> URI thrift://hostnam.example.net:9083
> 18/06/07 10:02:37 INFO hive.metastore: Opened a connection to metastore, 
> current connections: 1
> 18/06/07 10:02:37 INFO hive.metastore: Connected to metastore.
> 18/06/07 10:02:37 INFO security.HiveCredentialProvider: Get Token from hive 
> metastore: Kind: HIVE_DELEGATION_TOKEN, Service: , Ident: 00 1b 61 6e 69 73 
> 68 2d 61 64 6d 69 6e 40 43 4f 52 50 2e 49 4e 54 55 49 54 2e 4e 45 54 04 68 69 
> 76 65 00 8a 01 63 db 33 3a 83 8a 01 63 ff 3f be 83 8e 17 8d 8e 06 96
> Please implement KMS DT acquisition events at INFO level as it will help 
> supportability of encrypted HDSF filesystems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13716) hdfs.DFSclient should log KMS DT acquisition at INFO level

2018-07-03 Thread genericqa (JIRA) 


[ 
https://issues.apache.org/jira/browse/HDFS-13716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531619#comment-16531619
 ] 

genericqa commented on HDFS-13716:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m 
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m 
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
11m 19s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 25m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 25m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}  
9m 36s{color} | {color:green} patch has no errors when building and testing our 
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  8m 
12s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
32s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}115m 55s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HDFS-13716 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12930123/HDFS-13716.001.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  |
| uname | Linux e62da2690e7c 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 344f324 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/24548/testReport/ |
| Max. process+thread count | 1429 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/24548/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> hdfs.DFSclient should log KM