[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[
https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16979622#comment-16979622
]
Chao Sun commented on HDFS-12300:
-
This DT related info in audit log is very useful for us as well and we are in
branch-2.
[~xiaochen]: Regarding the difficulties of backporting. Can we not depend on
{{toStringStable}} but make a util method with the same implementation? those
fields required by {{toStringStable}} are already available by
{{DelegationTokenIdentifier}}.
> Audit-log delegation token related operations
> -
>
> Key: HDFS-12300
> URL: https://issues.apache.org/jira/browse/HDFS-12300
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 0.22.0
>Reporter: Xiao Chen
>Assignee: Xiao Chen
>Priority: Major
> Fix For: 3.0.0-beta1
>
> Attachments: HDFS-12300.01.patch, HDFS-12300.02.patch
>
>
> When inspecting the code, I found that the following methods in FSNamesystem
> are not audit logged:
> - getDelegationToken
> - renewDelegationToken
> - cancelDelegationToken
> The audit log itself does have a logTokenTrackingId field to additionally log
> some details when a token is used for authentication.
> After emailing the community, we should add that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[
https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16152076#comment-16152076
]
Xiao Chen commented on HDFS-12300:
--
I tried to backport to branch-2, which isn't too hard. But
{{DelegationTokenIdentifier#toStringStable}} was added by HDFS-9732, which is
very messy to get backported, according to Yongjun's
[comment|https://issues.apache.org/jira/browse/HDFS-9732?focusedCommentId=15287588&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15287588].
(Also roughly tried myself, HDFS-5570 is a source of headache, where
{{DelegationTokenFetcher}} was largely modified)
So for this reason I will resolve this as branch-3 only, and link to HDFS-9732
as a dependent.
Thanks again Ravi!
> Audit-log delegation token related operations
> -
>
> Key: HDFS-12300
> URL: https://issues.apache.org/jira/browse/HDFS-12300
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 0.22.0
>Reporter: Xiao Chen
>Assignee: Xiao Chen
> Fix For: 3.0.0-beta1
>
> Attachments: HDFS-12300.01.patch, HDFS-12300.02.patch
>
>
> When inspecting the code, I found that the following methods in FSNamesystem
> are not audit logged:
> - getDelegationToken
> - renewDelegationToken
> - cancelDelegationToken
> The audit log itself does have a logTokenTrackingId field to additionally log
> some details when a token is used for authentication.
> After emailing the community, we should add that.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[ https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150765#comment-16150765 ] Ravi Prakash commented on HDFS-12300: - Thanks for the work Xiao! Up to you to push into branch-2. I'm supportive of it. > Audit-log delegation token related operations > - > > Key: HDFS-12300 > URL: https://issues.apache.org/jira/browse/HDFS-12300 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 0.22.0 >Reporter: Xiao Chen >Assignee: Xiao Chen > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12300.01.patch, HDFS-12300.02.patch > > > When inspecting the code, I found that the following methods in FSNamesystem > are not audit logged: > - getDelegationToken > - renewDelegationToken > - cancelDelegationToken > The audit log itself does have a logTokenTrackingId field to additionally log > some details when a token is used for authentication. > After emailing the community, we should add that. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[ https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150134#comment-16150134 ] Hudson commented on HDFS-12300: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12295 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12295/]) HDFS-12300. Audit-log delegation token related operations. (xiao: rev 1b3b9938cf663c71d2e5d9032fdfb1460bae0d3f) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAuditLoggerWithCommands.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java > Audit-log delegation token related operations > - > > Key: HDFS-12300 > URL: https://issues.apache.org/jira/browse/HDFS-12300 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 0.22.0 >Reporter: Xiao Chen >Assignee: Xiao Chen > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12300.01.patch, HDFS-12300.02.patch > > > When inspecting the code, I found that the following methods in FSNamesystem > are not audit logged: > - getDelegationToken > - renewDelegationToken > - cancelDelegationToken > The audit log itself does have a logTokenTrackingId field to additionally log > some details when a token is used for authentication. > After emailing the community, we should add that. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[ https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150104#comment-16150104 ] Xiao Chen commented on HDFS-12300: -- Committed this to trunk. Thanks Ravi for the review. What do people think about branch-2? The cherry pick isn't clean, but I can put up a branch-2 patch if that's considered valuable. > Audit-log delegation token related operations > - > > Key: HDFS-12300 > URL: https://issues.apache.org/jira/browse/HDFS-12300 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 0.22.0 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12300.01.patch, HDFS-12300.02.patch > > > When inspecting the code, I found that the following methods in FSNamesystem > are not audit logged: > - getDelegationToken > - renewDelegationToken > - cancelDelegationToken > The audit log itself does have a logTokenTrackingId field to additionally log > some details when a token is used for authentication. > After emailing the community, we should add that. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[
https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150095#comment-16150095
]
Hadoop QA commented on HDFS-12300:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 250 unchanged - 1 fixed = 250 total (was 251) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 54s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}121m 12s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.TestLeaseRecoveryStriped |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.TestReadStripedFileWithDecoding |
| | hadoop.hdfs.server.datanode.TestDataNodeUUID |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060 |
| Timed out junit tests | org.apache.hadoop.hdfs.TestReplication |
| | org.apache.hadoop.hdfs.TestWriteReadStripedFile |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12300 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12884824/HDFS-12300.02.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 09f0dbcae0c4 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 27359b7 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20962/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20962/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[ https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150021#comment-16150021 ] Xiao Chen commented on HDFS-12300: -- Thanks again Ravi! Last pre-commit mvn install seems to failed with a race condition, retriggerred. > Audit-log delegation token related operations > - > > Key: HDFS-12300 > URL: https://issues.apache.org/jira/browse/HDFS-12300 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 0.22.0 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12300.01.patch, HDFS-12300.02.patch > > > When inspecting the code, I found that the following methods in FSNamesystem > are not audit logged: > - getDelegationToken > - renewDelegationToken > - cancelDelegationToken > The audit log itself does have a logTokenTrackingId field to additionally log > some details when a token is used for authentication. > After emailing the community, we should add that. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[
https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149906#comment-16149906
]
Hadoop QA commented on HDFS-12300:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
57s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 250 unchanged - 1 fixed = 250 total (was 251) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 9s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130 |
| | hadoop.hdfs.TestFileAppendRestart |
| | hadoop.hdfs.tools.TestDFSAdminWithHA |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110 |
| | hadoop.hdfs.TestEncryptionZones |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090 |
| | hadoop.hdfs.TestLeaseRecoveryStriped |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040 |
| | hadoop.hdfs.TestReadStripedFileWithDecoding |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
| Timed out junit tests | org.apache.hadoop.hdfs.TestWriteReadStripedFile |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12300 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12884824/HDFS-12300.02.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 3eae518577a8 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/pro
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[ https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149796#comment-16149796 ] Ravi Prakash commented on HDFS-12300: - Sounds good to me. Patch looks good to me. +1. > Audit-log delegation token related operations > - > > Key: HDFS-12300 > URL: https://issues.apache.org/jira/browse/HDFS-12300 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 0.22.0 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12300.01.patch, HDFS-12300.02.patch > > > When inspecting the code, I found that the following methods in FSNamesystem > are not audit logged: > - getDelegationToken > - renewDelegationToken > - cancelDelegationToken > The audit log itself does have a logTokenTrackingId field to additionally log > some details when a token is used for authentication. > After emailing the community, we should add that. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[ https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149787#comment-16149787 ] Xiao Chen commented on HDFS-12300: -- Hey Ravi, Thanks a lot for reviewing! I was only deducting about the reasoning why FSN writes is own code instead of calling the general helper method, so I don't have numbers. Considering reflections are usually resource-heavy, and we tend to optimize NN within the namespace lock, it seems plausible. The overhead I was referring to are specific to [these lines|https://github.com/apache/hadoop/blob/branch-3.0.0-alpha4/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java#L170-L174]. IMHO it's up to each module to decide whether this is costly enough for optimization, likely from the result of stress tests. bq. why we don't audit log when some exceptions are thrown and not others I think HDFS-10776 (and its first comment) is the best answer available - we only log AccessControlExceptions, and don't care about others. Patch 2 to fix the checkstyle. > Audit-log delegation token related operations > - > > Key: HDFS-12300 > URL: https://issues.apache.org/jira/browse/HDFS-12300 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 0.22.0 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12300.01.patch, HDFS-12300.02.patch > > > When inspecting the code, I found that the following methods in FSNamesystem > are not audit logged: > - getDelegationToken > - renewDelegationToken > - cancelDelegationToken > The audit log itself does have a logTokenTrackingId field to additionally log > some details when a token is used for authentication. > After emailing the community, we should add that. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[ https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149738#comment-16149738 ] Ravi Prakash commented on HDFS-12300: - Hi Xiao! Thanks a lot for your effort on this. Is the performance overhead of reflection appreciable over here? Should different modules really be decoding tokens? I'm not sure I understand why we don't audit log when some exceptions are thrown and not others. But that is not related to this JIRA. Otherwise patch looks good to me. > Audit-log delegation token related operations > - > > Key: HDFS-12300 > URL: https://issues.apache.org/jira/browse/HDFS-12300 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 0.22.0 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12300.01.patch > > > When inspecting the code, I found that the following methods in FSNamesystem > are not audit logged: > - getDelegationToken > - renewDelegationToken > - cancelDelegationToken > The audit log itself does have a logTokenTrackingId field to additionally log > some details when a token is used for authentication. > After emailing the community, we should add that. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[ https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149172#comment-16149172 ] Xiao Chen commented on HDFS-12300: -- Any watchers have cycle to review? (Will address checkstyle with any comments together in the next rev.) Thank you. > Audit-log delegation token related operations > - > > Key: HDFS-12300 > URL: https://issues.apache.org/jira/browse/HDFS-12300 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 0.22.0 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12300.01.patch > > > When inspecting the code, I found that the following methods in FSNamesystem > are not audit logged: > - getDelegationToken > - renewDelegationToken > - cancelDelegationToken > The audit log itself does have a logTokenTrackingId field to additionally log > some details when a token is used for authentication. > After emailing the community, we should add that. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12300) Audit-log delegation token related operations
[
https://issues.apache.org/jira/browse/HDFS-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143433#comment-16143433
]
Hadoop QA commented on HDFS-12300:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 36s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 1 new + 250 unchanged - 1 fixed = 251 total (was 251) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 85m 33s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}111m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy |
| | hadoop.hdfs.TestReadStripedFileWithDecoding |
| | hadoop.hdfs.TestLeaseRecoveryStriped |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180 |
| | hadoop.hdfs.TestClientProtocolForPipelineRecovery |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050 |
| Timed out junit tests | org.apache.hadoop.hdfs.TestWriteReadStripedFile |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12300 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883991/HDFS-12300.01.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 1649c7fb5a43 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ae7abad |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/20887/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20887/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20887/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreC
