[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428774#comment-15428774 ] Rushabh S Shah commented on HDFS-7959: -- Thank you !! > WebHdfs logging is missing on Datanode > -- > > Key: HDFS-7959 > URL: https://issues.apache.org/jira/browse/HDFS-7959 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Critical > Labels: BB2015-05-TBR > Fix For: 2.8.0 > > Attachments: HDFS-7959.1.branch-2.patch, HDFS-7959.1.trunk.patch, > HDFS-7959.2.branch-2.patch, HDFS-7959.2.trunk.patch, > HDFS-7959.3.branch-2.patch, HDFS-7959.3.trunk.patch, > HDFS-7959.branch-2.patch, HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch, > HDFS-7959.trunk.patch > > > After the conversion to netty, webhdfs requests are not logged on datanodes. > The existing jetty log only logs the non-webhdfs requests that come through > the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428770#comment-15428770 ] Kihwal Lee commented on HDFS-7959: -- Cherry-picked to 2.8. > WebHdfs logging is missing on Datanode > -- > > Key: HDFS-7959 > URL: https://issues.apache.org/jira/browse/HDFS-7959 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Critical > Labels: BB2015-05-TBR > Fix For: 2.8.0 > > Attachments: HDFS-7959.1.branch-2.patch, HDFS-7959.1.trunk.patch, > HDFS-7959.2.branch-2.patch, HDFS-7959.2.trunk.patch, > HDFS-7959.3.branch-2.patch, HDFS-7959.3.trunk.patch, > HDFS-7959.branch-2.patch, HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch, > HDFS-7959.trunk.patch > > > After the conversion to netty, webhdfs requests are not logged on datanodes. > The existing jetty log only logs the non-webhdfs requests that come through > the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428722#comment-15428722 ] Rushabh S Shah commented on HDFS-7959: -- [~kihwal],[~sjlee0]: any reason why this change shouldn't go in branch-2.8 ? If it should then I can prepare a branch-2.8 patch. > WebHdfs logging is missing on Datanode > -- > > Key: HDFS-7959 > URL: https://issues.apache.org/jira/browse/HDFS-7959 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Critical > Labels: BB2015-05-TBR > Fix For: 2.9.0 > > Attachments: HDFS-7959.1.branch-2.patch, HDFS-7959.1.trunk.patch, > HDFS-7959.2.branch-2.patch, HDFS-7959.2.trunk.patch, > HDFS-7959.3.branch-2.patch, HDFS-7959.3.trunk.patch, > HDFS-7959.branch-2.patch, HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch, > HDFS-7959.trunk.patch > > > After the conversion to netty, webhdfs requests are not logged on datanodes. > The existing jetty log only logs the non-webhdfs requests that come through > the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348742#comment-15348742 ] Hudson commented on HDFS-7959: -- SUCCESS: Integrated in Hadoop-trunk-Commit #10018 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10018/]) HDFS-7959. WebHdfs logging is missing on Datanode (Kihwal Lee via sjlee) (sjlee: rev bf74dbf80dc9379d669779a598950908adffb8a7) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java * hadoop-common-project/hadoop-common/src/main/conf/log4j.properties > WebHdfs logging is missing on Datanode > -- > > Key: HDFS-7959 > URL: https://issues.apache.org/jira/browse/HDFS-7959 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Critical > Labels: BB2015-05-TBR > Fix For: 2.9.0 > > Attachments: HDFS-7959.1.branch-2.patch, HDFS-7959.1.trunk.patch, > HDFS-7959.2.branch-2.patch, HDFS-7959.2.trunk.patch, > HDFS-7959.3.branch-2.patch, HDFS-7959.3.trunk.patch, > HDFS-7959.branch-2.patch, HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch, > HDFS-7959.trunk.patch > > > After the conversion to netty, webhdfs requests are not logged on datanodes. > The existing jetty log only logs the non-webhdfs requests that come through > the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348716#comment-15348716
]
Kihwal Lee commented on HDFS-7959:
--
{{TestHttpServerLifecycle}} works fine. Ran multiple times.
{noformat}
---
T E S T S
---
Running org.apache.hadoop.http.TestHttpServerLifecycle
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.307 sec - in
org.apache.hadoop.http.TestHttpServerLifecycle
Results :
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
{noformat}
{{TestOfflineEditsViewer}} HDFS-10572
> WebHdfs logging is missing on Datanode
> --
>
> Key: HDFS-7959
> URL: https://issues.apache.org/jira/browse/HDFS-7959
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Kihwal Lee
>Assignee: Kihwal Lee
>Priority: Critical
> Labels: BB2015-05-TBR
> Attachments: HDFS-7959.1.branch-2.patch, HDFS-7959.1.trunk.patch,
> HDFS-7959.2.branch-2.patch, HDFS-7959.2.trunk.patch,
> HDFS-7959.3.branch-2.patch, HDFS-7959.3.trunk.patch,
> HDFS-7959.branch-2.patch, HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch,
> HDFS-7959.trunk.patch
>
>
> After the conversion to netty, webhdfs requests are not logged on datanodes.
> The existing jetty log only logs the non-webhdfs requests that come through
> the internal proxy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348575#comment-15348575 ] Sangjin Lee commented on HDFS-7959: --- Thanks for updating the patch Kihwal! I'll take a look at it as soon as I can access the JIRA (it's not responding atm). > WebHdfs logging is missing on Datanode > -- > > Key: HDFS-7959 > URL: https://issues.apache.org/jira/browse/HDFS-7959 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Critical > Labels: BB2015-05-TBR > Attachments: HDFS-7959.1.branch-2.patch, HDFS-7959.1.trunk.patch, > HDFS-7959.2.branch-2.patch, HDFS-7959.2.trunk.patch, > HDFS-7959.3.branch-2.patch, HDFS-7959.3.trunk.patch, > HDFS-7959.branch-2.patch, HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch, > HDFS-7959.trunk.patch > > > After the conversion to netty, webhdfs requests are not logged on datanodes. > The existing jetty log only logs the non-webhdfs requests that come through > the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348570#comment-15348570
]
Kihwal Lee commented on HDFS-7959:
--
{{TestHttpServerLifecycle}} works fine. Ran multiple times.
{noformat}
---
T E S T S
---
Running org.apache.hadoop.http.TestHttpServerLifecycle
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.307 sec - in
org.apache.hadoop.http.TestHttpServerLifecycle
Results :
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
{noformat}
{{TestOfflineEditsViewer}} HDFS-10572
> WebHdfs logging is missing on Datanode
> --
>
> Key: HDFS-7959
> URL: https://issues.apache.org/jira/browse/HDFS-7959
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Kihwal Lee
>Assignee: Kihwal Lee
>Priority: Critical
> Labels: BB2015-05-TBR
> Attachments: HDFS-7959.1.branch-2.patch, HDFS-7959.1.trunk.patch,
> HDFS-7959.2.branch-2.patch, HDFS-7959.2.trunk.patch,
> HDFS-7959.3.branch-2.patch, HDFS-7959.3.trunk.patch,
> HDFS-7959.branch-2.patch, HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch,
> HDFS-7959.trunk.patch
>
>
> After the conversion to netty, webhdfs requests are not logged on datanodes.
> The existing jetty log only logs the non-webhdfs requests that come through
> the internal proxy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348525#comment-15348525
]
Hadoop QA commented on HDFS-7959:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
53s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} root: The patch generated 0 new + 13 unchanged - 1
fixed = 13 total (was 14) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 24s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 57m 49s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}127m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer |
| Timed out junit tests | org.apache.hadoop.http.TestHttpServerLifecycle |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:85209cc |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12813094/HDFS-7959.3.trunk.patch
|
| JIRA Issue | HDFS-7959 |
| Optional Tests | asflicense mvnsite unit compile javac javadoc
mvninstall findbugs checkstyle |
| uname | Linux eb1a2860a75c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 6314843 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/15904/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| unit |
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342650#comment-15342650
]
Sangjin Lee commented on HDFS-7959:
---
Thanks for updating the patch [~kihwal]! Just one more nit: can we use
{{INTERNAL_SERVER_ERROR}} instead of hard-coded numeral 500?
> WebHdfs logging is missing on Datanode
> --
>
> Key: HDFS-7959
> URL: https://issues.apache.org/jira/browse/HDFS-7959
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Kihwal Lee
>Assignee: Kihwal Lee
>Priority: Critical
> Labels: BB2015-05-TBR
> Attachments: HDFS-7959.1.branch-2.patch, HDFS-7959.1.trunk.patch,
> HDFS-7959.2.branch-2.patch, HDFS-7959.2.trunk.patch,
> HDFS-7959.branch-2.patch, HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch,
> HDFS-7959.trunk.patch
>
>
> After the conversion to netty, webhdfs requests are not logged on datanodes.
> The existing jetty log only logs the non-webhdfs requests that come through
> the internal proxy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342211#comment-15342211
]
Hadoop QA commented on HDFS-7959:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
17s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 31s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 43s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 2s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 40s
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s {color} | {color:green} root: The patch generated 0 new + 13 unchanged - 1
fixed = 13 total (was 14) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 39s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 51s {color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 59m 8s {color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 105m 4s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.net.TestDNS |
| | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer |
| | hadoop.hdfs.server.namenode.TestCacheDirectives |
| | hadoop.hdfs.server.namenode.TestNamenodeRetryCache |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:e2f6409 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12812210/HDFS-7959.2.trunk.patch
|
| JIRA Issue | HDFS-7959 |
| Optional Tests | asflicense mvnsite unit compile javac javadoc
mvninstall findbugs checkstyle |
| uname | Linux 1e5c2e9133b2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / e15cd43 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| unit |
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336674#comment-15336674
]
Sangjin Lee commented on HDFS-7959:
---
Thanks [~kihwal] for updating the patch! It looks good to me for the most part.
I only have a couple of minor questions.
{code}
125 String host =
((InetSocketAddress)ctx.channel().remoteAddress()).
126 getAddress().getHostAddress();
127 HttpMethod method = req.method();
{code}
Could this throw an exception (esp. host) in some situations? I think it might
be a good idea to move this into the {{finally}} clause itself and make an
exception here non-fatal. Thoughts?
{code}
140 if (resp == null) {
141 return "000";
142 }
{code}
"000" is not a valid HTTP status code. If the response is null for any reason,
"500" might be as good a choice as any. Also, if we change it to 500, we can
then simply use {{int}} as the return type of {{getResponseCode()}}.
> WebHdfs logging is missing on Datanode
> --
>
> Key: HDFS-7959
> URL: https://issues.apache.org/jira/browse/HDFS-7959
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Kihwal Lee
>Assignee: Kihwal Lee
>Priority: Critical
> Labels: BB2015-05-TBR
> Attachments: HDFS-7959.1.branch-2.patch, HDFS-7959.1.trunk.patch,
> HDFS-7959.branch-2.patch, HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch,
> HDFS-7959.trunk.patch
>
>
> After the conversion to netty, webhdfs requests are not logged on datanodes.
> The existing jetty log only logs the non-webhdfs requests that come through
> the internal proxy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328371#comment-15328371
]
Kihwal Lee commented on HDFS-7959:
--
The test failure is not related to the patch.
The whitespace warning is from
{{./hadoop-build-tools/src/main/resources/META-INF/LICENSE.tx}}
> WebHdfs logging is missing on Datanode
> --
>
> Key: HDFS-7959
> URL: https://issues.apache.org/jira/browse/HDFS-7959
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Kihwal Lee
>Assignee: Kihwal Lee
>Priority: Critical
> Labels: BB2015-05-TBR
> Attachments: HDFS-7959.1.branch-2.patch, HDFS-7959.1.trunk.patch,
> HDFS-7959.branch-2.patch, HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch,
> HDFS-7959.trunk.patch
>
>
> After the conversion to netty, webhdfs requests are not logged on datanodes.
> The existing jetty log only logs the non-webhdfs requests that come through
> the internal proxy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328357#comment-15328357
]
Hadoop QA commented on HDFS-7959:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 54s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 0s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
29s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 50s
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s {color} | {color:green} root: The patch generated 0 new + 13 unchanged - 1
fixed = 13 total (was 14) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 20 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 47s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 57s {color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 59m 26s
{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 113m 4s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ha.TestZKFailoverController |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809982/HDFS-7959.1.trunk.patch
|
| JIRA Issue | HDFS-7959 |
| Optional Tests | asflicense mvnsite unit compile javac javadoc
mvninstall findbugs checkstyle |
| uname | Linux 4a21625b4a74 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 709a814 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/15759/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/15759/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| unit test logs |
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327719#comment-15327719
]
Hadoop QA commented on HDFS-7959:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 30s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 31s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
39s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 59s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 44s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 2s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 40s
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 20s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 6m 20s {color}
| {color:red} root generated 3 new + 691 unchanged - 0 fixed = 694 total (was
691) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 18s
{color} | {color:red} root: The patch generated 1 new + 13 unchanged - 1 fixed
= 14 total (was 14) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 20 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m 53s
{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 56m 45s {color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 104m 0s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestAsyncHDFSWithHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809865/HDFS-7959.trunk.patch
|
| JIRA Issue | HDFS-7959 |
| Optional Tests | asflicense mvnsite unit compile javac javadoc
mvninstall findbugs checkstyle |
| uname | Linux 81fd6cfd23fb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 28b66ae |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| javac |
https://builds.apache.org/job/PreCommit-HDFS-Build/15751/artifact/patchprocess/diff-compile-javac-root.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/15751/artifact/patchprocess/diff-checkstyle-root.txt
|
|
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282154#comment-15282154 ] Sangjin Lee commented on HDFS-7959: --- [~kihwal], would you be able to update the patch for this so it can be reviewed for 2.8.0? > WebHdfs logging is missing on Datanode > -- > > Key: HDFS-7959 > URL: https://issues.apache.org/jira/browse/HDFS-7959 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Critical > Labels: BB2015-05-TBR > Attachments: HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch > > > After the conversion to netty, webhdfs requests are not logged on datanodes. > The existing jetty log only logs the non-webhdfs requests that come through > the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14511894#comment-14511894 ] Kihwal Lee commented on HDFS-7959: -- I tried LoggingHandler, but it was getting ALL events. We could implement our own and do filtering, but thought this is simpler. WebHdfs logging is missing on Datanode -- Key: HDFS-7959 URL: https://issues.apache.org/jira/browse/HDFS-7959 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Kihwal Lee Priority: Critical Attachments: HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch After the conversion to netty, webhdfs requests are not logged on datanodes. The existing jetty log only logs the non-webhdfs requests that come through the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509956#comment-14509956 ] Haohui Mai commented on HDFS-7959: -- The current approach looks okay. A cleaner approach might be to introduce a LoggingHandler which catches all the response in downstream of the netty pipeline. WebHdfs logging is missing on Datanode -- Key: HDFS-7959 URL: https://issues.apache.org/jira/browse/HDFS-7959 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Kihwal Lee Priority: Critical Attachments: HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch After the conversion to netty, webhdfs requests are not logged on datanodes. The existing jetty log only logs the non-webhdfs requests that come through the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503511#comment-14503511
]
Hadoop QA commented on HDFS-7959:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726592/HDFS-7959.patch
against trunk revision c17cd4f.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/10317//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10317//console
This message is automatically generated.
WebHdfs logging is missing on Datanode
--
Key: HDFS-7959
URL: https://issues.apache.org/jira/browse/HDFS-7959
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Attachments: HDFS-7959.patch, HDFS-7959.patch, HDFS-7959.patch
After the conversion to netty, webhdfs requests are not logged on datanodes.
The existing jetty log only logs the non-webhdfs requests that come through
the internal proxy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498457#comment-14498457 ] Haohui Mai commented on HDFS-7959: -- [~kihwal], is latest patch ready to be committed? WebHdfs logging is missing on Datanode -- Key: HDFS-7959 URL: https://issues.apache.org/jira/browse/HDFS-7959 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Kihwal Lee Priority: Critical Attachments: HDFS-7959.patch, HDFS-7959.patch After the conversion to netty, webhdfs requests are not logged on datanodes. The existing jetty log only logs the non-webhdfs requests that come through the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384098#comment-14384098 ] Kihwal Lee commented on HDFS-7959: -- I think that the right fix is to move delegation token from the URL to the header. Related: HADOOP-10880. We will make it happen for HDFS in a separate jira. WebHdfs logging is missing on Datanode -- Key: HDFS-7959 URL: https://issues.apache.org/jira/browse/HDFS-7959 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Kihwal Lee Priority: Critical Attachments: HDFS-7959.patch, HDFS-7959.patch After the conversion to netty, webhdfs requests are not logged on datanodes. The existing jetty log only logs the non-webhdfs requests that come through the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14383082#comment-14383082
]
Hadoop QA commented on HDFS-7959:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707593/HDFS-7959.patch
against trunk revision 61df1b2.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.ipc.TestRPC
The following test timeouts occurred in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestDFSClientExcludedNodes
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/10079//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10079//console
This message is automatically generated.
WebHdfs logging is missing on Datanode
--
Key: HDFS-7959
URL: https://issues.apache.org/jira/browse/HDFS-7959
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Attachments: HDFS-7959.patch, HDFS-7959.patch
After the conversion to netty, webhdfs requests are not logged on datanodes.
The existing jetty log only logs the non-webhdfs requests that come through
the internal proxy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382787#comment-14382787 ] Kihwal Lee commented on HDFS-7959: -- I see a couple of lines of unnecessary code. I will remove them in the next round. WebHdfs logging is missing on Datanode -- Key: HDFS-7959 URL: https://issues.apache.org/jira/browse/HDFS-7959 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Kihwal Lee Priority: Critical Attachments: HDFS-7959.patch, HDFS-7959.patch After the conversion to netty, webhdfs requests are not logged on datanodes. The existing jetty log only logs the non-webhdfs requests that come through the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380413#comment-14380413 ] Kihwal Lee commented on HDFS-7959: -- It is probably a good idea to sanitize the tokens in the request URI before logging. I will update the patch. WebHdfs logging is missing on Datanode -- Key: HDFS-7959 URL: https://issues.apache.org/jira/browse/HDFS-7959 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Kihwal Lee Priority: Critical Attachments: HDFS-7959.patch After the conversion to netty, webhdfs requests are not logged on datanodes. The existing jetty log only logs the non-webhdfs requests that come through the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380576#comment-14380576 ] Kihwal Lee commented on HDFS-7959: -- I think that the right fix is to move delegation token from the URL to the header. Agreed. Daryn was looking at making that change, but the rolling upgrade scenario makes it a bit complicated. Right now token in the request is very useful in correlating the requests to a job or to a client. If token is not part of the request URI, there should be something else for tracing and correlating requests. It could be a dummy param derived from client ID, etc. WebHdfs logging is missing on Datanode -- Key: HDFS-7959 URL: https://issues.apache.org/jira/browse/HDFS-7959 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Kihwal Lee Priority: Critical Attachments: HDFS-7959.patch After the conversion to netty, webhdfs requests are not logged on datanodes. The existing jetty log only logs the non-webhdfs requests that come through the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380427#comment-14380427 ] Haohui Mai commented on HDFS-7959: -- bq. Besides, not all the components in the current jetty log is actually useful (e.g. response code) The response code is useful in terms of debugging authentication issues (401 / 500 / etc.). Maybe it is a good idea to keep them? bq. It is probably a good idea to sanitize the tokens in the request URI before logging. I will update the patch. I think that the right fix is to move delegation token from the URL to the header. Logging requests seem really general and it might be a bad idea to mix it with the requirements of the application layers. WebHdfs logging is missing on Datanode -- Key: HDFS-7959 URL: https://issues.apache.org/jira/browse/HDFS-7959 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Kihwal Lee Priority: Critical Attachments: HDFS-7959.patch After the conversion to netty, webhdfs requests are not logged on datanodes. The existing jetty log only logs the non-webhdfs requests that come through the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371802#comment-14371802 ] Haohui Mai commented on HDFS-7959: -- Thanks for taking care of this [~kihwal]. One quick question: jetty server outputs the log in the NCSA format by default, does the code need to do the same thing here? WebHdfs logging is missing on Datanode -- Key: HDFS-7959 URL: https://issues.apache.org/jira/browse/HDFS-7959 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Kihwal Lee Priority: Critical Attachments: HDFS-7959.patch After the conversion to netty, webhdfs requests are not logged on datanodes. The existing jetty log only logs the non-webhdfs requests that come through the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372184#comment-14372184 ] Kihwal Lee commented on HDFS-7959: -- I don't think we need to. Besides, not all the components in the current jetty log is actually useful (e.g. response code). WebHdfs logging is missing on Datanode -- Key: HDFS-7959 URL: https://issues.apache.org/jira/browse/HDFS-7959 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Kihwal Lee Priority: Critical Attachments: HDFS-7959.patch After the conversion to netty, webhdfs requests are not logged on datanodes. The existing jetty log only logs the non-webhdfs requests that come through the internal proxy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode
[
https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370433#comment-14370433
]
Hadoop QA commented on HDFS-7959:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12705691/HDFS-7959.patch
against trunk revision 61a4c7f.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.tracing.TestTracing
The following test timeouts occurred in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestLeaseRecovery2
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9985//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9985//console
This message is automatically generated.
WebHdfs logging is missing on Datanode
--
Key: HDFS-7959
URL: https://issues.apache.org/jira/browse/HDFS-7959
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Attachments: HDFS-7959.patch
After the conversion to netty, webhdfs requests are not logged on datanodes.
The existing jetty log only logs the non-webhdfs requests that come through
the internal proxy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
