[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245286#comment-16245286 ] Zoran Dimitrijevic commented on HDFS-12052: --- Thank you! > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245268#comment-16245268 ] Zoran Dimitrijevic commented on HDFS-12052: --- Sure. But I think it was exactly the same fix as for 3+. I did it for Altiscale 2.7+ branch. What do you want me to do? > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12193) Fix style issues in HttpFS tests
[ https://issues.apache.org/jira/browse/HDFS-12193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-12193: -- Attachment: HDFS-12193.patch > Fix style issues in HttpFS tests > > > Key: HDFS-12193 > URL: https://issues.apache.org/jira/browse/HDFS-12193 > Project: Hadoop HDFS > Issue Type: Improvement > Components: httpfs >Affects Versions: 3.0.0-beta1 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic >Priority: Trivial > Attachments: HDFS-12193.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > While refactoring httpfs tests for HDFS-12052 I've noticed many style issues > that are easy to fix, but should not be fixed in the same patch when we are > fixing the bug in the code. > I've been asked by at least two committers to create a separate patch which > will only cover these trivial style fixes. So, here it is. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12193) Fix style issues in HttpFS tests
Zoran Dimitrijevic created HDFS-12193: - Summary: Fix style issues in HttpFS tests Key: HDFS-12193 URL: https://issues.apache.org/jira/browse/HDFS-12193 Project: Hadoop HDFS Issue Type: Improvement Components: httpfs Affects Versions: 3.0.0-beta1 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial While refactoring httpfs tests for HDFS-12052 I've noticed many style issues that are easy to fix, but should not be fixed in the same patch when we are fixing the bug in the code. I've been asked by at least two committers to create a separate patch which will only cover these trivial style fixes. So, here it is. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081560#comment-16081560 ] Zoran Dimitrijevic commented on HDFS-12052: --- [~jzhuge] thanks. Removed unnecessary import and it seems it's finally all good. Please submit the patch when you have time. Cheers. > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-12052: -- Attachment: HDFS-12052.07.patch > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch, HDFS-12052.07.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081472#comment-16081472 ] Zoran Dimitrijevic commented on HDFS-12052: --- I run dev-support/bin/test-patch --dirty-workspace --plugins=checkstyle,maven HDFS-12052.06.patch and it was +1. So, how do I see which new style offense still exists in this patch? | Vote | Subsystem | Runtime | Comment | +1 |mvninstall | 9m 48s | trunk passed | +1 |checkstyle | 0m 13s | trunk passed | +1 |checkstyle | 0m 9s | the patch passed | || 10m 21s | > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-12052: -- Attachment: HDFS-12052.06.patch OK, so I am using the const which I made package private and corrected it to httpfs.ssl.enabled. [~jzhuge] I have removed unrelated style fixes (about 50 of those) from this patch. However, we should fix them and I can fix them in a different patch - because it's really hard to refactor code when even a simple test file contains 60 style issues that are trivial to fix... All good now? > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch, HDFS-12052.06.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-12052: -- Attachment: HDFS-12052.05.patch Removed unrelated style fixes from all but tests. I'll upload one more patch which doesn't have any unrelated style fixes even in that file if needed. Can you please comment about hadoop prefix and whether we want to fix that in this patch or in a different patch? > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch, > HDFS-12052.05.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12052) Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080797#comment-16080797 ] Zoran Dimitrijevic commented on HDFS-12052: --- Sure [~jzhuge]. I had to reformat the tests which are then detected as new code by style checks - I'll remove all style fixes unrelated to test refactoring. You can then review the new patch which will contain these fixes so that we don't waste this opportunity to clean up the style as well? > Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12052) Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-12052: -- Attachment: HDFS-12052.04.patch > Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch, HDFS-12052.04.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12052) Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-12052: -- Attachment: HDFS-12052.03.patch > Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch, HDFS-12052.03.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12052) Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-12052: -- Attachment: HDFS-12052.02.patch Cleaned up few style issues to make style check happy. > Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch, > HDFS-12052.02.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12052) Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-12052: -- Attachment: HDFS-12052.01.patch Do we want to use httpfs.ssl.enabled or hadoop.httpfs.ssl.enabled ? I searched over git repo, and it seems we set one in hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/libexec/shellprofile.d/hadoop-httpfs.sh and the other one in hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml > Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic >Assignee: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch, HDFS-12052.01.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12052) Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-12052: -- Status: Patch Available (was: Open) > Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 3.0.0-alpha3, 2.7.3 >Reporter: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12052) Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS
[ https://issues.apache.org/jira/browse/HDFS-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-12052: -- Attachment: HDFS-12052.00.patch This simple patch fixes the issue. I'll add tests soon. > Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS > > > Key: HDFS-12052 > URL: https://issues.apache.org/jira/browse/HDFS-12052 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs, webhdfs >Affects Versions: 2.7.3, 3.0.0-alpha3 >Reporter: Zoran Dimitrijevic > Attachments: HDFS-12052.00.patch > > > When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation > tokens. > Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of > whether ssl is enabled or not. If clients directly connect to renew tokens > (for example, hdfs dfs) all works because httpfs doesn't check whether token > kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to > renew the token for the job (for example, when running hadoop distcp). Since > DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and > fails. > I've tested a simple patch which I'll upload to this jira, and it fixes this > issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12052) Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS
Zoran Dimitrijevic created HDFS-12052: - Summary: Set "SWEBHDFS delegation" as DT kind if ssl is enabled in HttpFS Key: HDFS-12052 URL: https://issues.apache.org/jira/browse/HDFS-12052 Project: Hadoop HDFS Issue Type: Bug Components: httpfs, webhdfs Affects Versions: 3.0.0-alpha3, 2.7.3 Reporter: Zoran Dimitrijevic When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation tokens. Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of whether ssl is enabled or not. If clients directly connect to renew tokens (for example, hdfs dfs) all works because httpfs doesn't check whether token kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to renew the token for the job (for example, when running hadoop distcp). Since DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and fails. I've tested a simple patch which I'll upload to this jira, and it fixes this issue (hadoop distcp works). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9612) DistCp worker threads are not terminated after jobs are done.
[ https://issues.apache.org/jira/browse/HDFS-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15095172#comment-15095172 ] Zoran Dimitrijevic commented on HDFS-9612: -- There is a change from Log to slf4j: -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; I don't know what is the reason for this change (but I'm sure there is a reason since distcp is an old codebase), but it is unrelated to the core change (InterruptedException change). If someone needs to cherry-pick this patch to older version of hadoop, then maybe they don't want to change logger. But, this is up to hadoop committer to decide - alternative is to make another patch which goes through all distcp .java files and changes loggers. > DistCp worker threads are not terminated after jobs are done. > - > > Key: HDFS-9612 > URL: https://issues.apache.org/jira/browse/HDFS-9612 > Project: Hadoop HDFS > Issue Type: Bug > Components: distcp >Affects Versions: 2.8.0 >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang > Attachments: HDFS-9612.001.patch, HDFS-9612.002.patch, > HDFS-9612.003.patch, HDFS-9612.004.patch, HDFS-9612.005.patch, > HDFS-9612.006.patch > > > In HADOOP-11827, a producer-consumer style thread pool was introduced to > parallelize the task of listing files/directories. > We have a use case where a distcp job is run during the commit phase of a MR2 > job. However, it was found distcp does not terminate ProducerConsumer thread > pools properly. Because threads are not terminated, those MR2 jobs never > finish. > In a more typical use case where distcp is run as a standalone job, those > threads are terminated forcefully when the java process is terminated. So > these leaked threads did not become a problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9612) DistCp worker threads are not terminated after jobs are done.
[
https://issues.apache.org/jira/browse/HDFS-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090835#comment-15090835
]
Zoran Dimitrijevic commented on HDFS-9612:
--
168:
try {
work = inputQueue.take();
} catch (InterruptedException e) {
LOG.debug("Interrupted while waiting for request from inputQueue.");
// if interrupt is triggered by shutdown(), terminate the thread
// otherwise, attempt to take again
Thread.currentThread().interrupt();
return;
}
I don't see how you distinguish between shutdown and other interrupts? How I
read this code, you actually stop the thread using return every time thread
gets any InterruptedException. Whether it is possible to get this exception for
any reason except calling shutdown, I am not 100% certain - that is why I
protected this thread (but apparently failed to realize shutdown is not able to
complete). What you want is to let thread execute shutdown() but continue
working if it is interrupted while waiting to take() from input queue.
Give me a day to find time to check how to do this, or explain me how this
return does not get out of run() method and finishes the thread. Or maybe it
does work if you simply remove "return" from your patch?
> DistCp worker threads are not terminated after jobs are done.
> -
>
> Key: HDFS-9612
> URL: https://issues.apache.org/jira/browse/HDFS-9612
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: distcp
>Affects Versions: 2.8.0
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
> Attachments: HDFS-9612.001.patch, HDFS-9612.002.patch,
> HDFS-9612.003.patch, HDFS-9612.004.patch, HDFS-9612.005.patch,
> HDFS-9612.006.patch
>
>
> In HADOOP-11827, a producer-consumer style thread pool was introduced to
> parallelize the task of listing files/directories.
> We have a use case where a distcp job is run during the commit phase of a MR2
> job. However, it was found distcp does not terminate ProducerConsumer thread
> pools properly. Because threads are not terminated, those MR2 jobs never
> finish.
> In a more typical use case where distcp is run as a standalone job, those
> threads are terminated forcefully when the java process is terminated. So
> these leaked threads did not become a problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9612) DistCp worker threads are not terminated after jobs are done.
[ https://issues.apache.org/jira/browse/HDFS-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090851#comment-15090851 ] Zoran Dimitrijevic commented on HDFS-9612: -- OK, so I read a bit more about this: http://www.ibm.com/developerworks/library/j-jtp05236/ >From what I see, the only way to get InterruptedException is by some other >thread sending it - so what you do here is correct (sorry for my C++ mind not >really understanding what java expects causing this issue). LGTM. Only suggestion I might have is that logging change is not related to this fix, which might make downporting this patch a bit harder. But, it is up to Hadoop committers to make that decision. Sorry for not properly shutting down these threads after they are not needed anymore. > DistCp worker threads are not terminated after jobs are done. > - > > Key: HDFS-9612 > URL: https://issues.apache.org/jira/browse/HDFS-9612 > Project: Hadoop HDFS > Issue Type: Bug > Components: distcp >Affects Versions: 2.8.0 >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang > Attachments: HDFS-9612.001.patch, HDFS-9612.002.patch, > HDFS-9612.003.patch, HDFS-9612.004.patch, HDFS-9612.005.patch, > HDFS-9612.006.patch > > > In HADOOP-11827, a producer-consumer style thread pool was introduced to > parallelize the task of listing files/directories. > We have a use case where a distcp job is run during the commit phase of a MR2 > job. However, it was found distcp does not terminate ProducerConsumer thread > pools properly. Because threads are not terminated, those MR2 jobs never > finish. > In a more typical use case where distcp is run as a standalone job, those > threads are terminated forcefully when the java process is terminated. So > these leaked threads did not become a problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Status: Open (was: Patch Available) i'll resubmit the same patch since apache build system seems to generate unrelated issues. Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Status: Patch Available (was: Open) Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Attachment: HDFS-7232.patch Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Status: Open (was: Patch Available) Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Attachment: (was: HDFS-7232.patch) Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Status: Patch Available (was: Open) Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Status: Open (was: Patch Available) Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Attachment: (was: HDFS-7232.patch) Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Attachment: HDFS-7232.patch Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Status: Patch Available (was: Open) Resubmitting the patch created with git diff --no-prefix. Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Attachment: HDFS-7232.patch Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic updated HDFS-7232: - Affects Version/s: 3.0.0 Status: Patch Available (was: Open) This is a simple patch for audit logs. I'm not sure if it'll require unit-tests, but for now I don't have them. Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0 Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Attachments: HDFS-7232.patch Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-6884) Include the hostname in HTTPFS log filenames
[
https://issues.apache.org/jira/browse/HDFS-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoran Dimitrijevic updated HDFS-6884:
-
Attachment: HDFS-6884.patch
Include the hostname in HTTPFS log filenames
Key: HDFS-6884
URL: https://issues.apache.org/jira/browse/HDFS-6884
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.5.0
Reporter: Andrew Wang
Assignee: Alejandro Abdelnur
Attachments: HDFS-6884.patch
It'd be good to include the hostname in the httpfs log filenames. Right now
we have httpfs.log and httpfs-audit.log, it'd be nice to have e.g.
httpfs-${hostname}.log.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-6884) Include the hostname in HTTPFS log filenames
[
https://issues.apache.org/jira/browse/HDFS-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoran Dimitrijevic updated HDFS-6884:
-
Status: Patch Available (was: Open)
Include the hostname in HTTPFS log filenames
Key: HDFS-6884
URL: https://issues.apache.org/jira/browse/HDFS-6884
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.5.0
Reporter: Andrew Wang
Assignee: Alejandro Abdelnur
Attachments: HDFS-6884.patch
It'd be good to include the hostname in the httpfs log filenames. Right now
we have httpfs.log and httpfs-audit.log, it'd be nice to have e.g.
httpfs-${hostname}.log.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HDFS-7232) Populate hostname in httpfs audit log
Zoran Dimitrijevic created HDFS-7232: Summary: Populate hostname in httpfs audit log Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Reporter: Zoran Dimitrijevic Priority: Trivial Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HDFS-7232) Populate hostname in httpfs audit log
[ https://issues.apache.org/jira/browse/HDFS-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoran Dimitrijevic reassigned HDFS-7232: Assignee: Zoran Dimitrijevic Populate hostname in httpfs audit log - Key: HDFS-7232 URL: https://issues.apache.org/jira/browse/HDFS-7232 Project: Hadoop HDFS Issue Type: Bug Reporter: Zoran Dimitrijevic Assignee: Zoran Dimitrijevic Priority: Trivial Currently httpfs audit logs do not log the request's IP address. Since they use hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-log4j.properties which already contains hostname, it would be nice to add code to populate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
