[jira] [Commented] (YARN-4562) YARN WebApp ignores the configuration passed to it for keystore settings
[
https://issues.apache.org/jira/browse/YARN-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003049#comment-16003049
]
Sanjay M Pujare commented on YARN-4562:
---
[~vvasudev] Thanks. There is a branch-2.8.1 where I didn't see a commit for
this https://github.com/apache/hadoop/tree/branch-2.8.1 . Could you backport
there as well? Thanks
> YARN WebApp ignores the configuration passed to it for keystore settings
>
>
> Key: YARN-4562
> URL: https://issues.apache.org/jira/browse/YARN-4562
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha1, 2.8.1
>
> Attachments: YARN-4562.patch
>
>
> The conf can be passed to WebApps builder, however the following code in
> WebApps.java that builds the HttpServer2 object:
> {noformat}
> if (httpScheme.equals(WebAppUtils.HTTPS_PREFIX)) {
> WebAppUtils.loadSslConfiguration(builder);
> }
> {noformat}
> ...results in loadSslConfiguration creating a new Configuration object; the
> one that is passed in is ignored, as far as the keystore/etc. settings are
> concerned. loadSslConfiguration has another overload with Configuration
> parameter that should be used instead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4562) YARN WebApp ignores the configuration passed to it for keystore settings
[
https://issues.apache.org/jira/browse/YARN-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002043#comment-16002043
]
Sanjay M Pujare commented on YARN-4562:
---
[~vvasudev] That's will be great. Even if there is no release planned for 2.6,
a patch from the current branch is possible and somebody could make one, I
suppose. Although I prefer 2.6 also to be included, if you can backport to 2.7
and 2.8 that will be good.
> YARN WebApp ignores the configuration passed to it for keystore settings
>
>
> Key: YARN-4562
> URL: https://issues.apache.org/jira/browse/YARN-4562
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: YARN-4562.patch
>
>
> The conf can be passed to WebApps builder, however the following code in
> WebApps.java that builds the HttpServer2 object:
> {noformat}
> if (httpScheme.equals(WebAppUtils.HTTPS_PREFIX)) {
> WebAppUtils.loadSslConfiguration(builder);
> }
> {noformat}
> ...results in loadSslConfiguration creating a new Configuration object; the
> one that is passed in is ignored, as far as the keystore/etc. settings are
> concerned. loadSslConfiguration has another overload with Configuration
> parameter that should be used instead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-4562) YARN WebApp ignores the configuration passed to it for keystore settings
[
https://issues.apache.org/jira/browse/YARN-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001733#comment-16001733
]
Sanjay M Pujare edited comment on YARN-4562 at 5/8/17 11:12 PM:
Hello, I would like to request a backport of this fix into 2.6 (and everything
in between). Please let me know if I need to clone this JIRA.
was (Author: sanjaypujare):
Hello, I would like to request a backport of this fix into 2.7 (and everything
between 2.7 and 2.9.0). Please let me know if I need to clone this JIRA.
> YARN WebApp ignores the configuration passed to it for keystore settings
>
>
> Key: YARN-4562
> URL: https://issues.apache.org/jira/browse/YARN-4562
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: YARN-4562.patch
>
>
> The conf can be passed to WebApps builder, however the following code in
> WebApps.java that builds the HttpServer2 object:
> {noformat}
> if (httpScheme.equals(WebAppUtils.HTTPS_PREFIX)) {
> WebAppUtils.loadSslConfiguration(builder);
> }
> {noformat}
> ...results in loadSslConfiguration creating a new Configuration object; the
> one that is passed in is ignored, as far as the keystore/etc. settings are
> concerned. loadSslConfiguration has another overload with Configuration
> parameter that should be used instead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4562) YARN WebApp ignores the configuration passed to it for keystore settings
[
https://issues.apache.org/jira/browse/YARN-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001733#comment-16001733
]
Sanjay M Pujare commented on YARN-4562:
---
Hello, I would like to request a backport of this fix into 2.7 (and everything
between 2.7 and 2.9.0). Please let me know if I need to clone this JIRA.
> YARN WebApp ignores the configuration passed to it for keystore settings
>
>
> Key: YARN-4562
> URL: https://issues.apache.org/jira/browse/YARN-4562
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: YARN-4562.patch
>
>
> The conf can be passed to WebApps builder, however the following code in
> WebApps.java that builds the HttpServer2 object:
> {noformat}
> if (httpScheme.equals(WebAppUtils.HTTPS_PREFIX)) {
> WebAppUtils.loadSslConfiguration(builder);
> }
> {noformat}
> ...results in loadSslConfiguration creating a new Configuration object; the
> one that is passed in is ignored, as far as the keystore/etc. settings are
> concerned. loadSslConfiguration has another overload with Configuration
> parameter that should be used instead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001729#comment-16001729 ] Sanjay M Pujare commented on YARN-6457: --- [~haibochen] Will do (I was about to ask that... :-) ). > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: YARN-6457.00.patch, YARN-6457.01.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001547#comment-16001547 ] Sanjay M Pujare commented on YARN-6457: --- [~haibochen] the Javadoc issue seems to have been fixed. Looks like we are good to go > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Attachments: YARN-6457.00.patch, YARN-6457.01.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay M Pujare updated YARN-6457: -- Attachment: YARN-6457.01.patch Attaching next patch from the Pull Request to initiate pre-commit check on the change > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Attachments: YARN-6457.00.patch, YARN-6457.01.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001414#comment-16001414 ] Sanjay M Pujare commented on YARN-6457: --- Will do... > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Attachments: YARN-6457.00.patch > > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000345#comment-16000345 ] Sanjay M Pujare commented on YARN-6457: --- [~haibochen] thanks. I have a new PR https://github.com/apache/hadoop/pull/219 . Pls let me know if you have questions/comments. Thanks > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995922#comment-15995922 ] Sanjay M Pujare commented on YARN-6457: --- Hi [~haibochen], I have verified the latest fix suggested by [~pra...@datatorrent.com] that it works. If you are okay, I will modify my PR and it can then be merged. Pls let me know > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[
https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991464#comment-15991464
]
Sanjay M Pujare commented on YARN-6457:
---
I agree with [~PramodSSImmaneni] with a small change (check for conf being
non-null)
{code}
public static HttpServer2.Builder loadSslConfiguration(
HttpServer2.Builder builder, Configuration conf) {
Configuration sslConf = new Configuration(false);
boolean needsClientAuth =
YarnConfiguration.YARN_SSL_CLIENT_HTTPS_NEED_AUTH_DEFAULT;
sslConf.addResource(YarnConfiguration.YARN_SSL_SERVER_RESOURCE_DEFAULT);
if (conf != null) {
sslConf.addResource(conf);
}
{code}
> Allow custom SSL configuration to be supplied in WebApps
>
>
> Key: YARN-6457
> URL: https://issues.apache.org/jira/browse/YARN-6457
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: webapp, yarn
>Reporter: Sanjay M Pujare
>Assignee: Sanjay M Pujare
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Currently a custom SSL store cannot be passed on to WebApps which forces the
> embedded web-server to use the default keystore set up in ssl-server.xml for
> the whole Hadoop cluster. There are cases where the Hadoop app needs to use
> its own/custom keystore.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay M Pujare updated YARN-6457: -- Comment: was deleted (was: [~haibochen] I understand the issue you have raised but I see couple of problems with your suggestion: - in the current code in WebAppUtils.java in the function loadSslConfiguration(HttpServer2.Builder, Configuration) it doesn't get the value of hadoop.ssl.server.conf key but the default value YarnConfiguration.YARN_SSL_SERVER_RESOURCE_DEFAULT (i.e. ssl-server.xml) is hardcoded in the loadResource call. Unless you are proposing fixing this, your suggestion won't work - the Hadoop app (in our case) reads the same set of config files as the other Hadoop components so it is going read the yarn-site.xml file and use the same value of hadoop.ssl.server.conf but of course the app can get the value from somewhere else and override it in the Confguration object before passing it to WebApps builder. But in that case doesn't it defeat the purpose of marking it final in yarn-site.xml? Also we have coded and tested our fix against the change in the PR so we would like to go ahead with this fix (assuming it passes all the reviews)) > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[
https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990109#comment-15990109
]
Sanjay M Pujare edited comment on YARN-6457 at 4/30/17 3:33 AM:
[~haibochen] I would have suggested the same code you have but to my
disappointment I noticed that Configuration.addResource(String) and
Configuration.addResource(Path) behave differently and
Configuration.addResource(String) didn't work for me :-( and that is why I
suggested my code. If there is no issue with backward compatibility I can
modify your code as follows:
{code}
if (sslConf == null)
{ sslConf = new Configuration(false); }
boolean needsClientAuth =
YarnConfiguration.YARN_SSL_CLIENT_HTTPS_NEED_AUTH_DEFAULT;
String sslConfResource = conf.get("hadoop.ssl.server.conf",
YarnConfiguration.YARN_SSL_SERVER_RESOURCE_DEFAULT);
sslConf.addResource(new Path(sslConfResource));
{code}
Also, is it okay to use "hadoop.ssl.server.conf" as it is without defining it
as a constant in YarnConfiguration?
was (Author: sanjaypujare):
[~haibochen] I would have suggested the same code you have but to my
disappointment I noticed that Configuration.addResource(String) and
Configuration.addResource(Path) behave differently and
Configuration.addResource(String) didn't work for me :-( and that is why I
suggested my code. If there is no issue with backward compatibility I can
modify your code as follows:
{code}
if (sslConf == null)
{ sslConf = new Configuration(false); }
boolean needsClientAuth =
YarnConfiguration.YARN_SSL_CLIENT_HTTPS_NEED_AUTH_DEFAULT;
String sslConfResource = conf.get("hadoop.ssl.server.conf",
YarnConfiguration.YARN_SSL_SERVER_RESOURCE_DEFAULT);
sslConf.addResource(new Path(sslConfResource));
{code}
> Allow custom SSL configuration to be supplied in WebApps
>
>
> Key: YARN-6457
> URL: https://issues.apache.org/jira/browse/YARN-6457
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: webapp, yarn
>Reporter: Sanjay M Pujare
>Assignee: Sanjay M Pujare
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Currently a custom SSL store cannot be passed on to WebApps which forces the
> embedded web-server to use the default keystore set up in ssl-server.xml for
> the whole Hadoop cluster. There are cases where the Hadoop app needs to use
> its own/custom keystore.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[
https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990109#comment-15990109
]
Sanjay M Pujare commented on YARN-6457:
---
[~haibochen] I would have suggested the same code you have but to my
disappointment I noticed that Configuration.addResource(String) and
Configuration.addResource(Path) behave differently and
Configuration.addResource(String) didn't work for me :-( and that is why I
suggested my code. If there is no issue with backward compatibility I can
modify your code as follows:
{code}
if (sslConf == null)
{ sslConf = new Configuration(false); }
boolean needsClientAuth =
YarnConfiguration.YARN_SSL_CLIENT_HTTPS_NEED_AUTH_DEFAULT;
String sslConfResource = conf.get("hadoop.ssl.server.conf",
YarnConfiguration.YARN_SSL_SERVER_RESOURCE_DEFAULT);
sslConf.addResource(new Path(sslConfResource));
{code}
> Allow custom SSL configuration to be supplied in WebApps
>
>
> Key: YARN-6457
> URL: https://issues.apache.org/jira/browse/YARN-6457
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: webapp, yarn
>Reporter: Sanjay M Pujare
>Assignee: Sanjay M Pujare
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Currently a custom SSL store cannot be passed on to WebApps which forces the
> embedded web-server to use the default keystore set up in ssl-server.xml for
> the whole Hadoop cluster. There are cases where the Hadoop app needs to use
> its own/custom keystore.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[
https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989356#comment-15989356
]
Sanjay M Pujare commented on YARN-6457:
---
[~haibochen] thinking more about it, I can see how this can be made to work. I
am thinking the following code in WebAppUtils.loadSslConfiguration(Builder,
Configuration) will do the trick:
if (sslConf == null) {
sslConf = new Configuration(false);
sslConf.addResource(YarnConfiguration.YARN_SSL_SERVER_RESOURCE_DEFAULT);
} else {
String customSslServerConf = sslConf.get("hadoop.ssl.server.conf"); //
TODO define the string in YarnConfigruation or elsewhere
if (customSslServerConf != null) {
sslConf.addResource(new Path(customSslServerConf));
} else {
sslConf.addResource(YarnConfiguration.YARN_SSL_SERVER_RESOURCE_DEFAULT);
}
}
Let me know if this is what you had in mind.
> Allow custom SSL configuration to be supplied in WebApps
>
>
> Key: YARN-6457
> URL: https://issues.apache.org/jira/browse/YARN-6457
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: webapp, yarn
>Reporter: Sanjay M Pujare
>Assignee: Sanjay M Pujare
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Currently a custom SSL store cannot be passed on to WebApps which forces the
> embedded web-server to use the default keystore set up in ssl-server.xml for
> the whole Hadoop cluster. There are cases where the Hadoop app needs to use
> its own/custom keystore.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988124#comment-15988124 ] Sanjay M Pujare commented on YARN-6457: --- [~haibochen] I understand the issue you have raised but I see couple of problems with your suggestion: - in the current code in WebAppUtils.java in the function loadSslConfiguration(HttpServer2.Builder, Configuration) it doesn't get the value of hadoop.ssl.server.conf key but the default value YarnConfiguration.YARN_SSL_SERVER_RESOURCE_DEFAULT (i.e. ssl-server.xml) is hardcoded in the loadResource call. Unless you are proposing fixing this, your suggestion won't work - the Hadoop app (in our case) reads the same set of config files as the other Hadoop components so it is going read the yarn-site.xml file and use the same value of hadoop.ssl.server.conf but of course the app can get the value from somewhere else and override it in the Confguration object before passing it to WebApps builder. But in that case doesn't it defeat the purpose of marking it final in yarn-site.xml? Also we have coded and tested our fix against the change in the PR so we would like to go ahead with this fix (assuming it passes all the reviews) > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988123#comment-15988123 ] Sanjay M Pujare commented on YARN-6457: --- [~haibochen] I understand the issue you have raised but I see couple of problems with your suggestion: - in the current code in WebAppUtils.java in the function loadSslConfiguration(HttpServer2.Builder, Configuration) it doesn't get the value of hadoop.ssl.server.conf key but the default value YarnConfiguration.YARN_SSL_SERVER_RESOURCE_DEFAULT (i.e. ssl-server.xml) is hardcoded in the loadResource call. Unless you are proposing fixing this, your suggestion won't work - the Hadoop app (in our case) reads the same set of config files as the other Hadoop components so it is going read the yarn-site.xml file and use the same value of hadoop.ssl.server.conf but of course the app can get the value from somewhere else and override it in the Confguration object before passing it to WebApps builder. But in that case doesn't it defeat the purpose of marking it final in yarn-site.xml? Also we have coded and tested our fix against the change in the PR so we would like to go ahead with this fix (assuming it passes all the reviews) > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15987581#comment-15987581 ] Sanjay M Pujare commented on YARN-6457: --- [~haibochen] just a reminder - do you see any issues with the PR? Is there anything you want me to do or is it ready to be merged? Thanks > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979541#comment-15979541 ] Sanjay M Pujare commented on YARN-6457: --- [~haibochen] I have the new PR https://github.com/apache/hadoop/pull/216 . Pls let me know, thanks! > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978106#comment-15978106 ] Sanjay M Pujare commented on YARN-6457: --- I can create the PR against master (if that's what you mean by "creating a patch against trunk"). Or could you point me to how to create this patch (contributor guidelines probably?) > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15976141#comment-15976141 ] Sanjay M Pujare commented on YARN-6457: --- [~haibochen] thanks. Couple of comments: - are you sure the "final" works the way you have described? The default ssl-server.xml could mark its property final in which case my value in my config object won't take effect (which is the impression I get when I see "Administrators typically define parameters as final in core-site.xml for values that user applications may not alter." at https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/conf/Configuration.html - the fix for YARN-4562 you mentioned seems to be in 2.9.0, 3.0.0-alpha1 whereas I need it in 2.7.1 or even 2.6.0. Can we incorporate the fix in these earlier releases? > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
[ https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962299#comment-15962299 ] Sanjay M Pujare commented on YARN-6457: --- Discussed this enhancement with [~kbotzum] and [~billjodonnell] > Allow custom SSL configuration to be supplied in WebApps > > > Key: YARN-6457 > URL: https://issues.apache.org/jira/browse/YARN-6457 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp, yarn >Reporter: Sanjay M Pujare > Original Estimate: 96h > Remaining Estimate: 96h > > Currently a custom SSL store cannot be passed on to WebApps which forces the > embedded web-server to use the default keystore set up in ssl-server.xml for > the whole Hadoop cluster. There are cases where the Hadoop app needs to use > its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6457) Allow custom SSL configuration to be supplied in WebApps
Sanjay M Pujare created YARN-6457: - Summary: Allow custom SSL configuration to be supplied in WebApps Key: YARN-6457 URL: https://issues.apache.org/jira/browse/YARN-6457 Project: Hadoop YARN Issue Type: Improvement Components: webapp, yarn Reporter: Sanjay M Pujare Currently a custom SSL store cannot be passed on to WebApps which forces the embedded web-server to use the default keystore set up in ssl-server.xml for the whole Hadoop cluster. There are cases where the Hadoop app needs to use its own/custom keystore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
