[
https://issues.apache.org/jira/browse/APEXCORE-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001657#comment-16001657
]
Sanjay M Pujare commented on APEXCORE-711:
------------------------------------------
Recapping my earlier analysis about eliminating dependencies on Hadoop/Yarn:
- In org.apache.hadoop.yarn.webapp.WebApps.Builder<T> the start(WebApp) method
does bulk of the work in creating the WebApp container object. The problematic
line in this method currently is
WebAppUtils.loadSslConfiguration(builder);
which does not pass (honor) the config object passed to
Builder.with(Configuration). The above line should be changed to
WebAppUtils.loadSslConfiguration(builder, conf);
which passes the required config object. Since a class static method
(WebAppUtils.loadSslConfiguration) is being called there is no way to get the
new behavior by subclassing a class and overriding a simple method. All of the
code in org.apache.hadoop.yarn.webapp.WebApps.Builder.start(WebApp) will need
to be copied in a new class on the Apex core side. BTW, precisely because of
this, https://issues.apache.org/jira/browse/YARN-4562 was created and fixed
(albeit in 2.9.x and not in any earlier version)
- with this we still have an unresolved issue in the current Yarn. The current
code in
org.apache.hadoop.yarn.webapp.util.WebAppUtils.loadSslConfiguration(Builder,
Configuration) is as follows (trimmed):
public static HttpServer2.Builder loadSslConfiguration(
HttpServer2.Builder builder, Configuration sslConf) {
if (sslConf == null) {
sslConf = new Configuration(false);
}
boolean needsClientAuth =
YarnConfiguration.YARN_SSL_CLIENT_HTTPS_NEED_AUTH_DEFAULT;
sslConf.addResource(YarnConfiguration.YARN_SSL_SERVER_RESOURCE_DEFAULT);
return ...
}
which has 2 major issues:
a) The caller can set SSL properties in the passed sslConf object but they
are overridden by the default ssl-server.xml file because of the
sslConf.addResource(...) call. There is no way to point to a different XML file
(because it uses the hardcoded YARN_SSL_SERVER_RESOURCE_DEFAULT value and not
the value of a property) which defeats the very purpose.
b) the only way to solve (a) above is to mark the value final in the passed
sslConf object in which case the default ssl-server.xml values won't be applied
even if they are final there.
Both of these are addressed by the changes in this JIRA.
> Support custom SSL keystore for the Stram REST API web service
> --------------------------------------------------------------
>
> Key: APEXCORE-711
> URL: https://issues.apache.org/jira/browse/APEXCORE-711
> Project: Apache Apex Core
> Issue Type: Improvement
> Reporter: Sanjay M Pujare
> Assignee: Sanjay M Pujare
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> Currently StrAM supports only the default Hadoop SSL configuration for the
> web-service because it uses org.apache.hadoop.yarn.webapp.WebApps helper
> class which has the limitation of only using the default Hadoop SSL config
> that is read from Hadoop's ssl-server.xml resource file. Some users have run
> into a situation where Hadoops' SSL keystore is not available on most cluster
> nodes or the Stram process doesn't have read access to the keystore even when
> present. So there is a need for the Stram to use a custom SSL keystore and
> configuration that does not suffer from these limitations.
> There is already a PR https://github.com/apache/hadoop/pull/213 to Hadoop to
> support this in Hadoop and it is in the process of getting merged soon.
> After that Stram needs to be enhanced (this JIRA) to accept the location of a
> custom ssl-server.xml file (supplied by the client via a DAG attribute) and
> use the values from that file to set up the config object to be passed to
> WebApps which will end up using the custom SSL configuration. This approach
> has already been verified in a prototype.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
