Regarding distributing the SSL files by us, I agree it is not ideal because we are taking the responsibility of distributing sensitive security material. But it would also be very convenient for certain users. Also as some users have pointed out there is a precedent: H2O does it as per http://docs.h2o.ai/h2o/latest-stable/h2o-docs/security.html#ssl-internode-security . It says "This will tell h2odriver to automatically generate all the necessary files and distribute them to all mappers. This distribution may be secure depending on your YARN configuration."
On Fri, Apr 21, 2017 at 12:08 AM, Priyanka Gugale <pri...@apache.org> wrote: > +1 for this support, it's important to let users use their own own keystore > files. > > Is it okay to distribute files inside our package along with jar/resources, > are there any security restrictions? Or we should use other medium like > HDFS or other shared file system to host these key files? > > -Priyanka > > > > On Fri, Apr 21, 2017 at 12:30 PM, Sanjay Pujare <san...@datatorrent.com> > wrote: > > > Currently StrAM supports only the default Hadoop SSL configuration > because > > it uses org.apache.hadoop.yarn.webapp.WebApps helper class which has the > > limitation of only using the default Hadoop SSL config that is read from > > Hadoop's ssl-server.xml resource file. Some users have run into a > situation > > where Hadoops' SSL keystore is not available on most cluster nodes or the > > Stram process doesn't have read access to the keystore even when present. > > So there is a need for the Stram to use a custom SSL keystore and > > configuration that does not suffer from these limitations. > > > > I am planning to fix this by first fixing WebApps in Hadoop and then > > enhancing Stram to use this new fix in Hadoop. I have already submitted a > > PR https://github.com/apache/hadoop/pull/213 to Hadoop and one of the > the > > Hadoop distributors has agreed to accept this fix so I expect it to be > > merged very soon. > > > > After that I will enhance Stram to accept the location of a custom > > ssl-server.xml file (supplied by the client via a DAG attribute or > > property) and use the values from that file to set up the config object > to > > be passed to WebApps which will end up using the custom SSL > configuration. > > I have already verified this approach in a prototype. > > > > We will also enhance the Apex client/launcher to distribute the custom > SSL > > files (XML and the keystore) along with the application jars/resources so > > the user does not need to pre-distribute the custom SSL files. > > > > Please let me know your comments. > > > > Sanjay > > >
