> On Feb. 1, 2016, 9:40 p.m., Venkat Ranganathan wrote: > > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java, line > > 211 > > <https://reviews.apache.org/r/42642/diff/2/?file=1227071#file1227071line211> > > > > I think you are doing this for ServiceLoader META-INF entries not > > properly merged. We should see if we can have some maven strategy to fix > > this > > > > How will this work other filesystem solutions like WASB? > > > > How are we passing the principal for remote Filesystems when the file > > is on a remote system? > > Venkatesan Ramachandran wrote: > The datasouce definition specifies the HDFS pathname for the jceks > provider -- it is inferred as HDFS for now and so the conf.set(). Regarding > WASB, I have not tried it, but I assume Sqoop can work with WASB OOB. If so, > this should also work since I used the CredentiaProviderFactory comes with > hadoop-common. > > Similarly, the jceks provider file has to be on the cluster where the > datasource entity exists. > > Venkat Ranganathan wrote: > You are right. Sqoop can work with HDFS and WASB OOB - and sqoop > actions when run via oozie have the jks file localized or accessed as a > HDFS/WASB location as the case may be. But here the FS implementation is > explicitly set for HDFS because of the implementation for other FS are not > available because of some issues. May be I don't fully grok the details, but > can't we make it work independent of the FS where the file is located? For > example, in a HDI installation, fs.defaultFS will be a WASB location - it > will not work there. It will be good to not restrict to HDFS like this as > there are other filesystems used also. > > Regarding the security, here there is a call to new Configuration(). It > will get the current Falcon cluster configuration with the principals from > the current cluster where Falcon server is running. What if the file is on > a remote cluster? Don't we have to make sure we use the filesystem and > principals related to that Filesystem that we pick up from the corresponding > cluster entity?
Let me check if it works without HDFS impl hard coded. if so, we should be ok with WASB. Not sure I understand the issue about file on a remote cluster. Let's talk about it offline. - Venkatesan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42642/#review117274 ----------------------------------------------------------- On Jan. 30, 2016, 3:01 a.m., Venkatesan Ramachandran wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42642/ > ----------------------------------------------------------- > > (Updated Jan. 30, 2016, 3:01 a.m.) > > > Review request for Falcon. > > > Repository: falcon-git > > > Description > ------- > > Support password alias for database import and export using java keystore > > > Diffs > ----- > > addons/designer/pom.xml 28a977a > client/src/main/resources/datasource-0.1.xsd beb82cc > common/pom.xml 8605358 > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193 > > common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java > e58b1e9 > > common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java > PRE-CREATION > common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea > > common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java > 9567eab > common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed > common/src/test/resources/config/datasource/datasource-file-0.2.xml > PRE-CREATION > falcon-regression/pom.xml 6bf3d50 > > oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java > f1fb337 > > oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java > 19fa931 > oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java > PRE-CREATION > oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java > 4892ecb > pom.xml a80822f > webapp/pom.xml 7ecfbaf > webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88 > webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf > webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f > webapp/src/test/resources/credential_provider.jceks PRE-CREATION > webapp/src/test/resources/datasource-template.xml fb7a329 > webapp/src/test/resources/datasource-template1.xml PRE-CREATION > webapp/src/test/resources/datasource-template2.xml PRE-CREATION > webapp/src/test/resources/datasource-template3.xml PRE-CREATION > webapp/src/test/resources/datasource-template4.xml PRE-CREATION > webapp/src/test/resources/feed-template3.xml a6c1d6b > webapp/src/test/resources/feed-template4.xml PRE-CREATION > > Diff: https://reviews.apache.org/r/42642/diff/ > > > Testing > ------- > > Unit tests and Manual end to end testing both on regular and secure cluster. > > > Thanks, > > Venkatesan Ramachandran > >
