> On Feb. 1, 2016, 1:40 p.m., Venkat Ranganathan wrote: > > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java, line > > 211 > > <https://reviews.apache.org/r/42642/diff/2/?file=1227071#file1227071line211> > > > > I think you are doing this for ServiceLoader META-INF entries not > > properly merged. We should see if we can have some maven strategy to fix > > this > > > > How will this work other filesystem solutions like WASB? > > > > How are we passing the principal for remote Filesystems when the file > > is on a remote system? > > Venkatesan Ramachandran wrote: > The datasouce definition specifies the HDFS pathname for the jceks > provider -- it is inferred as HDFS for now and so the conf.set(). Regarding > WASB, I have not tried it, but I assume Sqoop can work with WASB OOB. If so, > this should also work since I used the CredentiaProviderFactory comes with > hadoop-common. > > Similarly, the jceks provider file has to be on the cluster where the > datasource entity exists.
You are right. Sqoop can work with HDFS and WASB OOB - and sqoop actions when run via oozie have the jks file localized or accessed as a HDFS/WASB location as the case may be. But here the FS implementation is explicitly set for HDFS because of the implementation for other FS are not available because of some issues. May be I don't fully grok the details, but can't we make it work independent of the FS where the file is located? For example, in a HDI installation, fs.defaultFS will be a WASB location - it will not work there. It will be good to not restrict to HDFS like this as there are other filesystems used also. Regarding the security, here there is a call to new Configuration(). It will get the current Falcon cluster configuration with the principals from the current cluster where Falcon server is running. What if the file is on a remote cluster? Don't we have to make sure we use the filesystem and principals related to that Filesystem that we pick up from the corresponding cluster entity? - Venkat ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42642/#review117274 ----------------------------------------------------------- On Jan. 29, 2016, 7:01 p.m., Venkatesan Ramachandran wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42642/ > ----------------------------------------------------------- > > (Updated Jan. 29, 2016, 7:01 p.m.) > > > Review request for Falcon. > > > Repository: falcon-git > > > Description > ------- > > Support password alias for database import and export using java keystore > > > Diffs > ----- > > addons/designer/pom.xml 28a977a > client/src/main/resources/datasource-0.1.xsd beb82cc > common/pom.xml 8605358 > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193 > > common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java > e58b1e9 > > common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java > PRE-CREATION > common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea > > common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java > 9567eab > common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed > common/src/test/resources/config/datasource/datasource-file-0.2.xml > PRE-CREATION > falcon-regression/pom.xml 6bf3d50 > > oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java > f1fb337 > > oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java > 19fa931 > oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java > PRE-CREATION > oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java > 4892ecb > pom.xml a80822f > webapp/pom.xml 7ecfbaf > webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88 > webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf > webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f > webapp/src/test/resources/credential_provider.jceks PRE-CREATION > webapp/src/test/resources/datasource-template.xml fb7a329 > webapp/src/test/resources/datasource-template1.xml PRE-CREATION > webapp/src/test/resources/datasource-template2.xml PRE-CREATION > webapp/src/test/resources/datasource-template3.xml PRE-CREATION > webapp/src/test/resources/datasource-template4.xml PRE-CREATION > webapp/src/test/resources/feed-template3.xml a6c1d6b > webapp/src/test/resources/feed-template4.xml PRE-CREATION > > Diff: https://reviews.apache.org/r/42642/diff/ > > > Testing > ------- > > Unit tests and Manual end to end testing both on regular and secure cluster. > > > Thanks, > > Venkatesan Ramachandran > >
