----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38465/#review99524 -----------------------------------------------------------
client/src/main/java/org/apache/falcon/entity/v0/EntityType.java (line 100) <https://reviews.apache.org/r/38465/#comment156450> Can you please clarify the statement. Feed and process entity can be scheduled but cluster can't be scheduled. common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java (line 42) <https://reviews.apache.org/r/38465/#comment156453> In this class, you are trying to fetch the Sqoop jar files and then loading into JVM to get set to classpath. Can you try any one of the following options: 1. Package required sqoop jar files with Falcon and then use it locally. 2. Try to use distributed cache for jar files. For the above you can explore the class SharedLibraryHostingService in Falcon. common/src/test/resources/config/datasource/datasource-0.1.xml (line 44) <https://reviews.apache.org/r/38465/#comment156457> Should not the driver classname and required jar file get handle internally instead of specifying explictly. To get the required jar file, you can think of using distributed cache or package required jar file with Falcon to be available locally. common/src/test/resources/config/datasource/datasource-invalid-0.1.xml (line 43) <https://reviews.apache.org/r/38465/#comment156454> Similar comment as above. oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java (line 86) <https://reviews.apache.org/r/38465/#comment156447> Sqoop command can be built using command element or arg elements. Is there any downside of using arg elements instead of command element. oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java (line 61) <https://reviews.apache.org/r/38465/#comment156445> Uncomment the statement or remove it, if it is not required. retention/src/test/java/org/apache/falcon/retention/FeedEvictorTest.java (line 374) <https://reviews.apache.org/r/38465/#comment156444> Uncomment the @Test . webapp/pom.xml (line 546) <https://reviews.apache.org/r/38465/#comment156455> Is localhost required. webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java (line 78) <https://reviews.apache.org/r/38465/#comment156443> Please remove the commented section, if it is not required. webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java (line 133) <https://reviews.apache.org/r/38465/#comment156441> Remove the commented statement. webapp/src/test/resources/datasource-template.xml (line 44) <https://reviews.apache.org/r/38465/#comment156442> See the above comments about handling the driver class name and required jar file in datasource xml file - Peeyush Bishnoi On Sept. 17, 2015, 7:40 p.m., Venkatesan Ramachandran wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/38465/ > ----------------------------------------------------------- > > (Updated Sept. 17, 2015, 7:40 p.m.) > > > Review request for Falcon, Ajay Yadava, Balu Vellanki, Peeyush Bishnoi, > Sowmya Ramesh, and Venkat Ranganathan. > > > Repository: falcon-git > > > Description > ------- > > FALCON-1459 : Ability to import from database > > > Diffs > ----- > > client/src/main/java/org/apache/falcon/LifeCycle.java 58a2a6c > client/src/main/java/org/apache/falcon/Tag.java beeb812 > client/src/main/java/org/apache/falcon/entity/v0/EntityType.java 0657124 > client/src/main/java/org/apache/falcon/metadata/RelationshipType.java > f034772 > client/src/main/resources/datasource-0.1.xsd PRE-CREATION > client/src/main/resources/feed-0.1.xsd 4ff8baa > client/src/main/resources/jaxb-binding.xjb f644f40 > client/src/main/resources/mysql_database.xml PRE-CREATION > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java > PRE-CREATION > common/src/main/java/org/apache/falcon/entity/EntityUtil.java 2f05b1f > common/src/main/java/org/apache/falcon/entity/FeedHelper.java 572923b > > common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java > PRE-CREATION > > common/src/main/java/org/apache/falcon/entity/parser/EntityParserFactory.java > 5a33201 > common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java > 992fc51 > common/src/main/java/org/apache/falcon/entity/store/ConfigurationStore.java > e27187b > common/src/main/java/org/apache/falcon/entity/v0/EntityGraph.java bd4c6cf > > common/src/main/java/org/apache/falcon/entity/v0/EntityIntegrityChecker.java > bd32852 > > common/src/main/java/org/apache/falcon/metadata/EntityRelationshipGraphBuilder.java > 8c3876c > common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java > PRE-CREATION > > common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionContext.java > 4454239 > common/src/test/java/org/apache/falcon/entity/AbstractTestBase.java 6179855 > common/src/test/java/org/apache/falcon/entity/EntityTypeTest.java 640e87d > common/src/test/java/org/apache/falcon/entity/FeedHelperTest.java c70cfcc > > common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java > PRE-CREATION > > common/src/test/java/org/apache/falcon/entity/parser/FeedEntityParserTest.java > d203b7c > common/src/test/java/org/apache/falcon/entity/v0/EntityGraphTest.java > 3863b11 > common/src/test/resources/config/datasource/datasource-0.1.xml PRE-CREATION > common/src/test/resources/config/datasource/datasource-invalid-0.1.xml > PRE-CREATION > common/src/test/resources/config/feed/feed-import-0.1.xml PRE-CREATION > common/src/test/resources/config/feed/feed-import-invalid-0.1.xml > PRE-CREATION > > falcon-regression/merlin-core/src/main/java/org/apache/falcon/regression/core/util/HiveAssert.java > 2a934b5 > > oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java > PRE-CREATION > > oozie/src/main/java/org/apache/falcon/oozie/FeedImportCoordinatorBuilder.java > PRE-CREATION > oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java > PRE-CREATION > oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java > 85f5330 > > oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java > 3213a70 > oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java > b819dee > oozie/src/main/resources/action/feed/import-sqoop-database-action.xml > PRE-CREATION > pom.xml 646de69 > retention/src/test/java/org/apache/falcon/retention/FeedEvictorTest.java > 72447da > webapp/pom.xml 828f7f5 > webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java > PRE-CREATION > webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java > 220e5a7 > webapp/src/test/java/org/apache/falcon/resource/TestContext.java f031137 > webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java > PRE-CREATION > webapp/src/test/resources/datasource-template.xml PRE-CREATION > webapp/src/test/resources/feed-template3.xml PRE-CREATION > > Diff: https://reviews.apache.org/r/38465/diff/ > > > Testing > ------- > > * Unit tests > * Integration tests > * Manual tests > * Setup MySQL, create table and populate > * Create datasource and feed entity with import policy in Falcon > * Made sure the data lands up in the HDFS. > > > Thanks, > > Venkatesan Ramachandran > >
