> On Oct. 23, 2015, 8:34 p.m., Sowmya Ramesh wrote:
> > oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java,
> > line 133
> > <https://reviews.apache.org/r/38465/diff/5/?file=1102932#file1102932line133>
> >
> > I see that WorkflowExecutionArgs.DATASOURCE_NAME.getName is set with
> > "NONE" or "IGNORE". What is the logic here? When should be None and what
> > cases it will be IGNORE?
>
> Balu Vellanki wrote:
> @Sowmya - Look at Jira https://issues.apache.org/jira/browse/FALCON-1401.
> NONE is the right value to use when there is no input/output exists. IGNORE
> is supported for backwards compatibility, but should not be used going
> forward.
>
> Sowmya Ramesh wrote:
> @Balu - I know about that jira. I am asking in this jira why different
> values are used and what is the logic behind that if I am missing something.
> Also that is for feed names, I am not sure if we check
> DATASOURCE_NAME.getName() for NONE or IGNORE anywhere. Backward compatibilty
> makes sense in that jira as it spans multiple apache version. This is new
> code and hence not required to use both.
>
> Balu Vellanki wrote:
> Agree with you. IGNORE should not be used here.
>
> Venkatesan Ramachandran wrote:
> NONE is used. The post process xml has the following for datasource
> argument
> <arg>-datasource</arg>
> <arg>${datasource == 'NA' ? 'NONE' : datasource}</arg>
Now, IGNORE is used. The post process xml has the following for datasource
argument
<arg>-datasource</arg>
<arg>${datasource == 'NA' ? 'IGNORE' : datasource}</arg>
- Venkatesan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38465/#review103847
-----------------------------------------------------------
On Oct. 28, 2015, 9:55 p.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38465/
> -----------------------------------------------------------
>
> (Updated Oct. 28, 2015, 9:55 p.m.)
>
>
> Review request for Falcon, Ajay Yadava, Balu Vellanki, Peeyush Bishnoi,
> Sowmya Ramesh, and Venkat Ranganathan.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> FALCON-1459 : Ability to import from database
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/LifeCycle.java 58a2a6c
> client/src/main/java/org/apache/falcon/Tag.java beeb812
> client/src/main/java/org/apache/falcon/entity/v0/EntityType.java 0657124
> client/src/main/java/org/apache/falcon/metadata/RelationshipType.java
> f034772
> client/src/main/resources/datasource-0.1.xsd PRE-CREATION
> client/src/main/resources/feed-0.1.xsd 77b8f4b
> client/src/main/resources/jaxb-binding.xjb 6f1d6c7
> client/src/main/resources/mysql_database.xml PRE-CREATION
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java
> PRE-CREATION
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java ceefb17
> common/src/main/java/org/apache/falcon/entity/FeedHelper.java 5c252a8
>
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java
> PRE-CREATION
>
> common/src/main/java/org/apache/falcon/entity/parser/EntityParserFactory.java
> 5a33201
> common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java
> c5cfdd2
> common/src/main/java/org/apache/falcon/entity/store/ConfigurationStore.java
> 4dd1c68
> common/src/main/java/org/apache/falcon/entity/v0/EntityGraph.java bd4c6cf
>
> common/src/main/java/org/apache/falcon/entity/v0/EntityIntegrityChecker.java
> bd32852
>
> common/src/main/java/org/apache/falcon/metadata/EntityRelationshipGraphBuilder.java
> 8c3876c
>
> common/src/main/java/org/apache/falcon/metadata/InstanceRelationshipGraphBuilder.java
> f485764
> common/src/main/java/org/apache/falcon/metadata/MetadataMappingService.java
> 56fbde0
> common/src/main/java/org/apache/falcon/metadata/RelationshipLabel.java
> 5b312da
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java
> PRE-CREATION
> common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionArgs.java
> ac7140c
>
> common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionContext.java
> 9bfc51b
> common/src/test/java/org/apache/falcon/entity/AbstractTestBase.java aab9cee
> common/src/test/java/org/apache/falcon/entity/EntityTypeTest.java 640e87d
> common/src/test/java/org/apache/falcon/entity/FeedHelperTest.java 4020d36
>
> common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java
> PRE-CREATION
>
> common/src/test/java/org/apache/falcon/entity/parser/FeedEntityParserTest.java
> 905be68
> common/src/test/java/org/apache/falcon/entity/v0/EntityGraphTest.java
> 3863b11
> common/src/test/resources/config/datasource/datasource-0.1.xml PRE-CREATION
> common/src/test/resources/config/datasource/datasource-file-0.1.xml
> PRE-CREATION
> common/src/test/resources/config/datasource/datasource-invalid-0.1.xml
> PRE-CREATION
> common/src/test/resources/config/feed/feed-import-0.1.xml PRE-CREATION
> common/src/test/resources/config/feed/feed-import-exclude-fields-0.1.xml
> PRE-CREATION
> common/src/test/resources/config/feed/feed-import-invalid-0.1.xml
> PRE-CREATION
> common/src/test/resources/config/feed/feed-import-noargs-0.1.xml
> PRE-CREATION
> docs/src/site/twiki/EntitySpecification.twiki e07fe12
> docs/src/site/twiki/FalconCLI.twiki d994866
> messaging/src/main/java/org/apache/falcon/messaging/JMSMessageProducer.java
> 515562a
>
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java
> PRE-CREATION
>
> oozie/src/main/java/org/apache/falcon/oozie/FeedImportCoordinatorBuilder.java
> PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java
> PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java
> a04ae95
>
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java
> 3213a70
>
> oozie/src/main/java/org/apache/falcon/oozie/feed/FSReplicationWorkflowBuilder.java
> 0dc09ee
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java
> 9e55edf
>
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedRetentionWorkflowBuilder.java
> f953557
>
> oozie/src/main/java/org/apache/falcon/oozie/feed/HCatReplicationWorkflowBuilder.java
> 6e2a631
>
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java
> 3f49adb
> oozie/src/main/resources/action/feed/import-sqoop-database-action.xml
> PRE-CREATION
> oozie/src/main/resources/action/post-process.xml df0d286
> pom.xml 87c55e3
> webapp/pom.xml 9e4dc8f
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java
> PRE-CREATION
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java d067dee
> webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java
> PRE-CREATION
> webapp/src/test/resources/datasource-template.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38465/diff/
>
>
> Testing
> -------
>
> * Unit tests
> * Integration tests
> * Manual tests
> * Setup MySQL, create table and populate
> * Create datasource and feed entity with import policy in Falcon
> * Made sure the data lands up in the HDFS.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
