[
https://issues.apache.org/jira/browse/FALCON-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638164#comment-14638164
]
Pallavi Rao commented on FALCON-36:
-----------------------------------
[~ajayyadava], [~me.venkatr], basic questions:
Are these XSDs as per the definitions that are going to be defined by the
lifecycle framework? Or, will we revisit this later?
{code}
+ <cluster name="primaryCluster" type="source">
+ <validity start="2015-07-13T00:00Z" end="2016-03-31T00:00Z"/>
+ <retention limit="days(90)" action="delete"/>
+ <import>
+ <source name="mysql-db" tableName="simple" extract-type="full"
deltacolumn="timestamp" merge-type="snapshot"/>
{code}
Shouldn't this be something like the one below as lifecycle is going to
extensible and pluggable?
{code}
<cluster name="primaryCluster" type="source">
<validity start="2015-07-13T00:00Z" end="2016-03-31T00:00Z"/>
<lifecycle type = retention provider="..."
configuration="retention.xml"/>
<lifecycle type = import provider="..."
configuration="mysql_database.xml"/>
{code}
Also, are we introducing a new entity type called datasource?
> Ability to ingest data from databases
> -------------------------------------
>
> Key: FALCON-36
> URL: https://issues.apache.org/jira/browse/FALCON-36
> Project: Falcon
> Issue Type: Improvement
> Components: acquisition
> Affects Versions: 0.3
> Reporter: Venkatesh Seetharam
> Assignee: Venkat Ramachandran
> Attachments: FALCON-36.patch, FALCON-36.rebase.patch,
> FALCON-36.review.patch, Falcon Data Ingestion - Proposal.docx,
> falcon-36.xsd.patch.1
>
>
> Attempt to address data import from RDBMS into hadoop and export of data from
> Hadoop into RDBMS. The plan is to use sqoop 1.x to materialize data motion
> from/to RDBMS to/from HDFS. Hive will not be integrated in the first pass
> until Falcon has a first class integration with HCatalog.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
