[
https://issues.apache.org/jira/browse/FALCON-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
karan kumar updated FALCON-1133:
--------------------------------
Description:
As falcon has a bunch of handcrafted xmls ,this results in manual errors such
as typos, giving bad arguments, xml errors etc while deploying a new job.
For validating the xmls as well as checking the job is getting the correct set
of configs, a mock run is needed in a full fledged setup of
falcon/hadoop/oozie. This I feel is a overkill.
I propose that we start work on falcon unit which will intialize
falcon/hadoop/oozie in local mode and run the job locally to test the configs
and also can be extended to validate input and output datesets. This will
really help in easily onboarding to falcon jobs as in a matter of minutes u can
test your process.
was:
As falcon has a bunch of handcrafted xmls ,this results in manual errors such
as typos, giving bad arguments, xml errors etc while deploying a new job.
For validating the xmls as well as checking the job is getting the correct set
of configs, a mock run is needed in a full fledged setup of
falcon/hadoop/oozie. This I feel is a overkill.
I propose that we start work on falcon unit which will intialize
falcon/hadoop/oozie in local mode and run the job locally to test the configs
and also can be extended to validate input and output datesets. This will
really help in easily unboarding to falcon jobs as in a matter of minutes u can
test your process.
> Adding Falcon Unit
> -------------------
>
> Key: FALCON-1133
> URL: https://issues.apache.org/jira/browse/FALCON-1133
> Project: Falcon
> Issue Type: New Feature
> Reporter: karan kumar
> Assignee: karan kumar
>
> As falcon has a bunch of handcrafted xmls ,this results in manual errors such
> as typos, giving bad arguments, xml errors etc while deploying a new job.
> For validating the xmls as well as checking the job is getting the correct
> set of configs, a mock run is needed in a full fledged setup of
> falcon/hadoop/oozie. This I feel is a overkill.
> I propose that we start work on falcon unit which will intialize
> falcon/hadoop/oozie in local mode and run the job locally to test the configs
> and also can be extended to validate input and output datesets. This will
> really help in easily onboarding to falcon jobs as in a matter of minutes u
> can test your process.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
