[
https://issues.apache.org/jira/browse/FALCON-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067603#comment-15067603
]
pavan kumar kolamuri commented on FALCON-1686:
----------------------------------------------
Thanks [~ajayyadava] for clarifying on this, but their use case is only code
change not adding new data . In that case rerun should work if user updates the
jar. Yes with effective time even new data will be handled.
> Support for reprocessing
> ------------------------
>
> Key: FALCON-1686
> URL: https://issues.apache.org/jira/browse/FALCON-1686
> Project: Falcon
> Issue Type: Improvement
> Affects Versions: 0.7
> Reporter: Mass Dosage
>
> We have a number of ETL jobs which we schedule to run on a regular basis with
> Falcon. This works fine. However, we often have cases where we need to run
> the exact same jobs over past date ranges in order to reprocess data after a
> code change. There doesn't seem to be any easy way to do this in Falcon at
> the moment. Ideally we'd have a controlled way of saying "run this process
> for dates between X and Y". There should also be a way to control whether
> downstream processes are triggered by the data being reprocessed or not. In
> some cases you may want downstream jobs to also run on the new data but in
> other cases you might not.
> With Oozie, if one wants to reprocess data from any time in history, one can
> update the start & end-dates (using the job.properties file) and submit a new
> coordinator to run alongside the existing one. As the coordinator-ids are
> unique they do not clash. In Falcon, processes are defined by their readable
> name so one would need to update that in the process file directly.
> We are currently working around this issue by making a copy of the original
> Falcon process, giving it a different name and changing the dates. This isn't
> ideal and leads to a lot of XML duplication.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
