Venkatesh Seetharam created FALCON-634:
------------------------------------------
Summary: Add recipes in Falcon
Key: FALCON-634
URL: https://issues.apache.org/jira/browse/FALCON-634
Project: Falcon
Issue Type: Bug
Affects Versions: 0.6
Reporter: Venkatesh Seetharam
Falcon offers many services OOTB and caters to a wide array of use cases.
However, there has been many asks that does not fit the functionality offered
by Falcon. I'm proposing that we add recipes to Falcon which is similar to
recipes in Whirr and other management solutions such as puppet and chef.
Overview:
A recipe essentially is a static process template with parameterized workflow
to realize a specific use case. For example:
* replicating directories from one HDFS cluster to another (not timed
partitions)
* replicating hive metadata (database, table, views, etc.)
* replicating between HDFS and Hive - either way
* anonymization of data based on schema
* data masking
* etc.
Proposal:
Falcon provides a Process abstraction that encapsulates the configuration
for a user workflow with scheduling controls. All recipes can be modeled
as a Process with in Falcon which executes the user workflow
periodically. The process and its associated workflow are parameterized. The
user will provide a properties file with name value pairs that are substituted
by falcon before scheduling it.
This is a client side concept. The server does not know about a recipe but only
accepts the cooked recipe as a process entity.
The CLI would look something like this:
falcon -recipe $recipe_name -properties $properties_file
Recipes will reside inside addons (contrib) with source code and will have an
option to package 'em.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
