+1 for the capability. Some clarifications.
1. Is creation of recipe in user space or would that need to be bundled in falcon as part of distribution ? How do users provide support artifacts for the recipe 2. How do users track execution of instances of a particular recipe. Am assuming you can create multiple instances of a single recipe 3. Would simple properties table suffice? Regards Srikanth Sundarrajan > Date: Thu, 7 Aug 2014 19:34:13 -0700 > Subject: [DISCUSS] Recipes in Falcon > From: [email protected] > To: [email protected] > > Hi folks, > > Falcon offers many services OOTB and caters to a wide array of use cases. > However, there has been many asks that does not fit the functionality > offered by Falcon. I'm proposing that we add recipes to Falcon which is > similar to recipes in Whirr and other management solutions such as puppet > and chef. > > Overview: > A recipe essentially is a static process template with parameterized > workflow to realize a specific use case. For example: > > * replicating directories from one HDFS cluster to another (not timed > partitions) > * replicating hive metadata (database, table, views, etc.) > * replicating between HDFS and Hive - either way > * anonymization of data based on schema > * data masking > * etc. > > Proposal: > Falcon provides a Process abstraction that encapsulates the configuration > for a user workflow with scheduling controls. All recipes can be modeled > as a Process with in Falcon which executes the user workflow > periodically. The process and its associated workflow are parameterized. > The user will provide a properties file with name value pairs that are > substituted by falcon before scheduling it. > > The CLI would look something like this: > falcon -recipe $recipe_name -properties $properties_file > > Thoughts? > > -- > Regards, > Venkatesh > > “Perfection (in design) is achieved not when there is nothing more to add, > but rather when there is nothing more to take away.” > - Antoine de Saint-Exupéry
