Hi guys,
It sounds good. I'm back after a "long" period on customers engagements
;)
I think server-side for recipe is interesting as it would allow ready to
use webapp stuff.
Regarding the packaging, it could be part of falcon-extra: in other
Apache projects (like Camel, ServiceMix, etc), when I have a non Apache
compliant "extension" (for instance in terms of license), I put it in
*-extra.
It's hosted on github, approved by Apache but not under the Apache
"scope" (about release process vote, license, etc).
Regards
JB
On 2014-08-20 00:17, Seetharam Venkatesh wrote:
There are a couple of questions before I create a jira.
Is this a client-side only concept?
Do we provide a server-side recipe tool as well so web apps can use
this
instead. The downside to using the client side api is that it expects
kinit
to have been done.
How do we handle packaging?
There could be recipe dependent jars, libs and this cannot be released
in
apache.
Thoughts?
On Fri, Aug 8, 2014 at 2:46 PM, Seetharam Venkatesh
<[email protected]
wrote:
On Thu, Aug 7, 2014 at 10:25 PM, Srikanth Sundarrajan
<[email protected]
> wrote:
+1 for the capability.
Some clarifications.
1. Is creation of recipe in user space or would that need to be
bundled
in falcon as part of distribution ?
The thinking is to have it bundled in falcon as part of the
distribution
to start with.
User space is quite nice but falcon recipe can look at the templates
dir
to find one and then substitute the properties. This might be possible
too.
How do users provide support artifacts for the recipe
All artifacts must be on HDFS.
2. How do users track execution of instances of a particular recipe.
Am
assuming you can create multiple instances of a single recipe
Yes, name can also be templated and parameterized.
3. Would simple properties table suffice?
Thats the thinking. Sowmya has prototyped DR running on a source
cluster.
Regards
Srikanth Sundarrajan
> Date: Thu, 7 Aug 2014 19:34:13 -0700
> Subject: [DISCUSS] Recipes in Falcon
> From: [email protected]
> To: [email protected]
>
> Hi folks,
>
> Falcon offers many services OOTB and caters to a wide array of use
cases.
> However, there has been many asks that does not fit the functionality
> offered by Falcon. I'm proposing that we add recipes to Falcon which is
> similar to recipes in Whirr and other management solutions such as
puppet
> and chef.
>
> Overview:
> A recipe essentially is a static process template with parameterized
> workflow to realize a specific use case. For example:
>
> * replicating directories from one HDFS cluster to another (not timed
> partitions)
> * replicating hive metadata (database, table, views, etc.)
> * replicating between HDFS and Hive - either way
> * anonymization of data based on schema
> * data masking
> * etc.
>
> Proposal:
> Falcon provides a Process abstraction that encapsulates the
configuration
> for a user workflow with scheduling controls. All recipes can be modeled
> as a Process with in Falcon which executes the user workflow
> periodically. The process and its associated workflow are parameterized.
> The user will provide a properties file with name value pairs that are
> substituted by falcon before scheduling it.
>
> The CLI would look something like this:
> falcon -recipe $recipe_name -properties $properties_file
>
> Thoughts?
>
> --
> Regards,
> Venkatesh
>
> “Perfection (in design) is achieved not when there is nothing more to
add,
> but rather when there is nothing more to take away.”
> - Antoine de Saint-Exupéry
--
Regards,
Venkatesh
“Perfection (in design) is achieved not when there is nothing more to
add,
but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry
