Hi Sowmya,
Can you please put more details on Cons of Approach 1, how user cannot figure
out recipe processing status and thus adds to the operational pain.
Thanks,---Peeyush
On Thursday, 12 March 2015 1:49 PM, Ajay Yadav <[email protected]> wrote:
Hi Sowmya,
Can you please elaborate more on Approach 1? Does the client talk to oozie
in that case instead of Falcon or Falcon acts as a proxy for Oozie?
On Thu, Mar 12, 2015 at 1:22 PM, Srikanth Sundarrajan <[email protected]>
wrote:
> Hi Ying,
> JSP would still run in Falcon server JVM, so not sure if that is any
> different from implementing this natively inside the falcon server.
>
> Regards
> Srikanth Sundarrajan
>
> ----------------------------------------
> > Subject: Re: [DISCUSS]: Moving recipe cooking to server
> > From: [email protected]
> > To: [email protected]
> > Date: Thu, 12 Mar 2015 07:43:28 +0000
> >
> > Hi Sowmya, Srikanth, Venkatesh and Venkat,
> >
> > As far as I understand, recipe helps the user generate the process entity
> > xml and it looks best to keep it a client side logic, as what you did
> > before. The current question is how to run recipe logic on DR UI. To
> avoid
> > repeating all the recipe logic in JS, one choice you can consider is JSP.
> > JSP can be supported by Apache Tomcat server. If you package the recipe
> > code into JAR, it can be reused in JSP. JS and CSS can also be included
> in
> > the JSP page.
> >
> > Cheers,
> > Ying
> >
> >
> > On 3/11/15, 11:51 PM, "Srikanth Sundarrajan" <[email protected]>
> wrote:
> >
> >>Thanks Sowmya for capturing the discussion and also initiating this
> >>conversation to drive consensus around the design. I am inclined towards
> >>Approach 2.
> >>
> >>Regards
> >>Srikanth Sundarrajan
> >>
> >>----------------------------------------
> >>> Subject: [DISCUSS]: Moving recipe cooking to server
> >>> From: [email protected]
> >>> To: [email protected]
> >>> Date: Thu, 12 Mar 2015 05:17:16 +0000
> >>>
> >>> I had a discussion with Srikanth, Venkatesh and Venkat regarding making
> >>>recipe processing server side concept. I am sending out the summary of
> >>>the meeting and opening it for further discussion.
> >>>
> >>> Today Recipe cooking is a client side logic. Recipe also supports
> >>>extensions i.e. user can cook his/her own custom recipes.
> >>> Decision to make it client side logic was for the following reasons
> >>>
> >>> * Keep it isolated from falcon server
> >>>
> >>> * As custom recipe cooking is supported, user recipes can introduce
> >>>security vulnerabilities and also can bring down the falcon server
> >>>
> >>> Today, falcon provides HDFS DR recipe out of the box. There is a plan
> >>>to add UI support for DR in Falcon.
> >>> Rest API support cannot be added for recipe as it is client side
> >>>processing.
> >>> If the UI is pure java script[JS] then all the recipe cooking logic has
> >>>to be repeated in JS. This is not a feasible solution - if more recipes
> >>>are added say DR for hive, hbase and others, UI won't be extensible.
> >>>
> >>> For the above mentioned reasons Recipe should me made a server side
> >>>logic.
> >>> Provided recipes [recipes provided out of the box] can run as Falcon
> >>>process. Recipe cooking will be done in a new process if its custom
> >>>recipe [user code].
> >>>
> >>> For cooking of custom recipes, design proposed should consider handling
> >>>security implications, handling the issues where the custom user code
> >>>can bring down the Falcon server (trapping System.exit), handling class
> >>>path isolation.
> >>> Also it shouldn't in anyway destabilize the Falcon system.
> >>>
> >>> There are couple of approaches which was discussed
> >>>
> >>> Approach 1:
> >>> Custom Recipe cooking can be carried out separately in another Oozie
> >>>WF, this will ensure isolation. Oozie already has the ability to
> >>>schedule jobs as a user and handles all the security aspects of it.
> >>>
> >>> Pros:
> >>> - Provides isolation
> >>> - Piggyback on Oozie as it already provides the required functionality
> >>>
> >>> Cons:
> >>> - As recipe processing is done in different WF, from operations point
> >>>of view user cannot figure out recipe processing status and thus adds to
> >>>the operational pain.
> >>>
> >>> Approach 2:
> >>> Custom recipe cooking is done on the server side in a separate
> >>>independent process than Falcon process I.e. It runs in a different JVM.
> >>>Throttling should be added for how many recipe cooking processes can be
> >>>launched keeping in mind the machine configuration.
> >>>
> >>> Pros:
> >>> - Provides isolation as recipe cooking is done in a independent process
> >>>
> >>> Cons:
> >>> - Performance overhead as new process is launched for custom recipe
> >>>cooking
> >>> - Adds more complexity to the system
> >>>
> >>> Thoughts?
> >>>
> >>> Thanks,
> >>> Sowmya
> >>>
> >>>
> >>>
> >>>
> >>
> >
>
>
