Hi, I've checked the code more and the deployment aspects. Following are the points and this is similar to Tharik's suggestion.
- Current admin service can support batch scripts as it contains generic functions - With some refactorings, Templating logic can also be reused - Most of the differences are in deployment aspect and it needs to be abstracted out and implemented separately for Spark/ CEP etc.. So we can have pluggable components to handle the deployment process. WDYT? Thanks Rajeev On Mon, Feb 29, 2016 at 9:12 AM, Tharik Kanaka <[email protected]> wrote: > Hi All, > > I also think that Template functionality can be generalised and reused > from osgi service level by having services for core, batch real time and > expose admin services in an uniform manner. Things like deployment of > streams, execution plans can be separated in the osgi service level and > applying configurations to template can be generalised as its doing a > general regex operation. > > Regards, > > On Sun, Feb 28, 2016 at 2:35 AM, Lasantha Fernando <[email protected]> > wrote: > >> Am also thinking that the templating part of the execution manager can be >> abstracted out to a core component so that we can have the language/script >> specific part in a pluggable manner. This might need some effort initially, >> but will be easier to maintain in future IMHO. >> >> If the templating feature can be made generic enough, it might be useful >> for other products of the platform as well. >> >> WDYT? >> >> Thanks, >> Lasantha >> On 26 Feb 2016 7:24 am, "Sriskandarajah Suhothayan" <[email protected]> >> wrote: >> >>> Most of the backend code can be reused, so implementing something >>> separate will being lot of duplicates. We have to refactor some code >>> and make it as executionmanager.core + executionmanager.batch >>> + executionmanager.realtime componets. >>> >>> Please arrange a design review next Monday. >>> >>> Regards >>> Suho >>> >>> On Fri, Feb 26, 2016 at 4:36 PM, Nirmal Fernando <[email protected]> >>> wrote: >>> >>>> +1 >>>> >>>> On Fri, Feb 26, 2016 at 4:34 PM, Mohanadarshan Vivekanandalingam < >>>> [email protected]> wrote: >>>> >>>>> I think we need to consider few things here to find a better >>>>> approach.. >>>>> >>>>> - Since we are planning to use same Execution manager for CEP then we >>>>> need to maintain clear separation between realtime and batch part of it.. >>>>> That means, we need two different components for this.. >>>>> >>>>> - We have an single UI component (Jaggary UI) for both Realtime and >>>>> batch configuration.. Then have to decide whether to have different admin >>>>> and core components.. >>>>> >>>>> - IMO, for the time being let's implement as separate components >>>>> (since we already have realtime components) and decide on whether to merge >>>>> or not later based on the outcome.. >>>>> >>>>> Does this make sense ? >>>>> >>>>> >>>>> Thanks, >>>>> Mohan >>>>> >>>>> >>>>> On Fri, Feb 26, 2016 at 2:31 PM, Sriskandarajah Suhothayan < >>>>> [email protected]> wrote: >>>>> >>>>>> We will implement the batch & realtime part of the execution manager >>>>>> as separate backend components. >>>>>> And move the common aspects in the execution.manager.core. >>>>>> >>>>>> WDYT? >>>>>> >>>>>> Regards >>>>>> Suho >>>>>> >>>>>> On Fri, Feb 26, 2016 at 2:25 PM, Rajeev Sampath <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Currently the execution manager supports configuring templates for >>>>>>> CEP execution plans. We now have the requirement of supporting a similar >>>>>>> template configuration functionality for Spark SQL batch analytics >>>>>>> scripts >>>>>>> as well. >>>>>>> >>>>>>> I have checked the possibility of extending our current execution >>>>>>> manager to support batch scripts and following are some concerns. >>>>>>> >>>>>>> - Since execution plans and Spark scripts have different >>>>>>> configurations and handled in different mannger by components, at the >>>>>>> core, >>>>>>> there will be separate implementations for handling these two. >>>>>>> - But at admin service level, we can still expose these in a uniform >>>>>>> manner in one service. >>>>>>> - Maintainability issues that will arise when having these in one >>>>>>> component. >>>>>>> - From a user perspective these are quite similar, since it will be >>>>>>> template configuration support for both and probably makes it easier to >>>>>>> treat them uniformly for API analytics use cases etc. >>>>>>> >>>>>>> Hence, do we need to implment the batch script template feature >>>>>>> separately or just extend the existing execution manager? >>>>>>> Pls share your ideas on this. >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> Rajeev >>>>>>> >>>>>>> -- >>>>>>> Rajeev Sampath >>>>>>> Senior Software Engineer >>>>>>> WSO2, Inc.; http://www.wso2.com. >>>>>>> >>>>>>> Mobile: >>>>>>> * +94716265766 <%2B94716265766>* >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> *S. Suhothayan* >>>>>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>>>>> *WSO2 Inc. *http://wso2.com >>>>>> * <http://wso2.com/>* >>>>>> lean . enterprise . middleware >>>>>> >>>>>> >>>>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>>>>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>>>>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | >>>>>> linked-in: >>>>>> http://lk.linkedin.com/in/suhothayan >>>>>> <http://lk.linkedin.com/in/suhothayan>* >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *V. Mohanadarshan* >>>>> *Senior Software Engineer,* >>>>> *Data Technologies Team,* >>>>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>>>> *lean.enterprise.middleware.* >>>>> >>>>> email: [email protected] >>>>> phone:(+94) 771117673 >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks & regards, >>>> Nirmal >>>> >>>> Team Lead - WSO2 Machine Learner >>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>> Mobile: +94715779733 >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> >>> *S. Suhothayan* >>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>> *WSO2 Inc. *http://wso2.com >>> * <http://wso2.com/>* >>> lean . enterprise . middleware >>> >>> >>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>> http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > > *Tharik Kanaka* > > WSO2, Inc |#20, Palm Grove, Colombo 03, Sri Lanka > > Email: [email protected] | Web: www.wso2.com > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Rajeev Sampath Senior Software Engineer WSO2, Inc.; http://www.wso2.com. Mobile: * +94716265766*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
