On 29 February 2016 at 01:28, Rajeev Sampath <[email protected]> wrote:
> Hi, > > I've checked the code more and the deployment aspects. Following are the > points and this is similar to Tharik's suggestion. > > - Current admin service can support batch scripts as it contains generic > functions > > - With some refactorings, Templating logic can also be reused > > - Most of the differences are in deployment aspect and it needs to be > abstracted out and implemented separately for Spark/ CEP etc.. > So we can have pluggable components to handle the deployment process. > WDYT? > +1 for this approach. It will improve re-usability and looks like a cleaner design. Thanks, Lasantha > > Thanks > Rajeev > > On Mon, Feb 29, 2016 at 9:12 AM, Tharik Kanaka <[email protected]> wrote: > >> Hi All, >> >> I also think that Template functionality can be generalised and reused >> from osgi service level by having services for core, batch real time and >> expose admin services in an uniform manner. Things like deployment of >> streams, execution plans can be separated in the osgi service level and >> applying configurations to template can be generalised as its doing a >> general regex operation. >> >> Regards, >> >> On Sun, Feb 28, 2016 at 2:35 AM, Lasantha Fernando <[email protected]> >> wrote: >> >>> Am also thinking that the templating part of the execution manager can >>> be abstracted out to a core component so that we can have the >>> language/script specific part in a pluggable manner. This might need some >>> effort initially, but will be easier to maintain in future IMHO. >>> >>> If the templating feature can be made generic enough, it might be useful >>> for other products of the platform as well. >>> >>> WDYT? >>> >>> Thanks, >>> Lasantha >>> On 26 Feb 2016 7:24 am, "Sriskandarajah Suhothayan" <[email protected]> >>> wrote: >>> >>>> Most of the backend code can be reused, so implementing something >>>> separate will being lot of duplicates. We have to refactor some code >>>> and make it as executionmanager.core + executionmanager.batch >>>> + executionmanager.realtime componets. >>>> >>>> Please arrange a design review next Monday. >>>> >>>> Regards >>>> Suho >>>> >>>> On Fri, Feb 26, 2016 at 4:36 PM, Nirmal Fernando <[email protected]> >>>> wrote: >>>> >>>>> +1 >>>>> >>>>> On Fri, Feb 26, 2016 at 4:34 PM, Mohanadarshan Vivekanandalingam < >>>>> [email protected]> wrote: >>>>> >>>>>> I think we need to consider few things here to find a better >>>>>> approach.. >>>>>> >>>>>> - Since we are planning to use same Execution manager for CEP then we >>>>>> need to maintain clear separation between realtime and batch part of it.. >>>>>> That means, we need two different components for this.. >>>>>> >>>>>> - We have an single UI component (Jaggary UI) for both Realtime and >>>>>> batch configuration.. Then have to decide whether to have different >>>>>> admin >>>>>> and core components.. >>>>>> >>>>>> - IMO, for the time being let's implement as separate components >>>>>> (since we already have realtime components) and decide on whether to >>>>>> merge >>>>>> or not later based on the outcome.. >>>>>> >>>>>> Does this make sense ? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Mohan >>>>>> >>>>>> >>>>>> On Fri, Feb 26, 2016 at 2:31 PM, Sriskandarajah Suhothayan < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> We will implement the batch & realtime part of the execution manager >>>>>>> as separate backend components. >>>>>>> And move the common aspects in the execution.manager.core. >>>>>>> >>>>>>> WDYT? >>>>>>> >>>>>>> Regards >>>>>>> Suho >>>>>>> >>>>>>> On Fri, Feb 26, 2016 at 2:25 PM, Rajeev Sampath <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Currently the execution manager supports configuring templates for >>>>>>>> CEP execution plans. We now have the requirement of supporting a >>>>>>>> similar >>>>>>>> template configuration functionality for Spark SQL batch analytics >>>>>>>> scripts >>>>>>>> as well. >>>>>>>> >>>>>>>> I have checked the possibility of extending our current execution >>>>>>>> manager to support batch scripts and following are some concerns. >>>>>>>> >>>>>>>> - Since execution plans and Spark scripts have different >>>>>>>> configurations and handled in different mannger by components, at the >>>>>>>> core, >>>>>>>> there will be separate implementations for handling these two. >>>>>>>> - But at admin service level, we can still expose these in a >>>>>>>> uniform manner in one service. >>>>>>>> - Maintainability issues that will arise when having these in one >>>>>>>> component. >>>>>>>> - From a user perspective these are quite similar, since it will be >>>>>>>> template configuration support for both and probably makes it easier to >>>>>>>> treat them uniformly for API analytics use cases etc. >>>>>>>> >>>>>>>> Hence, do we need to implment the batch script template feature >>>>>>>> separately or just extend the existing execution manager? >>>>>>>> Pls share your ideas on this. >>>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> Rajeev >>>>>>>> >>>>>>>> -- >>>>>>>> Rajeev Sampath >>>>>>>> Senior Software Engineer >>>>>>>> WSO2, Inc.; http://www.wso2.com. >>>>>>>> >>>>>>>> Mobile: >>>>>>>> * +94716265766 <%2B94716265766>* >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> *S. Suhothayan* >>>>>>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>>>>>> *WSO2 Inc. *http://wso2.com >>>>>>> * <http://wso2.com/>* >>>>>>> lean . enterprise . middleware >>>>>>> >>>>>>> >>>>>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>>>>>> http://suhothayan.blogspot.com/ >>>>>>> <http://suhothayan.blogspot.com/>twitter: >>>>>>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | >>>>>>> linked-in: >>>>>>> http://lk.linkedin.com/in/suhothayan >>>>>>> <http://lk.linkedin.com/in/suhothayan>* >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *V. Mohanadarshan* >>>>>> *Senior Software Engineer,* >>>>>> *Data Technologies Team,* >>>>>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>>>>> *lean.enterprise.middleware.* >>>>>> >>>>>> email: [email protected] >>>>>> phone:(+94) 771117673 >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & regards, >>>>> Nirmal >>>>> >>>>> Team Lead - WSO2 Machine Learner >>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>> Mobile: +94715779733 >>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> *S. Suhothayan* >>>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>>> *WSO2 Inc. *http://wso2.com >>>> * <http://wso2.com/>* >>>> lean . enterprise . middleware >>>> >>>> >>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>>> http://lk.linkedin.com/in/suhothayan >>>> <http://lk.linkedin.com/in/suhothayan>* >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> >> *Tharik Kanaka* >> >> WSO2, Inc |#20, Palm Grove, Colombo 03, Sri Lanka >> >> Email: [email protected] | Web: www.wso2.com >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Rajeev Sampath > Senior Software Engineer > WSO2, Inc.; http://www.wso2.com. > > Mobile: > * +94716265766 <%2B94716265766>* > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Lasantha Fernando* Senior Software Engineer - Data Technologies Team WSO2 Inc. http://wso2.com email: [email protected] mobile: (+94) 71 5247551
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
