This is much cleaner approach to utilize both the features of BAM and CEP. Since the components for Batch layer and Speed layer are already implemented, it just need to get them properly placed. So the major concern is serving layer. IMO serving layer should provide drill down and data manipulation. From the user perspective writing hive scripts and Siddhi queries should ideally be a advance option and it would be nice to have predefined metrics which can be customized and run on top of compiled data and provide result (or views) .
Thanks, On Thu, Nov 27, 2014 at 4:34 PM, Maninda Edirisooriya <[email protected]> wrote: > This seems nice. But I am not sure how the precomputation works and how > the intermediate indexes are stored in Service Layer. > And also does the user have to write both hive scripts and Siddhi queries > for the same scenario? > > Thanks. > > > *Maninda Edirisooriya* > Senior Software Engineer > > *WSO2, Inc.*lean.enterprise.middleware. > > *Blog* : http://maninda.blogspot.com/ > *E-mail* : [email protected] > *Skype* : @manindae > *Twitter* : @maninda > > On Thu, Nov 27, 2014 at 2:25 PM, Dinithi De Silva <[email protected]> > wrote: > >> Hi all, >> >> Myself with Zamly ([email protected]) and Inshaf ([email protected]) are >> going to develop $subject as described below. >> >> *Introduction to Lambda architecture* >> The lambda architecture solves the problem of computing arbitrary >> functions on arbitrary data in real time by decomposing the problem into >> three layers: the batch layer, the serving layer, and the speed layer.[1] >> >> *Batch layer* >> The batch layer stores the master copy of the dataset and precomputes >> batch views on that master dataset. >> >> *Speed layer* >> The speed layer is needed to compensate for the high latency of the batch >> layer and it does this by computing real time views in Storm. >> >> *Serving layer* >> The serving layer is responsible for indexing and exposing the batch view >> and real time view, so that they can be queried for updated results. >> >> *Approach* >> We are planing to implement Lambda architecture using CEP and BAM. >> Please find the diagram for the architecture below. >> >> ​Feed backs and suggestions for improvements would be appreciated. >> >> [1] - >> http://jameskinley.tumblr.com/post/37398560534/the-lambda-architecture-principles-for >> >> -- >> *Dinithi De Silva* >> Associate Software Engineer, WSO2 Inc. >> m:+94716667655 | e:[email protected] | w: www.wso2.com >> | a: #20, Palm Grove, Colombo 03 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Malith Dhanushka Software Engineer - Data Technologies *WSO2, Inc. : wso2.com <http://wso2.com/>* *Mobile* : +94 716 506 693
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
