Hi, On Thu, Nov 27, 2014 at 6:36 PM, Inosh Goonewardena <[email protected]> wrote:
> Hi, > > On Thu, Nov 27, 2014 at 4:34 PM, Maninda Edirisooriya <[email protected]> > wrote: > >> This seems nice. But I am not sure how the precomputation works and how >> the intermediate indexes are stored in Service Layer. >> > > Basically the idea should be to store outputs from both batch and speed > layers in the serving layer so that service layer can respond to ad-hoc > queries using already processed data. According to the given architecture, > outputs from both BAM and CEP is going to be saved in CEP Event table which > can be a RDBMS table or in memory grid. > > For this use case RDBMS tables seem to be a better option over the in-memory tables. In addition to the RDBMS, Cassandra event tables can also be used through an existing extension. (however for the current version, this involves applying a patch to the siddhi core). Thanks Rajeev > And also does the user have to write both hive scripts and Siddhi queries >> for the same scenario? >> >> Thanks. >> >> >> *Maninda Edirisooriya* >> Senior Software Engineer >> >> *WSO2, Inc.*lean.enterprise.middleware. >> >> *Blog* : http://maninda.blogspot.com/ >> *E-mail* : [email protected] >> *Skype* : @manindae >> *Twitter* : @maninda >> >> On Thu, Nov 27, 2014 at 2:25 PM, Dinithi De Silva <[email protected]> >> wrote: >> >>> Hi all, >>> >>> Myself with Zamly ([email protected]) and Inshaf ([email protected]) are >>> going to develop $subject as described below. >>> >>> *Introduction to Lambda architecture* >>> The lambda architecture solves the problem of computing arbitrary >>> functions on arbitrary data in real time by decomposing the problem into >>> three layers: the batch layer, the serving layer, and the speed layer.[1] >>> >>> *Batch layer* >>> The batch layer stores the master copy of the dataset and precomputes >>> batch views on that master dataset. >>> >>> *Speed layer* >>> The speed layer is needed to compensate for the high latency of the >>> batch layer and it does this by computing real time views in Storm. >>> >>> *Serving layer* >>> The serving layer is responsible for indexing and exposing the batch >>> view and real time view, so that they can be queried for updated results. >>> >>> *Approach* >>> We are planing to implement Lambda architecture using CEP and BAM. >>> Please find the diagram for the architecture below. >>> >>> ​Feed backs and suggestions for improvements would be appreciated. >>> >>> [1] - >>> http://jameskinley.tumblr.com/post/37398560534/the-lambda-architecture-principles-for >>> >>> -- >>> *Dinithi De Silva* >>> Associate Software Engineer, WSO2 Inc. >>> m:+94716667655 | e:[email protected] | w: www.wso2.com >>> | a: #20, Palm Grove, Colombo 03 >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Regards, > > Inosh Goonewardena > Associate Technical Lead- WSO2 Inc. > Mobile: +94779966317 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Rajeev Sampath Senior Software Engineer WSO2, Inc.; http://www.wso2.com. Mobile: * +94716265766*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
