Daya, good chart. what is the percentage increase in performance? 50% faster? without a tick mark and value near the green line, hard to determine.
On Wed, Nov 20, 2013 at 8:33 AM, Daya Attapattu <[email protected]> wrote: > Here is a casual comparison between CEP 2.1.0 and CEP 3.0.0. > > It is casual because: > 1. I ran the tests in background while using the laptop for other things. > The load was not uniform. > 2. The throughput was measured from input to CEP with a blocking queue. > Initially the queue > was empty and throughput recorded is higher than the true rate while > the queue is filling. (But > this applies to both CEP versions and the comparison is fair.) > > The query used is: > from inStream#window.Time(n seconds) > select age, avg(age) as avgAge, count(age) as countAge > insert into outStream; > > n = 2, 4, 8, 16, 32, 64, 128 > > (CEP-2.1.0 threw "out of memory" at n = 256 and default heap settings.) > > - Daya > > > > > > On Tue, Nov 19, 2013 at 7:46 PM, Rebecca Hurst <[email protected]> wrote: > >> Hi Mohan, Sriskandarajah, Lasantha & Anjana, >> >> Thanks so much for the descriptions. These are very helpful! >> >> I will get started on the press release, and then if we can add any >> performance numbers, I can insert them later. >> >> Cheers, >> Rebecca >> >> >> On Tue, Nov 19, 2013 at 12:43 AM, Mohanadarshan Vivekanandalingam < >> [email protected]> wrote: >> >>> Hi Isabelle, >>> >>> We don't have any comparison performance numbers with us at the moment >>> (CEP 2.1.0 vs 3.0.0).. But Daya is doing some performance analysis >>> regarding CEP 3.0.0, we'll try to get some information once he completed.. >>> >>> Regards, >>> Mohan >>> >>> >>> On Tue, Nov 19, 2013 at 1:26 PM, Isabelle Mauny <[email protected]>wrote: >>> >>>> All, >>>> >>>> Do you have any numbers ? :) >>>> >>>> i.e. was performance increased by X % ? Has TPS increased in our tests >>>> ? Numbers help illustrate all the points below and give a point of >>>> reference vs. previous versions AND competitors. >>>> >>>> Thanks, >>>> Isabelle. >>>> >>>> >>>> ------ >>>> Isabelle Mauny >>>> Director, Product Management; WSO2, Inc.; http://wso2.com/ >>>> email: [email protected] <[email protected]> - mobile: +34 616050684 >>>> >>>> >>>> On Tue, Nov 19, 2013 at 7:31 AM, Lasantha Fernando >>>> <[email protected]>wrote: >>>> >>>>> Hi Rebecca, >>>>> >>>>> To add to what Mohan has already explained,... >>>>> >>>>> >>>>> On 19 November 2013 11:26, Mohanadarshan Vivekanandalingam < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Rebecca, >>>>>> >>>>>> Please find my comments below inline for CEP 3.0.0 >>>>>> >>>>>> >>>>>> >>>>>>> *CEP 3.0 *1) It underwent a significant architectural change. The >>>>>>> "what’s new versus old" is not explained in the documentation. What do >>>>>>> we >>>>>>> want to say about the architectural change, and why is it significant >>>>>>> for >>>>>>> users? >>>>>>> >>>>>> >>>>>> CEP 3.0.0 is a complete re-write of WSO2 CEP which reduces the tight >>>>>> coupling between the components and able to use these separate modules in >>>>>> other WSO2 products. >>>>>> >>>>>> >>>>>>> >>>>>>> 2) Enhanced Siddhi Capabilities >>>>>>> >>>>>>> - Support for Siddhi query partitioning and event tables – what >>>>>>> is the purpose of this? >>>>>>> >>>>>>> Partition a query based on some filter and processing happens at >>>>>> each individual partition separately. It increases the performance when >>>>>> processing events for certain queries. >>>>>> >>>>> The advantage over creating partitions statically with separate >>>>> queries is that, with partitioning, new partitions will be created >>>>> dynamically based on the incoming events to CEP. >>>>> >>>>> Event tables - Allows users to store, retrieve and process events in a >>>>> database table like structure. Currently CEP has support for 'In-Memory >>>>> database' event tables and 'Relational database' event tables. >>>>> >>>>>> >>>>>>> - >>>>>>> - New event adaptor manager components to handle broker manager >>>>>>> related configurations – what is the purpose of this? >>>>>>> >>>>>>> New Event adaptor manager components are replacement for the broker >>>>>> configuration which are in older CEP versions. New design gives more >>>>>> straight forward way of creating event adaptors and helps to write custom >>>>>> event adaptors quickly. And event adaptors are clearly divided as input >>>>>> and >>>>>> output to avoid confusions when using adaptors. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - Support for Siddhi output rate limiting, coalesce, and >>>>>>> external time window – what is the purpose of this? >>>>>>> >>>>>>> Output rate limiting - Limit the rate of output for a query. e.g. >>>>>> hold back events and send them all at once every 5 sec, hold back events >>>>>> and send the last event every 5 sec etc. This allows user to decide when >>>>>> CEP needs to trigger the events out. >>>>>> >>>>> >>>>> Coalesce - Provides a mechanism to coalesce attributes and get the >>>>> value from an attribute that has data available. (e.g. if an incoming >>>>> stream has attributes price1, price2, if your query says >>>>> "coalesce(price1,price2)", then if data is not available in price1, it >>>>> will >>>>> get data from price2) >>>>> >>>>> External time window - Make the Siddhi Engine consider timestamps from >>>>>> and external source, not the host machine clock, for time related >>>>>> processing. Siddhi can use the timestamps which are in the events for >>>>>> processing. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - Support for Siddhi stats calculation – what is the purpose of >>>>>>> this? >>>>>>> >>>>>>> CEP 3.0.0 has support for monitoring statistics for each and every >>>>> component. This capability is extended to Siddhi engine as well. This >>>>> would >>>>> enable users to monitor event statistics (incoming count, processed count >>>>> etc.) inside the Siddhi engine itself. >>>>> >>>>> >>>>>> >>>>>>> >>>>>>> - >>>>>>> - Support for Siddhi event tracing and tracing a message at >>>>>>> multiple stages of an event flow – what is the purpose of this? >>>>>>> >>>>>>> Event tracing feature helps to trace the events through-out the all >>>>>> CEP components, this gives information how events get changed in each and >>>>>> every components. very helpful for debugging purposes. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - Default timestamp and attribute type converters for Siddhi – >>>>>>> what is the purpose of this? >>>>>>> >>>>>>> This allows users to convert the event attribute datatype from one >>>>>> to another, no need to write an extension to convert the data types, can >>>>>> easily use these function to change the data types of the attributes. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - Ability for Siddhi to retrieve resources/properties from >>>>>>> registry – what is the purpose of this? >>>>>>> >>>>>>> Can retrieve resources and properties from registry, Users can set >>>>>> the resource location in siddhi queries and then siddhi will get the >>>>>> resource from the registry dynamically on run time. >>>>>> >>>>>> 3) Enhanced Event Management Capabilities >>>>>>> >>>>>>> - Support for multiple Input Event Adaptors and Output Event >>>>>>> Adaptors – what is the purpose of this? >>>>>>> >>>>>>> User requirement changes based on their use-case, then these >>>>>> multiple input and output event adaptors gives options to select a >>>>>> suitable >>>>>> approach for their usecase. If any of the default adaptots not matches >>>>>> their requirement then they can simply write a custom adaptor based on >>>>>> the >>>>>> guidelines given and use it. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - New Event Formatter component to handle events sent out from >>>>>>> the CEP – what is the purpose of this? >>>>>>> >>>>>>> This is used to do the output mapping part for the events which >>>>>> sending out from CEP. Events which are coming out from siddhi after >>>>>> processing is in the WSO2Event format then users can change the event >>>>>> format based on their requirement to other formats like xml, json, text >>>>>> or >>>>>> map and also they can filter some attributes on the event also before >>>>>> sending them out from CEP. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - New Event Builder component to handle incoming events to CEP – >>>>>>> what is the purpose of this? >>>>>>> >>>>>>> This is used to do the input mapping for the events which are coming >>>>>> to the CEP. Events can be send to CEP in different formats like >>>>>> wso2Event, >>>>>> xml, json, map and text, then event builder helps to filter some >>>>>> attributes >>>>>> of the events if it is needed and also to convert those events to a >>>>>> format >>>>>> which can be processed by Siddhi. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - Registry integration for Event Formatter (XML, Text and JSON >>>>>>> mapping) – what is the purpose of this? >>>>>>> >>>>>>> Users can define the mapping templates in registry and can use >>>>>> (re-use) them in different execution plans, Execution plan retrieves the >>>>>> resource from registry dynamically based on the resource path. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - Ability read data from the databases and use it within queries >>>>>>> using external calls – what is the purpose of this? >>>>>>> >>>>>>> Event tables can be used for this purposes. (Persist events to a >>>>>> RDBMS or similar and use that with queries), this helps to dealt with DB >>>>>> (can be used for some validation purposes) >>>>>> >>>>>>> >>>>>>> - >>>>>>> >>>>>>> >>>>>>> 4) Miscellaneous Enhancements >>>>>>> >>>>>>> - Support for system, per execution plan/per deployment artifact >>>>>>> monitoring – what is the purpose of this? >>>>>>> >>>>>>> This is again related to event monitoring support of CEP 3.0.0. >>>>> Statistics regarding events can be taken for the whole server, for an >>>>> executiion plan or per a deployment artifact (deployment artifacts in CEP >>>>> 3.0.0 will be input adaptors, event builders, execution plans, event >>>>> formatters, output adaptors) >>>>> >>>>>> >>>>>>> >>>>>>> >>>>>>> - Support for JMS Queue – what is the purpose of this? >>>>>>> >>>>>>> Older CEP versions only supports for JMS topic but JMS queue >>>>>> support available from CEP 3.0.0. This helps for durable subscription >>>>>> purposes. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - Ability to process XML/JSON input child elements as separate >>>>>>> input messages – what is the purpose of this? >>>>>>> >>>>>>> Older CEP version not supports for json format (input or output >>>>>> mapping) but CEP 3.0.0 supports for json format which is a commonly used >>>>>> data formats nowadays. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - Sample framework for CEP – what is the purpose of this? >>>>>>> >>>>>>> CEP server can be started with necessary configurations which needed >>>>>> for the samples to run, users don't need to configure manually. Just can >>>>>> start the CEP server in sample mode. >>>>>> >>>>>>> >>>>>>> - >>>>>>> - Dashboard integration using WSO2 User Engagement Server. – >>>>>>> what is the purpose of this? >>>>>>> >>>>>>> Can be used for real-time event analysis and visualization. >>>>>> >>>>>> >>>>>> >>>>>> @Lasantha & Suho, Please check and add if anything required to >>>>>> mention further. >>>>>> >>>>> >>>>> Please don't hesitate to ask if you need further explanations, drill >>>>> down regarding any of the features or need more information. >>>>> >>>>> Thanks, >>>>> Lasantha >>>>> >>>>> >>>>>> >>>>>> >>>>>> Thanks & Regards, >>>>>> Mohan >>>>>> >>>>>> >>>>>> -- >>>>>> *V. Mohanadarshan* >>>>>> *Software Engineer,* >>>>>> *Data Technologies Team,* >>>>>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>>>>> *lean.enterprise.middleware.* >>>>>> >>>>>> email: [email protected] >>>>>> phone:(+94) 771117673 >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Lasantha Fernando* >>>>> Software Engineer - Data Technologies Team >>>>> WSO2 Inc. http://wso2.com >>>>> >>>>> email: [email protected] >>>>> mobile: (+94) 71 5247551 >>>>> >>>> >>>> >>> >>> >>> -- >>> *V. Mohanadarshan* >>> *Software Engineer,* >>> *Data Technologies Team,* >>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>> *lean.enterprise.middleware.* >>> >>> email: [email protected] >>> phone:(+94) 771117673 >>> >> >> >> >> -- >> >> >> *Rebecca Hurst Office: 650-679-9282 <650-679-9282>Mobile: 650-274-7533 >> <650-274-7533>* >> > > > > -- > Daya Atapattu, Ph.D. > Senior Architect, WSO2 Inc. > Visiting Faculty, University of Moratuwa > Phone: +94 77 047 4730, +1 203 484 7099 > > -- +1.678.431.1656 mailto:[email protected] twitter @cobiacomm http://blog.cobia.net/cobiacomm (blog)
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
