Here is a casual comparison between CEP 2.1.0 and CEP 3.0.0.
It is casual because:
1. I ran the tests in background while using the laptop for other things.
The load was not uniform.
2. The throughput was measured from input to CEP with a blocking queue.
Initially the queue
was empty and throughput recorded is higher than the true rate while
the queue is filling. (But
this applies to both CEP versions and the comparison is fair.)
The query used is:
from inStream#window.Time(n seconds)
select age, avg(age) as avgAge, count(age) as countAge
insert into outStream;
n = 2, 4, 8, 16, 32, 64, 128
(CEP-2.1.0 threw "out of memory" at n = 256 and default heap settings.)
- Daya
On Tue, Nov 19, 2013 at 7:46 PM, Rebecca Hurst <[email protected]> wrote:
> Hi Mohan, Sriskandarajah, Lasantha & Anjana,
>
> Thanks so much for the descriptions. These are very helpful!
>
> I will get started on the press release, and then if we can add any
> performance numbers, I can insert them later.
>
> Cheers,
> Rebecca
>
>
> On Tue, Nov 19, 2013 at 12:43 AM, Mohanadarshan Vivekanandalingam <
> [email protected]> wrote:
>
>> Hi Isabelle,
>>
>> We don't have any comparison performance numbers with us at the moment
>> (CEP 2.1.0 vs 3.0.0).. But Daya is doing some performance analysis
>> regarding CEP 3.0.0, we'll try to get some information once he completed..
>>
>> Regards,
>> Mohan
>>
>>
>> On Tue, Nov 19, 2013 at 1:26 PM, Isabelle Mauny <[email protected]>wrote:
>>
>>> All,
>>>
>>> Do you have any numbers ? :)
>>>
>>> i.e. was performance increased by X % ? Has TPS increased in our tests ?
>>> Numbers help illustrate all the points below and give a point of reference
>>> vs. previous versions AND competitors.
>>>
>>> Thanks,
>>> Isabelle.
>>>
>>>
>>> ------
>>> Isabelle Mauny
>>> Director, Product Management; WSO2, Inc.; http://wso2.com/
>>> email: [email protected] <[email protected]> - mobile: +34 616050684
>>>
>>>
>>> On Tue, Nov 19, 2013 at 7:31 AM, Lasantha Fernando <[email protected]>wrote:
>>>
>>>> Hi Rebecca,
>>>>
>>>> To add to what Mohan has already explained,...
>>>>
>>>>
>>>> On 19 November 2013 11:26, Mohanadarshan Vivekanandalingam <
>>>> [email protected]> wrote:
>>>>
>>>>> Hi Rebecca,
>>>>>
>>>>> Please find my comments below inline for CEP 3.0.0
>>>>>
>>>>>
>>>>>
>>>>>> *CEP 3.0 *1) It underwent a significant architectural change. The
>>>>>> "what’s new versus old" is not explained in the documentation. What do we
>>>>>> want to say about the architectural change, and why is it significant for
>>>>>> users?
>>>>>>
>>>>>
>>>>> CEP 3.0.0 is a complete re-write of WSO2 CEP which reduces the tight
>>>>> coupling between the components and able to use these separate modules in
>>>>> other WSO2 products.
>>>>>
>>>>>
>>>>>>
>>>>>> 2) Enhanced Siddhi Capabilities
>>>>>>
>>>>>> - Support for Siddhi query partitioning and event tables – what
>>>>>> is the purpose of this?
>>>>>>
>>>>>> Partition a query based on some filter and processing happens at each
>>>>> individual partition separately. It increases the performance when
>>>>> processing events for certain queries.
>>>>>
>>>> The advantage over creating partitions statically with separate queries
>>>> is that, with partitioning, new partitions will be created dynamically
>>>> based on the incoming events to CEP.
>>>>
>>>> Event tables - Allows users to store, retrieve and process events in a
>>>> database table like structure. Currently CEP has support for 'In-Memory
>>>> database' event tables and 'Relational database' event tables.
>>>>
>>>>>
>>>>>> -
>>>>>> - New event adaptor manager components to handle broker manager
>>>>>> related configurations – what is the purpose of this?
>>>>>>
>>>>>> New Event adaptor manager components are replacement for the broker
>>>>> configuration which are in older CEP versions. New design gives more
>>>>> straight forward way of creating event adaptors and helps to write custom
>>>>> event adaptors quickly. And event adaptors are clearly divided as input
>>>>> and
>>>>> output to avoid confusions when using adaptors.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - Support for Siddhi output rate limiting, coalesce, and external
>>>>>> time window – what is the purpose of this?
>>>>>>
>>>>>> Output rate limiting - Limit the rate of output for a query. e.g.
>>>>> hold back events and send them all at once every 5 sec, hold back events
>>>>> and send the last event every 5 sec etc. This allows user to decide when
>>>>> CEP needs to trigger the events out.
>>>>>
>>>>
>>>> Coalesce - Provides a mechanism to coalesce attributes and get the
>>>> value from an attribute that has data available. (e.g. if an incoming
>>>> stream has attributes price1, price2, if your query says
>>>> "coalesce(price1,price2)", then if data is not available in price1, it will
>>>> get data from price2)
>>>>
>>>> External time window - Make the Siddhi Engine consider timestamps from
>>>>> and external source, not the host machine clock, for time related
>>>>> processing. Siddhi can use the timestamps which are in the events for
>>>>> processing.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - Support for Siddhi stats calculation – what is the purpose of
>>>>>> this?
>>>>>>
>>>>>> CEP 3.0.0 has support for monitoring statistics for each and every
>>>> component. This capability is extended to Siddhi engine as well. This would
>>>> enable users to monitor event statistics (incoming count, processed count
>>>> etc.) inside the Siddhi engine itself.
>>>>
>>>>
>>>>>
>>>>>>
>>>>>> -
>>>>>> - Support for Siddhi event tracing and tracing a message at
>>>>>> multiple stages of an event flow – what is the purpose of this?
>>>>>>
>>>>>> Event tracing feature helps to trace the events through-out the all
>>>>> CEP components, this gives information how events get changed in each and
>>>>> every components. very helpful for debugging purposes.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - Default timestamp and attribute type converters for Siddhi –
>>>>>> what is the purpose of this?
>>>>>>
>>>>>> This allows users to convert the event attribute datatype from one to
>>>>> another, no need to write an extension to convert the data types, can
>>>>> easily use these function to change the data types of the attributes.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - Ability for Siddhi to retrieve resources/properties from
>>>>>> registry – what is the purpose of this?
>>>>>>
>>>>>> Can retrieve resources and properties from registry, Users can set
>>>>> the resource location in siddhi queries and then siddhi will get the
>>>>> resource from the registry dynamically on run time.
>>>>>
>>>>> 3) Enhanced Event Management Capabilities
>>>>>>
>>>>>> - Support for multiple Input Event Adaptors and Output Event
>>>>>> Adaptors – what is the purpose of this?
>>>>>>
>>>>>> User requirement changes based on their use-case, then these multiple
>>>>> input and output event adaptors gives options to select a suitable
>>>>> approach
>>>>> for their usecase. If any of the default adaptots not matches their
>>>>> requirement then they can simply write a custom adaptor based on the
>>>>> guidelines given and use it.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - New Event Formatter component to handle events sent out from
>>>>>> the CEP – what is the purpose of this?
>>>>>>
>>>>>> This is used to do the output mapping part for the events which
>>>>> sending out from CEP. Events which are coming out from siddhi after
>>>>> processing is in the WSO2Event format then users can change the event
>>>>> format based on their requirement to other formats like xml, json, text or
>>>>> map and also they can filter some attributes on the event also before
>>>>> sending them out from CEP.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - New Event Builder component to handle incoming events to CEP –
>>>>>> what is the purpose of this?
>>>>>>
>>>>>> This is used to do the input mapping for the events which are coming
>>>>> to the CEP. Events can be send to CEP in different formats like wso2Event,
>>>>> xml, json, map and text, then event builder helps to filter some
>>>>> attributes
>>>>> of the events if it is needed and also to convert those events to a format
>>>>> which can be processed by Siddhi.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - Registry integration for Event Formatter (XML, Text and JSON
>>>>>> mapping) – what is the purpose of this?
>>>>>>
>>>>>> Users can define the mapping templates in registry and can use
>>>>> (re-use) them in different execution plans, Execution plan retrieves the
>>>>> resource from registry dynamically based on the resource path.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - Ability read data from the databases and use it within queries
>>>>>> using external calls – what is the purpose of this?
>>>>>>
>>>>>> Event tables can be used for this purposes. (Persist events to a
>>>>> RDBMS or similar and use that with queries), this helps to dealt with DB
>>>>> (can be used for some validation purposes)
>>>>>
>>>>>>
>>>>>> -
>>>>>>
>>>>>>
>>>>>> 4) Miscellaneous Enhancements
>>>>>>
>>>>>> - Support for system, per execution plan/per deployment artifact
>>>>>> monitoring – what is the purpose of this?
>>>>>>
>>>>>> This is again related to event monitoring support of CEP 3.0.0.
>>>> Statistics regarding events can be taken for the whole server, for an
>>>> executiion plan or per a deployment artifact (deployment artifacts in CEP
>>>> 3.0.0 will be input adaptors, event builders, execution plans, event
>>>> formatters, output adaptors)
>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> - Support for JMS Queue – what is the purpose of this?
>>>>>>
>>>>>> Older CEP versions only supports for JMS topic but JMS queue support
>>>>> available from CEP 3.0.0. This helps for durable subscription purposes.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - Ability to process XML/JSON input child elements as separate
>>>>>> input messages – what is the purpose of this?
>>>>>>
>>>>>> Older CEP version not supports for json format (input or output
>>>>> mapping) but CEP 3.0.0 supports for json format which is a commonly used
>>>>> data formats nowadays.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - Sample framework for CEP – what is the purpose of this?
>>>>>>
>>>>>> CEP server can be started with necessary configurations which needed
>>>>> for the samples to run, users don't need to configure manually. Just can
>>>>> start the CEP server in sample mode.
>>>>>
>>>>>>
>>>>>> -
>>>>>> - Dashboard integration using WSO2 User Engagement Server. – what
>>>>>> is the purpose of this?
>>>>>>
>>>>>> Can be used for real-time event analysis and visualization.
>>>>>
>>>>>
>>>>>
>>>>> @Lasantha & Suho, Please check and add if anything required to mention
>>>>> further.
>>>>>
>>>>
>>>> Please don't hesitate to ask if you need further explanations, drill
>>>> down regarding any of the features or need more information.
>>>>
>>>> Thanks,
>>>> Lasantha
>>>>
>>>>
>>>>>
>>>>>
>>>>> Thanks & Regards,
>>>>> Mohan
>>>>>
>>>>>
>>>>> --
>>>>> *V. Mohanadarshan*
>>>>> *Software Engineer,*
>>>>> *Data Technologies Team,*
>>>>> *WSO2, Inc. http://wso2.com <http://wso2.com> *
>>>>> *lean.enterprise.middleware.*
>>>>>
>>>>> email: [email protected]
>>>>> phone:(+94) 771117673
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Lasantha Fernando*
>>>> Software Engineer - Data Technologies Team
>>>> WSO2 Inc. http://wso2.com
>>>>
>>>> email: [email protected]
>>>> mobile: (+94) 71 5247551
>>>>
>>>
>>>
>>
>>
>> --
>> *V. Mohanadarshan*
>> *Software Engineer,*
>> *Data Technologies Team,*
>> *WSO2, Inc. http://wso2.com <http://wso2.com> *
>> *lean.enterprise.middleware.*
>>
>> email: [email protected]
>> phone:(+94) 771117673
>>
>
>
>
> --
>
>
> *Rebecca Hurst Office: 650-679-9282 <650-679-9282>Mobile: 650-274-7533
> <650-274-7533>*
>
--
Daya Atapattu, Ph.D.
Senior Architect, WSO2 Inc.
Visiting Faculty, University of Moratuwa
Phone: +94 77 047 4730, +1 203 484 7099
<<attachment: cepWin.png>>
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
