To answer Srinath's questions: - I used a simple query which calls built-in functions avg() and count() - Mohan installed a patch to print size of the queue. The run confirmed the previous hypothesis. The input queue gradually fills up and then gets full. The performance drop seen earlier coincides with time the queue gets full. - I ran with non-blocking queue before. The queue gets full and then the server throws an exception. If using a non blocking queue, we have to ensure inputStream_rate <= CEP_throuhput. (I don't think there is an easy way to do that.) - I tried to run with a larger sized window of 30 minutes. At servers default heap size of 1024K, the server was throwing "GC overhead limit exceeded". I manged to run after doubling the heap size to 2048K. I was also monitoring GC using visualvm. It ran. But came to the verge of throwing the same exception with GC taking a large percentage of time. So, I think, performance wise that run is not fair. - I will run Joins next.
Regards, - Daya On Wed, Nov 13, 2013 at 8:15 AM, Srinath Perera <[email protected]> wrote: > Hi Daya, > > What is the query you used? > Could you try this monitoring the size of the queue (should add a log to > show size of the queue for every 100k message or so)? Can we test with > non-blocking queue? > > Another explanation to drop is that there are too much events in the > window at 1.5M TPS. But we need to verify. > Can we test for longer times ..like 10m, 15m, 1, 3h? > > When this is done, the next step is Joins. > > --Srinath > > > > > > On Wed, Nov 13, 2013 at 8:12 AM, Srinath Perera <[email protected]> wrote: > >> Daya, no private mails pls .. should sent to arch@ .. I will respond >> there >> >> >> On Tue, Nov 12, 2013 at 6:37 PM, Daya Attapattu <[email protected]> wrote: >> >>> Here are the throughput I got with varying window sizes. Throughput was >>> measured from input stream with a blocking queue. I think the sudden drop >>> around 1.5M when queue gets full. So, values before 1.5M are not valid. >>> (Not all those events go through CEP, some just accumulate in input >>> queue.) But the more important thing is we don't see a substantial >>> performance variation. >>> >>> Regards, >>> >>> - Daya >>> >>> -- >>> Daya Atapattu, Ph.D. >>> Senior Architect, WSO2 Inc. >>> Visiting Faculty, University of Moratuwa >>> Phone: +94 77 047 4730, +1 203 484 7099 >>> >>> >> >> >> -- >> ============================ >> Srinath Perera, Ph.D. >> http://people.apache.org/~hemapani/ >> http://srinathsview.blogspot.com/ >> > > > > -- > ============================ > Srinath Perera, Ph.D. > http://people.apache.org/~hemapani/ > http://srinathsview.blogspot.com/ > -- Daya Atapattu, Ph.D. Senior Architect, WSO2 Inc. Visiting Faculty, University of Moratuwa Phone: +94 77 047 4730, +1 203 484 7099
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
