Ok, we had some snowflakes but they didn't make it down to the ground.... Anyway, I committed a first version (not really polished) to my sandbox. It runs a parallel test with some threads etc. and by looking at the log one can compare the time difference between event admin versions.
Carsten 2012/4/5 Carsten Ziegeler <[email protected]>: > Both is possible :) > > 2012/4/5 Marcel Offermans <[email protected]>: >> Snow? :) I feel for the easter bunnies ;) >> >> Thanks! >> >> Greetings, Marcel >> >> >> On Apr 5, 2012, at 13:03 , Carsten Ziegeler wrote: >> >>> The code is too ugly to make it publically available :) >>> But if we have snow at Easter (which might really happen), i might >>> have some time to look into this. Anyway, promised: i'll commit >>> something in the next two weeks >>> >>> Carsten >>> >>> 2012/4/4 Marcel Offermans <[email protected]>: >>>> How about just putting those bundles in your sandbox? I would be >>>> interested to have something to measure the performance of EventAdmin. It >>>> does not necessarily need to be a Pax Exam compatible test. >>>> >>>> Greetings, Marcel >>>> >>>> >>>> On Apr 4, 2012, at 19:50 PM, Carsten Ziegeler wrote: >>>> >>>>> I've a performance test (and several others checking for deadlocks, >>>>> event delivery etc). They all come as bundles which I deploy into a >>>>> running instance and then check the results. I guess these could be >>>>> converted to pax exam tests - but right now I don't have time to do >>>>> that :( >>>>> >>>>> Regards >>>>> Carsten >>>>> >>>>> 2012/4/4 Marcel Offermans <[email protected]>: >>>>>> Out of curiosity, did anybody ever write a "test" that we can run to >>>>>> benchmark the performance of EventAdmin. I don't mean an ad-hoc test but >>>>>> something we can put in the Felix repository (in a sandbox). >>>>>> >>>>>> Greetings, Marcel >>>>>> >>>>>> On Apr 4, 2012, at 19:30 PM, Carsten Ziegeler wrote: >>>>>> >>>>>>> Hi Joel, >>>>>>> >>>>>>> yes, we noticed some performance problems as well - apart from the >>>>>>> problem >>>>>>> you mentioned in our case it was a very high load on the service >>>>>>> registry >>>>>>> (which in addition could also cause deadlocks in the framework). >>>>>>> So I reimplemented the whole thing (FELIX-3321). You can find the code >>>>>>> now >>>>>>> in the latest trunk, so you could just build the version from trunk and >>>>>>> see >>>>>>> how it performs. It basically changes the way how event handlers are >>>>>>> fetched from the registry and how the event topic is compared - filters >>>>>>> are >>>>>>> now only used if really required, the main checks are done by simple >>>>>>> string >>>>>>> comparisons. >>>>>>> >>>>>>> For our really simple performance test case with more than 1 million >>>>>>> events, it took more than 30 seconds with the old implementation, the >>>>>>> new >>>>>>> one processes the same amount in less than a second. >>>>>>> >>>>>>> Not sure about your problem if events getting eaten up. With the new >>>>>>> impl >>>>>>> it should be much easier to track this down (if it still happens) >>>>>>> >>>>>>> Regards >>>>>>> Carsten >>>>>>> >>>>>>> 2012/4/4 Joel Schuster <[email protected]> >>>>>>> >>>>>>>> Carsten et al.,**** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> I've been running some profiling against some test components using >>>>>>>> EventAdmin. I'm seeing some really degenerate behavior that's causing >>>>>>>> some >>>>>>>> real slowdowns for message delivery coming from a few places.**** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> Environment: running 400+ messages per second and with 15+ topics with >>>>>>>> various levels of hierarchy.**** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> Because of the general ‘brute force’ means by which >>>>>>>> org.apache.felix.framework.capabilityset.CapabilitySet.convertArrayToList >>>>>>>> works, and how the multiple layers of >>>>>>>> org.apache.felix.framework.capabilityset.CapabilitySet.match work when >>>>>>>> topics have multiple layers of hierarchy with the names I can double >>>>>>>> the >>>>>>>> throughput of EventAdmin by simply changing my topic names to one >>>>>>>> layer of >>>>>>>> hierarchy. Of course I lose the advantages of the fiters, but if I can >>>>>>>> double the throughput then it seems worth it.**** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> Even then it’s still taking more than half a millisecond (optimally, >>>>>>>> sometimes many tens of milliseconds) for any particular message to >>>>>>>> make its >>>>>>>> way through the EventAdmin system, being pushed and handled. That seems >>>>>>>> like a long time. Part of the overhead comes from the >>>>>>>> org.apache.felix.framework.ServiceRegistry.getService and ungetService >>>>>>>> calls that happen for every single message push.**** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> It seems to me that some sort of caching of previous search results in >>>>>>>> each of these domains would be well worth the effort.**** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> Carsten, I noticed that you had commented about an improved EventAdmin >>>>>>>> in >>>>>>>> the sandbox. Can you please point that out to me, I’d like to try it >>>>>>>> out.* >>>>>>>> *** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> Also, I’ve notice that the EventAdmin is not delivering all the events >>>>>>>> that are pushed to a topic. Small percentages (1 / 1000) are simply >>>>>>>> being >>>>>>>> eaten up somewhere… I can’t seem to track down any reason or log for >>>>>>>> why >>>>>>>> the events simply disappeared.**** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> Thanks all for consideration.**** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> **** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> - Joel**** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>> ** ** >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>> >>>>>>>>> From: Carsten Ziegeler [mailto:[email protected]] >>>>>>>> >>>>>>>>> Sent: Tuesday, January 24, 2012 2:55 AM >>>>>>>> >>>>>>>>> To: [email protected] >>>>>>>> >>>>>>>>> Subject: Re: EventAdmin Asynchronous/Parallel EventHandler >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> Hi, >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> we haven't implemented this feature yet but will do once the final >>>>>>>> >>>>>>>>> spec is publically available. >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> I've written a new implementation of the event admin which is in the >>>>>>>> >>>>>>>>> felix sandbox atm, this one is significanlty faster than the old >>>>>>>> >>>>>>>>> implementation from trunk. >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> Regards >>>>>>>> >>>>>>>>> Carsten >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> 2012/1/18 DBinSD <[email protected]>: >>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>>> Thanks Holger, this is exactly the info I was looking for. It sounds >>>>>>>> like >>>>>>>> >>>>>>>>>> the Async Unordered Events might get me the performance I was looking >>>>>>>> >>>>>>>>> for >>>>>>>> >>>>>>>>>> (though the implications of 'unordered' events might be a problem in >>>>>>>> some >>>>>>>> >>>>>>>>>> applications). I will keep an eye out for an implementation of this >>>>>>>> >>>>>>>>>> feature. >>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>>> Thanks again >>>>>>>> >>>>>>>>>> Chris >>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>>> Holger Hoffstätte-4 wrote: >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> On 18.01.2012 03:35, DBinSD wrote: >>>>>>>> >>>>>>>>>>>> Does the felix implementation of the EventAdmin service support the >>>>>>>> >>>>>>>>>>>> delivery >>>>>>>> >>>>>>>>>>>> of events, asynchronously and in parallel to multiple listeners at >>>>>>>> once? >>>>>>>> >>>>>>>>>>>> It >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> This was addressed as part of RFC 157 ("Updates to EventAdmin") >>>>>>>> >>>>>>>>> sometime >>>>>>>> >>>>>>>>>>> in 2010 and accepted, though I don't remember offhand what the >>>>>>>> >>>>>>>>>>> implementation status was/is. IIRC it was part of 4.3. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> The solution added a set of constants that express basic delivery >>>>>>>> >>>>>>>>>>> behaviour/expectations. Not really enforceable end-to-end QoS >>>>>>>> >>>>>>>>> constraints >>>>>>>> >>>>>>>>>>> (which are impossible in Java anyway) but better than nothing. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> --snip-- >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> 5.2 Asynchronous Unordered Events >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> The following constants are added to EventConstants. This allows an >>>>>>>> >>>>>>>>> Event >>>>>>>> >>>>>>>>>>> Handler to indicate that it is willing to relax the event ordering >>>>>>>> >>>>>>>>>>> requirements so that Event Admin can optimize delivery. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> String EVENT_DELIVERY = "event.delivery" >>>>>>>> >>>>>>>>>>> Service Registration property specifying the delivery qualities >>>>>>>> requested >>>>>>>> >>>>>>>>>>> by an Event Handler service. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> Event handlers MAY be registered with this property. Each value of >>>>>>>> this >>>>>>>> >>>>>>>>>>> property is a string specifying a delivery quality for the Event >>>>>>>> handler. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> The value of this property must be of type String, String[], or >>>>>>>> >>>>>>>>>>> Collection<String>. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> Since: >>>>>>>> >>>>>>>>>>> 1.3 >>>>>>>> >>>>>>>>>>> See Also: >>>>>>>> >>>>>>>>>>> DELIVERY_ASYNC_ORDERED >>>>>>>> >>>>>>>>>>> DELIVERY_ASYNC_UNORDERED >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> String DELIVERY_ASYNC_ORDERED = "async.ordered" >>>>>>>> >>>>>>>>>>> Event Handler delivery quality value specifying the Event Handler >>>>>>>> requires >>>>>>>> >>>>>>>>>>> asynchronously delivered events be delivered in order. Ordered >>>>>>>> delivery >>>>>>>> >>>>>>>>> is >>>>>>>> >>>>>>>>>>> the default for asynchronously delivered events. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> This delivery quality value is mutually exclusive with >>>>>>>> >>>>>>>>>>> DELIVERY_ASYNC_UNORDERED. However, if both this value and >>>>>>>> >>>>>>>>>>> DELIVERY_ASYNC_UNORDERED are specifed for an event handler, this >>>>>>>> >>>>>>>>> value >>>>>>>> >>>>>>>>>>> takes precedence. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> Since: >>>>>>>> >>>>>>>>>>> 1.3 >>>>>>>> >>>>>>>>>>> See Also: >>>>>>>> >>>>>>>>>>> EVENT_DELIVERY >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> String DELIVERY_ASYNC_UNORDERED = "async.unordered" >>>>>>>> >>>>>>>>>>> Event Handler delivery quality value specifying the Event Handler >>>>>>>>>>> does >>>>>>>> >>>>>>>>> not >>>>>>>> >>>>>>>>>>> require asynchronously delivered events be delivered in order. This >>>>>>>> may >>>>>>>> >>>>>>>>>>> allow an Event Admin implementation to optimize asynchronous event >>>>>>>> >>>>>>>>>>> delivery by relaxing ordering requirements. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> This delivery quality value is mutually exclusive with >>>>>>>> >>>>>>>>>>> DELIVERY_ASYNC_ORDERED. However, if both this value and >>>>>>>> >>>>>>>>>>> DELIVERY_ASYNC_ORDERED are specifed for an event handler, >>>>>>>> >>>>>>>>>>> DELIVERY_ASYNC_ORDERED takes precedence. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> Since: >>>>>>>> >>>>>>>>>>> 1.3 >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> See Also: >>>>>>>> >>>>>>>>>>> EVENT_DELIVERY >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> --snip-- >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> Also see >>>>>>>> >>>>>>>>>>> http://stackoverflow.com/questions/7018762/when-to-use-osgi- >>>>>>>> >>>>>>>>> eventadmin-and-when-not >>>>>>>> >>>>>>>>>>> for BJ's official answer. >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> hope this helps >>>>>>>> >>>>>>>>>>> Holger >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>> >>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> >>>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>>> -- >>>>>>>> >>>>>>>>>> View this message in context: http://old.nabble.com/EventAdmin- >>>>>>>> >>>>>>>>> Asynchronous-Parallel-EventHandler-tp33158446p33162303.html >>>>>>>> >>>>>>>>>> Sent from the Apache Felix - Users mailing list archive at >>>>>>>>>> Nabble.com. >>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>> >>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> >>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> -- >>>>>>>> >>>>>>>>> Carsten Ziegeler >>>>>>>> >>>>>>>>> [email protected] >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>> >>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> >>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Carsten Ziegeler >>>>>>> [email protected] >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Carsten Ziegeler >>>>> [email protected] >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>> >>> >>> >>> -- >>> Carsten Ziegeler >>> [email protected] >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > > > -- > Carsten Ziegeler > [email protected] -- Carsten Ziegeler [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
