Thanks Carsten! I'll play around with it this evening! No snow here either btw, just rain. ;)
On Apr 10, 2012, at 14:49 PM, Carsten Ziegeler wrote: > Ok, we had some snowflakes but they didn't make it down to the ground.... > > Anyway, I committed a first version (not really polished) to my > sandbox. It runs a parallel test with some threads etc. and by looking > at the log one can compare the time difference between event admin > versions. > > Carsten > > 2012/4/5 Carsten Ziegeler <[email protected]>: >> Both is possible :) >> >> 2012/4/5 Marcel Offermans <[email protected]>: >>> Snow? :) I feel for the easter bunnies ;) >>> >>> Thanks! >>> >>> Greetings, Marcel >>> >>> >>> On Apr 5, 2012, at 13:03 , Carsten Ziegeler wrote: >>> >>>> The code is too ugly to make it publically available :) >>>> But if we have snow at Easter (which might really happen), i might >>>> have some time to look into this. Anyway, promised: i'll commit >>>> something in the next two weeks >>>> >>>> Carsten >>>> >>>> 2012/4/4 Marcel Offermans <[email protected]>: >>>>> How about just putting those bundles in your sandbox? I would be >>>>> interested to have something to measure the performance of EventAdmin. It >>>>> does not necessarily need to be a Pax Exam compatible test. >>>>> >>>>> Greetings, Marcel >>>>> >>>>> >>>>> On Apr 4, 2012, at 19:50 PM, Carsten Ziegeler wrote: >>>>> >>>>>> I've a performance test (and several others checking for deadlocks, >>>>>> event delivery etc). They all come as bundles which I deploy into a >>>>>> running instance and then check the results. I guess these could be >>>>>> converted to pax exam tests - but right now I don't have time to do >>>>>> that :( >>>>>> >>>>>> Regards >>>>>> Carsten >>>>>> >>>>>> 2012/4/4 Marcel Offermans <[email protected]>: >>>>>>> Out of curiosity, did anybody ever write a "test" that we can run to >>>>>>> benchmark the performance of EventAdmin. I don't mean an ad-hoc test >>>>>>> but something we can put in the Felix repository (in a sandbox). >>>>>>> >>>>>>> Greetings, Marcel >>>>>>> >>>>>>> On Apr 4, 2012, at 19:30 PM, Carsten Ziegeler wrote: >>>>>>> >>>>>>>> Hi Joel, >>>>>>>> >>>>>>>> yes, we noticed some performance problems as well - apart from the >>>>>>>> problem >>>>>>>> you mentioned in our case it was a very high load on the service >>>>>>>> registry >>>>>>>> (which in addition could also cause deadlocks in the framework). >>>>>>>> So I reimplemented the whole thing (FELIX-3321). You can find the code >>>>>>>> now >>>>>>>> in the latest trunk, so you could just build the version from trunk >>>>>>>> and see >>>>>>>> how it performs. It basically changes the way how event handlers are >>>>>>>> fetched from the registry and how the event topic is compared - >>>>>>>> filters are >>>>>>>> now only used if really required, the main checks are done by simple >>>>>>>> string >>>>>>>> comparisons. >>>>>>>> >>>>>>>> For our really simple performance test case with more than 1 million >>>>>>>> events, it took more than 30 seconds with the old implementation, the >>>>>>>> new >>>>>>>> one processes the same amount in less than a second. >>>>>>>> >>>>>>>> Not sure about your problem if events getting eaten up. With the new >>>>>>>> impl >>>>>>>> it should be much easier to track this down (if it still happens) >>>>>>>> >>>>>>>> Regards >>>>>>>> Carsten >>>>>>>> >>>>>>>> 2012/4/4 Joel Schuster <[email protected]> >>>>>>>> >>>>>>>>> Carsten et al.,**** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> I've been running some profiling against some test components using >>>>>>>>> EventAdmin. I'm seeing some really degenerate behavior that's causing >>>>>>>>> some >>>>>>>>> real slowdowns for message delivery coming from a few places.**** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> Environment: running 400+ messages per second and with 15+ topics with >>>>>>>>> various levels of hierarchy.**** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> Because of the general ‘brute force’ means by which >>>>>>>>> org.apache.felix.framework.capabilityset.CapabilitySet.convertArrayToList >>>>>>>>> works, and how the multiple layers of >>>>>>>>> org.apache.felix.framework.capabilityset.CapabilitySet.match work when >>>>>>>>> topics have multiple layers of hierarchy with the names I can double >>>>>>>>> the >>>>>>>>> throughput of EventAdmin by simply changing my topic names to one >>>>>>>>> layer of >>>>>>>>> hierarchy. Of course I lose the advantages of the fiters, but if I can >>>>>>>>> double the throughput then it seems worth it.**** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> Even then it’s still taking more than half a millisecond (optimally, >>>>>>>>> sometimes many tens of milliseconds) for any particular message to >>>>>>>>> make its >>>>>>>>> way through the EventAdmin system, being pushed and handled. That >>>>>>>>> seems >>>>>>>>> like a long time. Part of the overhead comes from the >>>>>>>>> org.apache.felix.framework.ServiceRegistry.getService and ungetService >>>>>>>>> calls that happen for every single message push.**** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> It seems to me that some sort of caching of previous search results in >>>>>>>>> each of these domains would be well worth the effort.**** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> Carsten, I noticed that you had commented about an improved >>>>>>>>> EventAdmin in >>>>>>>>> the sandbox. Can you please point that out to me, I’d like to try it >>>>>>>>> out.* >>>>>>>>> *** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> Also, I’ve notice that the EventAdmin is not delivering all the events >>>>>>>>> that are pushed to a topic. Small percentages (1 / 1000) are simply >>>>>>>>> being >>>>>>>>> eaten up somewhere… I can’t seem to track down any reason or log for >>>>>>>>> why >>>>>>>>> the events simply disappeared.**** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> Thanks all for consideration.**** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> **** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> - Joel**** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>> ** ** >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>> >>>>>>>>>> From: Carsten Ziegeler [mailto:[email protected]] >>>>>>>>> >>>>>>>>>> Sent: Tuesday, January 24, 2012 2:55 AM >>>>>>>>> >>>>>>>>>> To: [email protected] >>>>>>>>> >>>>>>>>>> Subject: Re: EventAdmin Asynchronous/Parallel EventHandler >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> we haven't implemented this feature yet but will do once the final >>>>>>>>> >>>>>>>>>> spec is publically available. >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> I've written a new implementation of the event admin which is in the >>>>>>>>> >>>>>>>>>> felix sandbox atm, this one is significanlty faster than the old >>>>>>>>> >>>>>>>>>> implementation from trunk. >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> Regards >>>>>>>>> >>>>>>>>>> Carsten >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> 2012/1/18 DBinSD <[email protected]>: >>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>>> Thanks Holger, this is exactly the info I was looking for. It >>>>>>>>>>> sounds >>>>>>>>> like >>>>>>>>> >>>>>>>>>>> the Async Unordered Events might get me the performance I was >>>>>>>>>>> looking >>>>>>>>> >>>>>>>>>> for >>>>>>>>> >>>>>>>>>>> (though the implications of 'unordered' events might be a problem in >>>>>>>>> some >>>>>>>>> >>>>>>>>>>> applications). I will keep an eye out for an implementation of this >>>>>>>>> >>>>>>>>>>> feature. >>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>>> Thanks again >>>>>>>>> >>>>>>>>>>> Chris >>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>>> Holger Hoffstätte-4 wrote: >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> On 18.01.2012 03:35, DBinSD wrote: >>>>>>>>> >>>>>>>>>>>>> Does the felix implementation of the EventAdmin service support >>>>>>>>>>>>> the >>>>>>>>> >>>>>>>>>>>>> delivery >>>>>>>>> >>>>>>>>>>>>> of events, asynchronously and in parallel to multiple listeners at >>>>>>>>> once? >>>>>>>>> >>>>>>>>>>>>> It >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> This was addressed as part of RFC 157 ("Updates to EventAdmin") >>>>>>>>> >>>>>>>>>> sometime >>>>>>>>> >>>>>>>>>>>> in 2010 and accepted, though I don't remember offhand what the >>>>>>>>> >>>>>>>>>>>> implementation status was/is. IIRC it was part of 4.3. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> The solution added a set of constants that express basic delivery >>>>>>>>> >>>>>>>>>>>> behaviour/expectations. Not really enforceable end-to-end QoS >>>>>>>>> >>>>>>>>>> constraints >>>>>>>>> >>>>>>>>>>>> (which are impossible in Java anyway) but better than nothing. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> --snip-- >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> 5.2 Asynchronous Unordered Events >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> The following constants are added to EventConstants. This allows an >>>>>>>>> >>>>>>>>>> Event >>>>>>>>> >>>>>>>>>>>> Handler to indicate that it is willing to relax the event ordering >>>>>>>>> >>>>>>>>>>>> requirements so that Event Admin can optimize delivery. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> String EVENT_DELIVERY = "event.delivery" >>>>>>>>> >>>>>>>>>>>> Service Registration property specifying the delivery qualities >>>>>>>>> requested >>>>>>>>> >>>>>>>>>>>> by an Event Handler service. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> Event handlers MAY be registered with this property. Each value of >>>>>>>>> this >>>>>>>>> >>>>>>>>>>>> property is a string specifying a delivery quality for the Event >>>>>>>>> handler. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> The value of this property must be of type String, String[], or >>>>>>>>> >>>>>>>>>>>> Collection<String>. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> Since: >>>>>>>>> >>>>>>>>>>>> 1.3 >>>>>>>>> >>>>>>>>>>>> See Also: >>>>>>>>> >>>>>>>>>>>> DELIVERY_ASYNC_ORDERED >>>>>>>>> >>>>>>>>>>>> DELIVERY_ASYNC_UNORDERED >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> String DELIVERY_ASYNC_ORDERED = "async.ordered" >>>>>>>>> >>>>>>>>>>>> Event Handler delivery quality value specifying the Event Handler >>>>>>>>> requires >>>>>>>>> >>>>>>>>>>>> asynchronously delivered events be delivered in order. Ordered >>>>>>>>> delivery >>>>>>>>> >>>>>>>>>> is >>>>>>>>> >>>>>>>>>>>> the default for asynchronously delivered events. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> This delivery quality value is mutually exclusive with >>>>>>>>> >>>>>>>>>>>> DELIVERY_ASYNC_UNORDERED. However, if both this value and >>>>>>>>> >>>>>>>>>>>> DELIVERY_ASYNC_UNORDERED are specifed for an event handler, this >>>>>>>>> >>>>>>>>>> value >>>>>>>>> >>>>>>>>>>>> takes precedence. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> Since: >>>>>>>>> >>>>>>>>>>>> 1.3 >>>>>>>>> >>>>>>>>>>>> See Also: >>>>>>>>> >>>>>>>>>>>> EVENT_DELIVERY >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> String DELIVERY_ASYNC_UNORDERED = "async.unordered" >>>>>>>>> >>>>>>>>>>>> Event Handler delivery quality value specifying the Event Handler >>>>>>>>>>>> does >>>>>>>>> >>>>>>>>>> not >>>>>>>>> >>>>>>>>>>>> require asynchronously delivered events be delivered in order. This >>>>>>>>> may >>>>>>>>> >>>>>>>>>>>> allow an Event Admin implementation to optimize asynchronous event >>>>>>>>> >>>>>>>>>>>> delivery by relaxing ordering requirements. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> This delivery quality value is mutually exclusive with >>>>>>>>> >>>>>>>>>>>> DELIVERY_ASYNC_ORDERED. However, if both this value and >>>>>>>>> >>>>>>>>>>>> DELIVERY_ASYNC_ORDERED are specifed for an event handler, >>>>>>>>> >>>>>>>>>>>> DELIVERY_ASYNC_ORDERED takes precedence. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> Since: >>>>>>>>> >>>>>>>>>>>> 1.3 >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> See Also: >>>>>>>>> >>>>>>>>>>>> EVENT_DELIVERY >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> --snip-- >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> Also see >>>>>>>>> >>>>>>>>>>>> http://stackoverflow.com/questions/7018762/when-to-use-osgi- >>>>>>>>> >>>>>>>>>> eventadmin-and-when-not >>>>>>>>> >>>>>>>>>>>> for BJ's official answer. >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> hope this helps >>>>>>>>> >>>>>>>>>>>> Holger >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> >>>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>> >>>>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>>> -- >>>>>>>>> >>>>>>>>>>> View this message in context: http://old.nabble.com/EventAdmin- >>>>>>>>> >>>>>>>>>> Asynchronous-Parallel-EventHandler-tp33158446p33162303.html >>>>>>>>> >>>>>>>>>>> Sent from the Apache Felix - Users mailing list archive at >>>>>>>>>>> Nabble.com. >>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> >>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>> >>>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> -- >>>>>>>>> >>>>>>>>>> Carsten Ziegeler >>>>>>>>> >>>>>>>>>> [email protected] >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> >>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>> >>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Carsten Ziegeler >>>>>>>> [email protected] >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>> For additional commands, e-mail: [email protected] >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Carsten Ziegeler >>>>>> [email protected] >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>> >>>> >>>> >>>> -- >>>> Carsten Ziegeler >>>> [email protected] >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>>> >>>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >> >> >> >> -- >> Carsten Ziegeler >> [email protected] > > > > -- > Carsten Ziegeler > [email protected] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
