On Sun, 2007-06-10 at 09:07 -0700, Brian Harring wrote: > Curious, how many folks are actually using dispatch at all? > > For my personal usage, I'm actually not using any of the hooks- I > suspect most folks aren't either. That said, I'm paying a fairly > hefty price for them. > > With Model.__init__'s send left in for 53.3k record instantiation > (just a walk of the records), time required is 9.2s. Without the > send, takes 7.0s. Personally, I'd like to get that quarter of the > time slice back. :)
Since you already have your own version of the Spanish Inquisition set up for testing, what portion of this overhead is just the function call? If you the dispatch function is replaced with just "return", do we save much. In case it's not clear: I'm trying to get a feeling for how much of the cost is caused by the dispatching itself and how much by processing the dispatch inside the signal module. Is avoiding the call altogether necessary or making the handlers much faster? (More for future direction than anything else). > > Via ticket 3439, I've already gone after dispatch to try and speed it > up- I probably can wring a bit more speed out of it, but the > improvements won't come anywhere near reclaiming the 2.2s from above. > > What is left, is flat out removing the send invocations if they're > not needed- specifically shifting the send calls out of __init__, and > wrapping __init__ on the fly *when* something tries to connect to it. > > Effectively, > > from django.dispatch.dispatcher import connect, disconnect > from django.db.model.signals import pre_init > from django.db.models import Model > > class m(Model): pass > > callback = lambda *a, **kw:None > > assert m.__init__ is Model.__init__ > connect(callback, sender=m, signal=pre_init) > assert m.__init__ is not Model.__init__ > > disconnect(callback, sender=m, signal=pre_init) > assert m.__init__ is Model.__init__ > > The pro of this is that the slowdown is limited to *only* the > instances where something is known to be listening- listening to model > class foo doesn't slow down class bar basically; listening to save > doesn't slow down __init__, etc. > > The cons I'll enumerate: > > 1) to do this requires a few tricks- specifically, wrapping methods on > a class on the fly when something starts listening, and reversing the > wrapping when nobody is listening anymore. Personally, I'm > comfortable with this (the misc contribute_to_class crap going on in > _meta already isn't too far off). Realize however others may not be > comfortable with it- thus speak up please. > > 2) usage of Any for a sender means we have to track the potential > senders. > > 3) usage of Any for a signal means we have to track the signals > involved in this trick (registration of the signal instance), and #2. > > 4) Not strictly required, but if sender is a class (and the only > listeners are listening to *that* class, not Any), any deriving > from that class will still fire the signal- meaning the performance > gain is lost for the derivative, sends are occuring that don't have > any listeners. This can be reclaimed via some tricks in > ModelBase.__new__ offhand if desired and the use scenario is at least > semi-common (how many people derive from a defined Model?). > > 5) the wrapping trick introduces an extra func into the callpath when > something is listening. That's basically a semi-ellusive way of > saying "it'll be slightly slower when there is a listener then what's > in place now"; haven't finished the implementation thus I don't have > specifics, but figure a few usecs hit from the wrapper itself (since > the codepaths are executed often enough, it's worth noting for cases > where listeners are expected). > > Would appreciate any thoughts on above; the cons are basically > implementation specific, that said I can work through them (I want > that 25% back, damn it ;)- question is if folks are game for it or > not, if the idea is palatable or not. > > > Aside from that, would really help if I had a clue what folks are > actually using dispatch for with django- which signals, common > patterns for implementing their own signals (pre/post I'd assume?), > common signals they're listening to, etc. I don't think we can hope to get a really accurate picture here beyond a statistical sample with a broad range for any real confidence interval. The problem is that there is a userbase of thousands and a lot of evidence to suggest that most people don't read mailing list threads that they didn't start themselves. Yes, almost everybody reading this is an exception, but that automatically makes you an outlier. However, to add to the sample, I'm using post_init in some cases and pre_save and post_save a lot. Looks like request_started and request_finished are making an appearance in my code, but mostly in diagnostic stuff that is not intended for production use. > Knowing it would help with optimizing dispatch further, and would be > useful if someone ever decides to gut dispatch and refactor the code > into something less fugly. Given that upstream pydispatcher isn't really being maintained, I don't think we should be too hesitant to tweak it for our needs. Regards, Malcolm --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~----------~----~----~----~------~----~------~--~---
