Re: RegistryMiddleware is a major overhead
On Tue, Jan 20, 2009 at 1:37 PM, Ian Bicking i...@colorstudy.com wrote: On Tue, Jan 20, 2009 at 11:36 AM, Ben Bangert b...@groovie.org wrote: Similarly, other request handling operations such as creating the request object may also be redundant, because the user may not be interested in all the attributes of the request object. So wouldn't it be better to provide those things on-demand (that is using a call interface to get things such as params, etc) instead of creating all that stuff by default even when it may not be needed ? Sure, and its possible some other things in webob.Request might be more lazily done, though at the moment, most every attribute on it only actually does its work upon access. If you can see any specific locations in it that may be further optimized, that'd be great. If anyone does put together something to target webob performance specifically, that'd make the webob optimization happen a lot faster ;) Instantiation *shouldn't* be slow (I can't say for sure if that's true, but if it's not I'll want to fix that). And probably there's some features in Request that should actually be taken out (like environ_getter), that might be causing some overhead. But I'd rather work from empirical data, and I haven't gotten around to putting that together. Chris McDonough also noticed some speed issues with WebOb, so I put together a simple profiler: http://svn.pythonpaste.org/Paste/WebOb/trunk/tests/performance_test.py Based on this I've made a number of improvements to Response.__init__ and Request.__init__ to improve the base performance. This doesn't affect anything after initialization, but for a lot of cases these may be the only functions to be hit. (Well, setting values in Response.headers might be slightly faster) -- Ian Bicking | http://blog.ianbicking.org --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: RegistryMiddleware is a major overhead
That's it I'm switching to Lisp. L8r suxx0rs! /kidding Tycon wrote: Benchmarks indicate it is taking as much as 20% of request handling time (not including the user action). The question is why is it even necessary to use this registry facility ? Similarly, other request handling operations such as creating the request object may also be redundant, because the user may not be interested in all the attributes of the request object. So wouldn't it be better to provide those things on-demand (that is using a call interface to get things such as params, etc) instead of creating all that stuff by default even when it may not be needed ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: RegistryMiddleware is a major overhead
Tycon wrote: Benchmarks indicate it is taking as much as 20% of request handling time (not including the user action). So, taking into account that 96.7% of the time of a request is inside the user action(empirically proved by the extensive benchmarks I've made which Im not going to publish), then setting up the registry takes than 1% of the total time. Not too bad IMHO. The question is why is it even necessary to use this registry facility ? So several apps can cohabitate the same process. Similarly, other request handling operations such as creating the request object may also be redundant, because the user may not be interested in all the attributes of the request object. So wouldn't it be better to provide those things on-demand (that is using a call interface to get things such as params, etc) instead of creating all that stuff by default even when it may not be needed ? Have you taken the time to study what webob.Request exactly does? I guess not since it actually *does* lazy load all attributes on demand. You're not penalized for attributes you don't access. Alberto --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: RegistryMiddleware is a major overhead
On Jan 19, 2009, at 10:37 PM, Tycon wrote: Benchmarks indicate it is taking as much as 20% of request handling time (not including the user action). Curious, I've done rather extensive benchmarks and never seen the registry middleware as being such a chunk. Can you elaborate? The question is why is it even necessary to use this registry facility ? It keeps the module globals not just thread-local, but request-local. In the future, it'll be more swappable to merely thread-locals if you don't anticipate WSGI app nesting (which many don't). Similarly, other request handling operations such as creating the request object may also be redundant, because the user may not be interested in all the attributes of the request object. So wouldn't it be better to provide those things on-demand (that is using a call interface to get things such as params, etc) instead of creating all that stuff by default even when it may not be needed ? Sure, and its possible some other things in webob.Request might be more lazily done, though at the moment, most every attribute on it only actually does its work upon access. If you can see any specific locations in it that may be further optimized, that'd be great. Cheers, Ben smime.p7s Description: S/MIME cryptographic signature
Re: RegistryMiddleware is a major overhead
On Tue, Jan 20, 2009 at 11:36 AM, Ben Bangert b...@groovie.org wrote: Similarly, other request handling operations such as creating the request object may also be redundant, because the user may not be interested in all the attributes of the request object. So wouldn't it be better to provide those things on-demand (that is using a call interface to get things such as params, etc) instead of creating all that stuff by default even when it may not be needed ? Sure, and its possible some other things in webob.Request might be more lazily done, though at the moment, most every attribute on it only actually does its work upon access. If you can see any specific locations in it that may be further optimized, that'd be great. If anyone does put together something to target webob performance specifically, that'd make the webob optimization happen a lot faster ;) Instantiation *shouldn't* be slow (I can't say for sure if that's true, but if it's not I'll want to fix that). And probably there's some features in Request that should actually be taken out (like environ_getter), that might be causing some overhead. But I'd rather work from empirical data, and I haven't gotten around to putting that together. -- Ian Bicking | http://blog.ianbicking.org --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: RegistryMiddleware is a major overhead
empirical data is that if you comment out the RegistryManager in config/middleware.py, you get much better req/sec (but you cant use any of the gloabals...). In fact, isn't the whole purpose of this registry facility to enable request-local objects (such as the request object) to be accessed as globals (e.g not having to pass them as arguments to the action) ? So to support a bad programming paradigm (using globals) the solution is to have another inefficient layer to make sure accessing those globals doesn't cause any scoping and synchronization issues ??? Why not just get rid of the bad paradigm ? In Pylons 2 all pertinent information about the request would be passed to the user's action method as explicit arguments. On Jan 20, 11:37 am, Ian Bicking i...@colorstudy.com wrote: On Tue, Jan 20, 2009 at 11:36 AM, Ben Bangert b...@groovie.org wrote: Similarly, other request handling operations such as creating the request object may also be redundant, because the user may not be interested in all the attributes of the request object. So wouldn't it be better to provide those things on-demand (that is using a call interface to get things such as params, etc) instead of creating all that stuff by default even when it may not be needed ? Sure, and its possible some other things in webob.Request might be more lazily done, though at the moment, most every attribute on it only actually does its work upon access. If you can see any specific locations in it that may be further optimized, that'd be great. If anyone does put together something to target webob performance specifically, that'd make the webob optimization happen a lot faster ;) Instantiation *shouldn't* be slow (I can't say for sure if that's true, but if it's not I'll want to fix that). And probably there's some features in Request that should actually be taken out (like environ_getter), that might be causing some overhead. But I'd rather work from empirical data, and I haven't gotten around to putting that together. -- Ian Bicking | http://blog.ianbicking.org --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: RegistryMiddleware is a major overhead
On Jan 20, 3:49 pm, Tycon adie...@gmail.com wrote: empirical data is that if you comment out the RegistryManager in config/middleware.py, you get much better req/sec (but you cant use any of the gloabals...). Or, to be more precise, the empirical data YOU have is that when YOU comment out the RegistryManager, YOU get a much better req/sec, for some definition of much better. In fact, isn't the whole purpose of this registry facility to enable request-local objects (such as the request object) to be accessed as globals (e.g not having to pass them as arguments to the action) ? So to support a bad programming paradigm (using globals) the solution is to have another inefficient layer to make sure accessing those globals doesn't cause any scoping and synchronization issues ??? Why not just get rid of the bad paradigm ? In Pylons 2 all pertinent information about the request would be passed to the user's action method as explicit arguments. On Jan 20, 11:37 am, Ian Bicking i...@colorstudy.com wrote: On Tue, Jan 20, 2009 at 11:36 AM, Ben Bangert b...@groovie.org wrote: Similarly, other request handling operations such as creating the request object may also be redundant, because the user may not be interested in all the attributes of the request object. So wouldn't it be better to provide those things on-demand (that is using a call interface to get things such as params, etc) instead of creating all that stuff by default even when it may not be needed ? Sure, and its possible some other things in webob.Request might be more lazily done, though at the moment, most every attribute on it only actually does its work upon access. If you can see any specific locations in it that may be further optimized, that'd be great. If anyone does put together something to target webob performance specifically, that'd make the webob optimization happen a lot faster ;) Instantiation *shouldn't* be slow (I can't say for sure if that's true, but if it's not I'll want to fix that). And probably there's some features in Request that should actually be taken out (like environ_getter), that might be causing some overhead. But I'd rather work from empirical data, and I haven't gotten around to putting that together. -- Ian Bicking | http://blog.ianbicking.org --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: RegistryMiddleware is a major overhead
On Tue, Jan 20, 2009 at 3:49 PM, Tycon adie...@gmail.com wrote: empirical data is that if you comment out the RegistryManager in config/middleware.py, you get much better req/sec (but you cant use any of the gloabals...). In fact, isn't the whole purpose of this registry facility to enable request-local objects (such as the request object) to be accessed as globals (e.g not having to pass them as arguments to the action) ? So to support a bad programming paradigm (using globals) the solution is to have another inefficient layer to make sure accessing those globals doesn't cause any scoping and synchronization issues ??? Why not just get rid of the bad paradigm ? You can already access the context objects as self._py_object.c etc in the action, which does not use proxies. And if you haven't had any problems commenting out the RegistryManager, I guess that's a viable way to go. In the interactive traceback: dir(self._py_object) ['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 'app_globals', 'buffet', 'c', 'cache', 'config', 'g', 'h', 'request', 'response', 'session', 'translator'] In Pylons 2 all pertinent information about the request would be passed to the user's action method as explicit arguments. I think Pylons users are too attached to the globals to eliminate them, plus it would break backward compatibility severely. And if this combined Pylons/TG/BFG framework gets underway, I don't think there will be a Pylons 2. (I did request that the design not include any StackedObjectProxies, or at least make them optional, so that part is covered.) -- Mike Orr sluggos...@gmail.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
RegistryMiddleware is a major overhead
Benchmarks indicate it is taking as much as 20% of request handling time (not including the user action). The question is why is it even necessary to use this registry facility ? Similarly, other request handling operations such as creating the request object may also be redundant, because the user may not be interested in all the attributes of the request object. So wouldn't it be better to provide those things on-demand (that is using a call interface to get things such as params, etc) instead of creating all that stuff by default even when it may not be needed ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
