Re: Django Middleware Architecture Proposal
Hi all, In the original post https://groups.google.com/d/topic/django-developers/eJFuiWdobbY/discussionI presented a problem with the current Django middleware archticture and how it can lead to "*leaks*" in the handling of responses and exceptions. I also proposed an alternative architecture in the form of middleware wrappers. *That is the clean solution*, deprecating and eventually removing the current middleware class architecture. If that is too radical or the issue I mentiond about deferred rendering cannot be solved or is too great to be worked around, I have another idea for consideration by the community, as outlined below. This second approach retains the current middleware class architecture and has no backward incompatibilities. When an application developer wants the try, except, finally semantics Django could offer two hooking points - an outer one, and an inner one, as described below: The proposed outer hooking point gives control to an application specified "settings.OUTER_DISPATCH" function (if specified) before Django does any middleware processing. This function is passed the request object and a reference to Django's get_response_inner method (into which the current code in the get_response method would be moved). It returns a response object. This outer dispatch function can return a short-circuit response, or call the passed get_response_inner method which performs the middleware processing and invokes the view function. After Django middleware has processed the response, control is returned to the outer dispatch function which can perform any required response processing, finalization or exception handling. The inner hooking point gives control to an application specified "settings.INNER_DISPATCH function (if specified), directly before Django invokes the view function; that is, after it has invoked any middleware process_request and process_view methods. Django will pass it the request object, the view function, and any argument list and keyword arguments. It returns a response object. This inner dispatch function can return a short-circuit response, or call the view function. Whether the view function returns the response, or raises an exception, the inner dispatch function can handle both situations (or allow the exception to propagate upwards). By default (e.g. in global_settings) both OUTER_DISPATCH and INNER_DISPATCH are initialized to None. An application developer wanting to have control over the request handling at the outer level would do something like: In the project settings file: OUTER_DISPATCH = my_outer_dispatch def my_outer_dispatch(request, handle_request): try: # Perform some request examination or initialization # and possibly return a short-circuit response. response = handle_request(request) # Pass to Django for middleware # processing and view function calling # Perform any processing on the response object that may be needed return response except Exception as e: # Handle the exception, and re-raise if desired or return # a response. finally: # Place finalization code here. An application developer wanting to have control over the request handling at the inner level would do womething like: INNER_DISPATCH = my_inner_dispatch def my_inner_dispatch(request, view_func, *view_args, **view_kwargs): try: # Perform some request and/or view_func examination or initialization # and possibly return a short-circuit response. response = view_func(request, *view_args, **view_kwargs) # Perform any processing on the response object that may be needed return response except Exception as e: # Handle the exception, and re-raise if desired or return # a response. finally: # Place finalization code here. Changes to Django software to faciliate this: 1. conf/global_settings.py Add lines: OUTER_DISPATCH = None INNER_DISPATCH = None 2. core/handlers/base.py Change to the get_response method: def get_response(self, request): if settings.OUTER_DISPATCH: return settings.OUTER_DISPATCH(request, self.get_response_inner) else: return self.get_response_inner(request) def get_response_inner(self, request): Place here the code which is currently in get_response with the following change: Replace the line 114 which currently contains response = wrapped_callback(request, *callback_args, **callback_kwargs) with: if settings.INNER_DISPATCH: response = settings.INNER_DISPATCH(request, wrapped_callback, *callback_args, **callback_kwargs) else: response = wrapped_callback(request, *callback_args, **callback_kwargs) *Conclusion:* Perhaps not as clean as the first proposed solution (
Django Middleware Architecture Proposal
Hi all, I'd like to get some discussion going on the subject of Django's middleware architecture. The current way middleware works provides a way for a developer to specify code executed at certain stages of the arrival of the request and the departure of the response from the system. This is done by the middleware class providing methods such as process_request, process_view, process_response etc. Recently, I tried to write a piece of middleware to use the new Django transaction API (see django.db.transaction.atomic, a nice piece of restructuring by Aymeric Augustin). What I wanted was to have requests which modified data (request.method being one of POST, PUT, DELETE, PATCH) use a transaction, but read-only requests (GET, HEAD), which form the majority of a typical web application's requests, not to have to incur transaction overhead. Right away I hit the problem posed by the current Django middleware architecture. There is a *clear mismatch*. Using transactions in general, and also the new transaction.atomic API, implies wrapping the code concerned (the view function) with an envelope which starts the transaction on the way in, and commits the transaction (on success) or rolls it back (on failure) on the way out. This wrapping (like a try, except, finally) ensures that a transaction is not left open becauses of a failure to execute the except or finally block. The current Django middleware archictecture does not give this certainty. *The current Django middleware architecture cannot guarantee to always call a middleware's process_response or process_exception.* There are scenarios as mentioned in https://groups.google.com/d/msg/django-developers/U4-XcmvWLJ4/SwWBb24IyXUJwhere the required finalization code in a middleware's process_response of process_exception is not given a chance to run. Furthermore, the architecture *gives the appearance of being layered, but in fact is not.*If one of the middleware components short-circuits the process by returning a response from its process_request method, then all middleware classes' process_response methods will be called (unless an exception occurs) even deeper layer middleware classes whose process_request methods were not called. So, this need for transaction handling in middleware, and there are probably many other use cases which require a leak-free semantic, is highlighting a serious deficiency in the current Django middleware architecture. How can we address this? Try to imagine middleware components as layers around the eventual view function. The layers of an onion come to mind. Incoming requests pass through each layer on the way in, and responses or exceptions pass back out through the layers on the way out. We need a solution to prevent responses or exceptions skipping over layers on the way out if it passed through the layer on the way in. Wrappers provide a way to do this. If a middleware component *takes the form of a wrapper*, which wraps a view function (or a view function wrapped by another middleware wrapper), then that middleware can insert wrapping code to perform whatever it needs to do with the request and view_function when called (including generating a short-circuit response) and can perform whatever response or exception handling that is needed after calling the view function. This enables the try, except, finally block structure to be used, which prevents the leaking (or layer skipping) which the current Django middleware architecture is prone to. What would such a wrapper look like: def someMiddlewareWrapper(view_func): @wraps(view_func) def inner(request, *view_args, **view_kwargs): try: # Place code here which you would previously have # placed in a middleware class's process_request # or process_view method. If a short-circuit response # is generated, return it. # Call the view function: response = view_func(request, *args, **kwargs) # Place code here which you would previously have # placed in a middleware's process_response method. return response except Exception as e: # Place code here which you would previously have # placed in a middleware's process_exception method. # Re-raise the exception or return a response. finally: # Place any finalization code here. return inner How to hook this into Django: The developer would need to specify what the middleware wrappers are. This could be done in the settings module: MIDDLEWARE_WRAPPERS = [someMiddlewareWrapper] # List of middleware wrappers The Django request handling code (core/handlers/base.py get_response method) would take care of applying the wrappers when processing a request: (It currently uses "callback" to refer to the view function). # Apply wrappers in reverse so that requests pass through wrappers in the
Re: BCrypt and PBKDF2 Password Hash Caching
On 15/11/13 18:42, Erik van Zijst wrote: > How do people feel about this approach and should it be merged into > Django? If not, then I can turn it into a library instead. Maybe at our > size we're not in Django's sweet spot anymore. However, in their current > version the recommended hashers are just not usable for us. >From my point of view, this is definitely something for an external library, not for Django itself. The additional complexity makes it much harder to review from a security point of view, and easier to make mistakes when deploying, and we want to avoid that. Also, many people will not need the additional performance, and we don't want to make it easy for people to use a less secure option just because they want a really fast site or something. It seems like this can work fine as external code, and so I can't see a reason why this needs to be in Django itself. Thanks, Luke -- "DO NOT DISTURB. I'm disturbed enough already." Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5295D76B.8010606%40cantab.net. For more options, visit https://groups.google.com/groups/opt_out.
