Re: [Web-SIG] PEP 444 / WSGI2 Proposal: Filters to suppliment middleware.

2010-12-14 Thread Ian Bicking 
On Sun, Dec 12, 2010 at 9:59 PM, Alice Bevan–McGregor
al...@gothcandy.comwrote:

 Howdy!

 There's one issue I've seen repeated a lot in working with WSGI1 and that
 is the use of middleware to process incoming data, but not outgoing, and
 vice-versa; middleware which filters the output in some way, but cares not
 about the input.

 Wrapping middleware around an application is simple and effective, but
 costly in terms of stack allocation overhead; it also makes debugging a bit
 more of a nightmare as the stack trace can be quite deep.

 My updated draft PEP 444[1] includes a section describing Filters, both
 ingress (input filtering) and egress (output filtering).  The API is
 trivially simple, optional (as filters can be easily adapted as middleware
 if the host server doesn't support filters) and easy to implement in a
 server.  (The Marrow HTTP/1.1 server implements them as two for loops.)


It's not clear to me how this can be composed or abstracted.

@webob.dec.wsgify does kind of handle this with its request/response
pattern; in a simplified form it's like:

def wsgify(func):
def replacement(environ):
req = Request(environ)
resp = func(req)
return resp(environ)
return replacement

This allows you to do an output filter like:

@wsgify
def output_filter(req):
resp = some_app(req.environ)
fiddle_with_resp(resp)
return resp

(Most output filters also need the request.)  And an input filter like:

@wsgify
def input_filter(req):
fiddle_with_req(req)
return some_app


But while it handles the input filter case, it doesn't try to generalize
this or move application composition into the server.  An application is an
application and servers are imagined but not actually concrete.  If you
handle filters at the server level you have to have some way of registering
these filters, and it's unclear what order they should be applied.  At
import?  Does the server have to poke around in the app it is running?  How
can it traverse down if you have dispatching apps (like paste.urlmap or
Routes)?

You can still implement this locally of course, as a class that takes an app
and input and output filters.


-- 
Ian Bicking  |  http://blog.ianbicking.org
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] PEP 444 / WSGI2 Proposal: Filters to suppliment middleware.

2010-12-14 Thread Ian Bicking 
On Tue, Dec 14, 2010 at 12:54 PM, Alice Bevan–McGregor
al...@gothcandy.comwrote:


  An application is an application and servers are imagined but not actually
 concrete.


 Could you elaborate?  (Define concrete in this context.)


WSGI applications never directly touch the server.  They are called by the
server, but have no reference to the server.  Servers in turn take an app
and parameters specific to there serveryness (which may or may not even
involve HTTP), but it's good we've gotten them out of the realm of
application composition (early on WSGI servers frequently handled mounting
apps at locations in the path, but that's been replaced with dispatching
middleware).  An application wrapped with middleware is also a single object
you can hand around; we don't have an object that represents all of
application, list of pre-filters, list of post-filters.




  If you handle filters at the server level you have to have some way of
 registering these filters, and it's unclear what order they should be
 applied.  At import?  Does the server have to poke around in the app it is
 running?  How can it traverse down if you have dispatching apps (like
 paste.urlmap or Routes)?


 Filters are unaffected by, and unaware of, dispatch.  They are defined at
 the same time your application middleware stack is constructed, and passed
 (in the current implementation) to the HTTPServer protocol as a list at the
 same time as your wrapped application stack.


  You can still implement this locally of course, as a class that takes an
 app and input and output filters.


 If you -do- need region specific filtering, you can ostensibly wrap
 multiple final applications in filter management middleware, as you say.
  That's a fairly advanced use-case regardless of filtering.

 I would love to see examples of what people might implement as filters
 (i.e. middleware that does ONE of ingress or egress processing, not both).
  From CherryPy I see things like:

 * BaseURLFilter (ingress Apache base path adjustments)
 * DecodingFilter (ingress request parameter decoding)
 * EncodingFilter (egress response header and body encoding)
 * GzipFilter (already mentioned)
 * LogDebugInfoFilter (egress insertion of page generation time into HTML
 stream)
 * TidyFilter (egress piping of response body to Tidy)
 * VirtualHostFilter (similar to BaseURLFilter)

 None of these (with the possible exception of LogDebugInfoFilter) I could
 imagine needing to be path-specific.


GzipFilter is wonky at best (it interacts oddly with range requests and
etags).  Prefix handling is useful (e.g.,
paste.deploy.config.PrefixMiddleware), and usually global and unconfigured.
Debugging and logging stuff often needs per-path configuration, which can
mean multiple instances applied after dispatch.  Encoding and Decoding don't
apply to WSGI.  Tidy is intrusive and I think questionable on a global
level.  I don't think the use cases are there.  Tightly bound pre-filters
and post-filters are particularly problematic.  This all seems like a lot of
work to avoid a few stack frames in a traceback.

-- 
Ian Bicking  |  http://blog.ianbicking.org
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com