Are you able to create ticket etc? If I have an account, can't
remember what it is and so can't log into issue tracker right now. :-)
Graham
On Jul 22, 12:46 am, Christopher Lenz <[EMAIL PROTECTED]> wrote:
> Hey Graham,
>
> On 20.07.2008, at 14:30, Graham Dumpleton wrote:
>
>
>
> > Can someone tell me what I am missing in the following analysis.
> > Because I am on holiday can only read code and don't have access to a
> > runnable Trac instance to play with and see what happens in practice.
>
> > When Trac runs under a WSGI server such as mod_wsgi, the Request class
> > in trac.web.api is used. In the send_file method of that class it has
> > code:
>
> > if self.method != 'HEAD':
> > self._response = file(path, 'rb')
> > file_wrapper = self.environ.get('wsgi.file_wrapper')
> > if file_wrapper:
> > self._response = file_wrapper(self._response, 4096)
>
> > What this code does is open the file specified by 'path'. It then sees
> > whether the WSGI server has provided wsgi.file_wrapper extension and
> > if it does then wraps the file object using by executing call of
> > wsgi.file_wrapper. This will allow the WSGI server to used any
> > optimised method it can to send the file. In the case of Apache/
> > mod_wsgi (>= 2.0) which provides this extension, it will use operating
> > system sendfile() or similar depending on what is available.
>
> > Now, wsgi.file_wrapper is optional and a WSGI server doesn't have to
> > provide it (and mod_wsgi < 2.0 doesn't implement it). In that case the
> > actual file object gets returned as response. The problem with that is
> > that when a file object is used as an iterable as it would then be by
> > a WSGI adapater, the file is broken up based on text lines in the
> > file, ie., LF as separator. Thus the content of the file can
> > drastically effect performance as far as throughput or in terms of
> > memory usage.
>
> > The worst case throughput scenario is a large file containing lots of
> > lines which very few characters on each line. Problem there is that
> > because WSGI requires each block returned by iterable to be separately
> > flushed, you end up writing and flushing a large number of small
> > blocks. Thus network performance sucks really bad.
>
> > At the other end of the scale is a binary file that doesn't have any
> > LFs in it. Here the complete file would be read into memory in one go
> > causing a spike in transient memory usage. When request complete,
> > memory freed, but process has still allocated it. Would be bad if you
> > had really large attachments.
>
> > What I don't understand, or can't see how it is occurring, is why when
> > wsgi.file_wrapper extension doesn't exist, that the code isn't
> > wrapping the file object in its own wrapper object to return it in
> > fixed length blocks, just as the WSGI PEP describes.
>
> > class FileWrapper:
>
> > def __init__(self, filelike, blksize=8192):
> > self.filelike = filelike
> > self.blksize = blksize
> > if hasattr(filelike,'close'):
> > self.close = filelike.close
>
> > def __getitem__(self,key):
> > data = self.filelike.read(self.blksize)
> > if data:
> > return data
> > raise IndexError
>
> > I am well aware that there is a similar _FileWrapper in trac.web.wsgi,
> > but that doesn't seem to be used by trac.web.api Request object unless
> > it is through some bit of magic that I cannot find.
>
> > So, my question is, does _FileWrapper somehow get used by trac.web.api
> > Request object in some way I haven't found yet,or is Trac shooting
> > itself in the foot by making static file serving perform worse or
> > cause unexpected memory usage for the case the WSGI server doesn't
> > implement wsgi.file_wrapper. Last time I looked, various WSGI servers
> > or adapters don't implement wsgi.file_wrapper and neither does older
> > mod_wsgi 1.X versions.
>
> > Oh, and in case you think it isn't a problem because people would be
> > serving static files from the web server itself, then realise that
> > above code, ie., send_file(), is used when returning attachments and
> > the web server can't be made to directly handle them that I know of.
>
> Good catch, I think this problem has simply not been noticed because
> mod_python (with the Trac WSGI adapter), mod_wsgi and tracd all have a
> sensible implementation of the file_wrapper interface.
>
> The fallback should be fixed to use the FileWrapper from trac.web.wsgi
> as you suggest.
>
> Thanks,
> --
> Christopher Lenz
> cmlenz at gmx.de
> http://www.cmlenz.net/
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac
Development" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/trac-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
