I have been working on a compression patch as a component of my PatchKit product for some time now, but I have been hitting a few problems, and had put it to one side while I worked on a couple of other products. If people still want to see something like this, I can bring the code out of moth-balls and start working on the problems.
Adrian... -- The difficulty of tactical maneuvering consists in turning the devious into the direct, and misfortune into gain. - Sun Tzu ----- Original Message ----- From: "Brad Clements" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Tuesday, February 05, 2002 10:34 PM Subject: [Zope-dev] Adding gzip compression to HTTPResponse.py > I'm looking for architectural suggestions for adding gzip compression to > HTTPResponse for text types. > > First, I just wanted to compress xml-rpc output, since I'm returing lots of table data as > XML text (not objects), then loading that text/xml into a DOM for XSLT processing. > > I hacked the attached code into HTTPResponse, at the end of setBody. It works for > xml-rpc responses and I suppose any text output, so long as the response object has a > header named "dogzip" set. > > I know "dogzip" is a stupid name, but this is just a testing thing. > > Representative compressions: > > compress oldlen 150366 new len 11926 > compress oldlen 204382 new len 14170 > compress oldlen 12746 new len 1364 > > As you can see, very useful compressions for xml-rpc output. > > But for HTML output, what's really needed is I think a special kind of Cache Object. > One that combines HTTP Caching with Ram caching to keep gzip compressed objects > "in memory". > > Some HTML pages are really quite large, and gzip compression can make a noticable > difference. Just the javascript code sizes themselves are .. really big. > > For xml-rpc, obviously every response must be compressed if it's "worth it", and I can > see that having to set a response property on a per request basis is appropriate for > xml-rpc. > > But for text file objects, Page Templates and stuff.. How does setBody work with Ram > Cache objects? I have some ideas... > > Anyone think this is worthwhile? > > Also, RESPONSE.setBody really should have access to REQUEST.headers. What's > the clean way to do that? Just pass the request object to response object's init > method? > > Here's quick gzip compression hack-in, based on code posted by Neil Schemenauer > > Thanks Neil. > > Added about line 265 in HTTPResponse.py in Zope 2.5 B3 > > try: > dogzip = self.headers['dogzip'] > del self.headers['dogzip'] > if dogzip and split(content_type,'/')[0] == 'text': > body = self.body > startlen = len(body) > import zlib, struct > _gzip_header = ("\037\213" # magic > "\010" # compression method > "\000" # flags > "\000\000\000\000" # time > "\002" > "\377") > co = zlib.compressobj(6,zlib.DEFLATED,-zlib.MAX_WBITS, > zlib.DEF_MEM_LEVEL,0) > chunks = [_gzip_header, co.compress(body), > co.flush(),struct.pack("<ll",zlib.crc32(body),startlen)] > z = join(chunks,"") > newlen = len(z) > print "compress oldlen ",startlen,"new len",newlen > if newlen < startlen: > self.body = z > self.setHeader('content-length', newlen) > self.setHeader('content-encoding','gzip') > except: > pass > > > > Brad Clements, [EMAIL PROTECTED] (315)268-1000 > http://www.murkworks.com (315)268-9812 Fax > netmeeting: ils://ils.murkworks.com AOL-IM: BKClements > > > _______________________________________________ > Zope-Dev maillist - [EMAIL PROTECTED] > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) > _______________________________________________ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )