Re: [Zope-dev] Adding gzip compression to HTTPResponse.py
I don't have much useful to add - I just wanted to mention that I know
there are people out there who have succesfully used mod_gzip with Zope;
and that I *like* the name dogzip :-)
seb
On Tue, 2002-02-05 at 22:34, Brad Clements wrote:
I'm looking for architectural suggestions for adding gzip compression to
HTTPResponse for text types.
First, I just wanted to compress xml-rpc output, since I'm returing lots of table
data as
XML text (not objects), then loading that text/xml into a DOM for XSLT processing.
I hacked the attached code into HTTPResponse, at the end of setBody. It works for
xml-rpc responses and I suppose any text output, so long as the response object has
a
header named dogzip set.
I know dogzip is a stupid name, but this is just a testing thing.
Representative compressions:
compress oldlen 150366 new len 11926
compress oldlen 204382 new len 14170
compress oldlen 12746 new len 1364
As you can see, very useful compressions for xml-rpc output.
But for HTML output, what's really needed is I think a special kind of Cache Object.
One that combines HTTP Caching with Ram caching to keep gzip compressed objects
in memory.
Some HTML pages are really quite large, and gzip compression can make a noticable
difference. Just the javascript code sizes themselves are .. really big.
For xml-rpc, obviously every response must be compressed if it's worth it, and I
can
see that having to set a response property on a per request basis is appropriate for
xml-rpc.
But for text file objects, Page Templates and stuff.. How does setBody work with Ram
Cache objects? I have some ideas...
Anyone think this is worthwhile?
Also, RESPONSE.setBody really should have access to REQUEST.headers. What's
the clean way to do that? Just pass the request object to response object's init
method?
Here's quick gzip compression hack-in, based on code posted by Neil Schemenauer
Thanks Neil.
Added about line 265 in HTTPResponse.py in Zope 2.5 B3
try:
dogzip = self.headers['dogzip']
del self.headers['dogzip']
if dogzip and split(content_type,'/')[0] == 'text':
body = self.body
startlen = len(body)
import zlib, struct
_gzip_header = (\037\213 # magic
\010 # compression method
\000 # flags
\000\000\000\000 # time
\002
\377)
co = zlib.compressobj(6,zlib.DEFLATED,-zlib.MAX_WBITS,
zlib.DEF_MEM_LEVEL,0)
chunks = [_gzip_header, co.compress(body),
co.flush(),struct.pack(ll,zlib.crc32(body),startlen)]
z = join(chunks,)
newlen = len(z)
print compress oldlen ,startlen,new len,newlen
if newlen startlen:
self.body = z
self.setHeader('content-length', newlen)
self.setHeader('content-encoding','gzip')
except:
pass
Brad Clements,[EMAIL PROTECTED] (315)268-1000
http://www.murkworks.com (315)268-9812 Fax
netmeeting: ils://ils.murkworks.com AOL-IM: BKClements
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] aq_parent not available in __setattr__
At 06.02.2002 11:19 +1000, Brian Oliver wrote: How does one receive a callback/create a hook that is called when an attribute is changed, especially changed via the default property editor for Zope objects (such that aquisition still works)? I hope I did not misunderstand your problem but I used to successfully create hooks with code derived from the following products: http://www.zope.org/Members/tmclaugh/ZClassEvents http://www.zope.org/Members/htrd/BetterCatalogAware HTH, Stefan ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Adding gzip compression to HTTPResponse.py
On Tue, 5 Feb 2002 17:34:26 -0500, Brad Clements [EMAIL PROTECTED] wrote: I hacked the attached code into HTTPResponse, at the end of setBody. It works for xml-rpc responses and I suppose any text output, so long as the response object has a header named dogzip set. I think you also need to check the accept-encoding header, to allow for clients that do not know how to gunzip. That also means you should set caching headers to prevent the compressed and uncompressed responses getting delivered to the wrong clients by a cache. Also, RESPONSE.setBody really should have access to REQUEST.headers. What's the clean way to do that? Just pass the request object to response object's init method? RESPONSE objects have a REQUEST attribute Anyone think this is worthwhile? I looked at this roughly 18 months ago and came to the conclusion that (at the time) adding content-encoding support in Zope was the wrong way to do it. It you are using Zope behind a front-end proxy and you really should be. then it seems like a better idea to deliver the message to that proxy in an uncompressed form, and let it negotiate a transfer-encoding on its own. (Note that is transfer-, not content-encoding) The advantages of this scheme come from the fact that transfer-encoding is a hop-by-hop property. Two downstream caches can negotiate the best compression for that hop. Pushing everything downstream takes load away from zope, and making it a local choice means that the choice is often a better one. At the time, this type of auto-compressing proxies looked like they were just coming of age (http://rproxy.samba.org looked good at the time too). Unfortunately nothing has changed since. Today I think only Apache can do this (and has done for ages). Support in squid has stalled (http://devel.squid-cache.org/projects.html#te). Although I still think this is the way of the future, I suspect the short-term advantage of content-encoding the way you implemented it may be an advanatge for longer than I originally thought. I hope this helps, Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Adding gzip compression to HTTPResponse.py
On 6 Feb 2002 at 10:02, seb bacon wrote: I don't have much useful to add - I just wanted to mention that I know there are people out there who have succesfully used mod_gzip with Zope; and that I *like* the name dogzip :-) That's my dog, zip! Brad Clements,[EMAIL PROTECTED] (315)268-1000 http://www.murkworks.com (315)268-9812 Fax netmeeting: ils://ils.murkworks.com AOL-IM: BKClements ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Adding gzip compression to HTTPResponse.py
On 6 Feb 2002 at 10:40, Toby Dickenson wrote: I think you also need to check the accept-encoding header, to allow for clients that do not know how to gunzip. That also means you should set caching headers to prevent the compressed and uncompressed responses getting delivered to the wrong clients by a cache. Right, but I couldn't figure out how to see the request headers from response.setBody Also, RESPONSE.setBody really should have access to REQUEST.headers. What's the clean way to do that? Just pass the request object to response object's init method? RESPONSE objects have a REQUEST attribute Are you sure? I know that request objects have a response object. But looking at publish.py doesn't look like it goes the other way. At the time, this type of auto-compressing proxies looked like they were just coming of age (http://rproxy.samba.org looked good at the time too). Unfortunately nothing has changed since. Today I think only Apache can do this (and has done for ages). Support in squid has stalled (http://devel.squid-cache.org/projects.html#te). Although I still think this is the way of the future, I suspect the short-term advantage of content-encoding the way you implemented it may be an advanatge for longer than I originally thought. I am using Apache with mod_rewrite. Sure, it'd be great to compression there, but Apache doesn't cache, you need squid for that, right? For non-xmlrpc responses I'd want the stuff cached. I agree, Transfer Encoding is the way to go, but based on remarks at: http://www.iol.ie/~alank/python/httpcomp.html#encoding I stuck with the simpler to understand content-encoding. -- Shouldn't downstream caching proxies ungzip a response if they get a connection from a client that doesn't support gzip? Or will they only do this if its Transfer-Encoded? Brad Clements,[EMAIL PROTECTED] (315)268-1000 http://www.murkworks.com (315)268-9812 Fax netmeeting: ils://ils.murkworks.com AOL-IM: BKClements ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] acquisition, traversal, __roles__(, and zpt?)
Hey Chris. Thanks for the response.
From: Chris McDonough [EMAIL PROTECTED]
Maybe silly questions, but: Do you call InitializeGlobals() on ObjectX's
class (usually at the end of the module where it's defined)?
Hmm. No...do you mean Globals.InitializeClass? I do that.
Does ObjectX
actually have a portal_url method defined on it (or inherited)?
if you are asking if I have masked the CMF portal_url, the answer is no.
I'm mucking about with pdb, and the portal_url I'm working with is
definitely the correct CMF object.
I'm encountering some very odd behavior while messing around with the
CMFSite in the pdb that makes me wonder if the CMF skinned object manager
__of__ or __getattr__ is not quite functioning as it should in the context
of returning portal tools from the main CMFSite instance. Take a gander at
this pdb dialog excerpt and see if it implies something is rotten in the
state of Denmark (I'll move this over to the CMF list if this does seem to
be something I should pursue as a possible error). Keep an eye out for the
'***' attribute errors, and the odd behavior of some tools having a
__roles__ property and some tools not... By the way, 'ObjectX' is an
'Alias' object in the below. 'modernsongs' is the name of my CMF instance.
(this is running on a Zope 2.5 with CMF 1.2 on a win 2k laptop.)
[the end of pdb 'where'...]
C:\Program
Files\ZopeSongs\lib\python\Products\PageTemplates\Expressions.py(32
2)restrictedTraverse()
- pdb.set_trace()
(Pdb) l
317 elif (has(get(object, 'aq_base', object), name)
318 and get(object, name) == o):
319 container = object
320 if not validate(object, container, name, o):
321 import pdb
322 - pdb.set_trace()
323 raise Unauthorized, name
324 else:
325 o=get(object, name, M)
326 if o is not M:
327 # Check security.
(Pdb) args
self = Alias instance at 01885340
path = []
securityManager = AccessControl.SecurityManager.SecurityManager instance at
01B
3586C
get = built-in function getattr
has = built-in function hasattr
N = None
M = []
TupleType = type 'tuple'
(Pdb) o
MembershipTool instance at 01887188
(Pdb) o.__roles__
*** AttributeError: __roles__
(Pdb) o.aq_chain
[MembershipTool instance at 01886D58, Alias instance at 01885340,
CMFSite instance at 017CF128, OrderedFolder instance at 017A3128,
Application instance at 017A19E8, RequestContainer instance at 017D60D8]
(Pdb) testObj=o.aq_chain[4]
(Pdb) testObj
Application instance at 017A19E8
(Pdb) testObj=testObj.modernsongs
(Pdb) testObj
CMFSite instance at 017CF128
(Pdb) testObj.__roles__
('Manager', 'Anonymous')
(Pdb) testObj.Members
PortalFolder instance at 0188C7D0
(Pdb) testObj.Members.__roles__
('Manager', 'Anonymous')
(Pdb) testObj.portal_url
URLTool instance at 0188A6C8
(Pdb) testObj.portal_url.__roles__
*** AttributeError: __roles__
(Pdb) testObj.portal_catalog.__roles__
('Manager', 'Anonymous')
(Pdb) testObj.portal_membership.__roles__
*** AttributeError: __roles__
(Pdb) testObj.portal_actions.__roles__
*** AttributeError: __roles__
(Pdb) testObj.portal_workflow
WorkflowTool instance at 0188BC48
(Pdb) testObj.portal_workflow.__roles__
('Manager', 'Anonymous')
(Pdb) testObj.portal_actions.__roles__
*** AttributeError: __roles__
I did various other tests as well of course, but again I'm aiming for as
much brevity here as possible... ;)
Thanks
Gary
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Dependency problem with SimpleItem
Yo dudes, I have problems with dependencies on Zope 2.4.3: If I try to import OFS.SimpleItem directly, I get following problem. from OFS import SimpleItem Traceback (most recent call last): File stdin, line 1, in ? File D:\ZopeDevel\lib\python\OFS\SimpleItem.py, line 95, in ? import re, sys, Globals, App.Management, Acquisition, App.Undo File D:\ZopeDevel\lib\python\Globals.py, line 90, in ? import Acquisition, ComputedAttribute, App.PersistentExtra, os File D:\ZopeDevel\lib\python\App\PersistentExtra.py, line 87, in ? from Persistence import Persistent ImportError: cannot import name Persistent If you do 'import Zope' first, you don't have this problem. But I can't do that because I use my SimpleItem in a process other than the zope server. and then I get problems with the lock on the Zope database. I just want to construct a SimpleItem, and store it somewhere else than the database that the zope server uses Anyone any ideas on this? TIA, Sloot. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependency problem with SimpleItem
Romain Slootmaekers wrote: Yo dudes, I have problems with dependencies on Zope 2.4.3: If I try to import OFS.SimpleItem directly, I get following problem. (snip) ImportError: cannot import name Persistent If you do 'import Zope' first, you don't have this problem. But I can't do that because I use my SimpleItem in a process other than the zope server. and then I get problems with the lock on the Zope database. I just want to construct a SimpleItem, and store it somewhere else than the database that the zope server uses It usually works to import ZODB instead. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Benchmarks: DTML vs. ZPT?
Hello! Has anyone done any performance comparisons between DTML and ZPT yet? The reason I'm asking is that we did some first(completely unscientific) tests and had the impression that ZPT were actually quite a bit SLOWER than DTML - and I just can't believe that ... Joachim ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Benchmarks: DTML vs. ZPT?
- Has anyone done any performance comparisons between DTML and ZPT yet? The - reason I'm asking is that we did some first(completely unscientific) tests - and had the impression that ZPT were actually quite a bit SLOWER than DTML - - and I just can't believe that ... I haven't done any tests (or even used ZPT on a production site yet), but I'd be shocked if there was any significant speed difference at all. If there *is* a measurable difference, then somebody really futzed up the ZPT parser (enough to have it classified as a bug). ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
