Re: [Zope-dev] manage_permission from script

2004-04-08 Thread Erik A . Dahl
Thanks for the suggestion but no joy here.  Here is the full trace...

Python 2.3.3 (#1, Jan 27 2004, 09:17:28)
[GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2
Type help, copyright, credits or license for more information.
 import Zope
 app = Zope.app()
 app.testfolder
Folder instance at 410d9f50
 tf = app.testfolder
 tf.getPhysicalRoot().manage_permission(View, [Owner,])
Traceback (most recent call last):
  File stdin, line 1, in ?
  File /home/edahl/zope-2.7.0/lib/python/AccessControl/Role.py, line 
164, in manage_permission
for p in self.ac_inherited_permissions(1):
  File /home/edahl/zope-2.7.0/lib/python/AccessControl/Role.py, line 
82, in ac_inherited_permissions
for p in self._subobject_permissions():
  File /home/edahl/zope-2.7.0/lib/python/OFS/ObjectManager.py, line 
212, in _subobject_permissions
return (Products.__ac_permissions__+
AttributeError: aq_acquire



On Apr 7, 2004, at 7:30 PM, [EMAIL PROTECTED] wrote:

For setting permission on the root application object, use
setPermissionDefault of SecurityInfo.
Eh, this only works for my own defined permissions. However I just 
tried
this from my product and it does what it supposed to do:
	
self.getPhysicalRoot().manage_permission('View', ['Owner'])

Sandor


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Re: CVS: Zope3/src/zope/tal - talparser.py:1.6

2004-04-08 Thread Philipp von Weitershausen
Gintautas Miliauskas wrote:
Update of /cvs-repository/Zope3/src/zope/tal
In directory cvs.zope.org:/tmp/cvs-serv27951/src/zope/tal
Modified Files:
	talparser.py 
Log Message:
Removed an assertion which disallows usage of Zope3 i18n in XML markup.

Added a few test cases which run the new code path.

=== Zope3/src/zope/tal/talparser.py 1.5 = 1.6 ===
--- Zope3/src/zope/tal/talparser.py:1.5 Fri Mar 19 16:42:04 2004
+++ Zope3/src/zope/tal/talparser.py Wed Apr  7 11:13:00 2004
@@ -81,7 +81,6 @@
 taldict[keybase] = value
 item = item + (tal,)
 elif ns == 'i18n':
-assert 0, dealing with i18n:  + `(keybase, value)`
 i18ndict[keybase] = value
 item = item + ('i18n',)
 fixedattrlist.append(item)
Hello,

I would like to backport this patch (including tests) to Zope 2, since I 
need to i18n XML generated by ZPTs.

Any objections?

Philipp

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Re: [Zope-Coders] Re: CVS: Zope3/src/zope/tal - talparser.py:1.6

2004-04-08 Thread Fred Drake
On Thursday 08 April 2004 10:00 am, Philipp von Weitershausen wrote:
  I would like to backport this patch (including tests) to Zope 2, since I
  need to i18n XML generated by ZPTs.

None here.


  -Fred

-- 
Fred L. Drake, Jr.  fred at zope.com
PythonLabs at Zope Corporation


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Re: manage_permission from script

2004-04-08 Thread Tres Seaver
Erik A.Dahl wrote:
Thanks for the suggestion but no joy here.  Here is the full trace...

Python 2.3.3 (#1, Jan 27 2004, 09:17:28)
[GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2
Type help, copyright, credits or license for more information.
  import Zope
  app = Zope.app()
  app.testfolder
Folder instance at 410d9f50
  tf = app.testfolder
  tf.getPhysicalRoot().manage_permission(View, [Owner,])
Traceback (most recent call last):
  File stdin, line 1, in ?
  File /home/edahl/zope-2.7.0/lib/python/AccessControl/Role.py, line 
164, in manage_permission
for p in self.ac_inherited_permissions(1):
  File /home/edahl/zope-2.7.0/lib/python/AccessControl/Role.py, line 
82, in ac_inherited_permissions
for p in self._subobject_permissions():
  File /home/edahl/zope-2.7.0/lib/python/OFS/ObjectManager.py, line 
212, in _subobject_permissions
return (Products.__ac_permissions__+
AttributeError: aq_acquire
'manage_permission' (really, the methods it calls) make the assumption 
(unwarranted in your case) that they are called on an 
acquisition-wrapped object.  In your case, the root object is *not* 
wrapped, and therefore does not have the 'aq_acquire' method.  Normally, 
when called via the publisher, the root object *is* wrapped (in a 
RequestContainer).

A workaround would be to wrap the root first:

 zopectl debug
 Starting debugger (the name app is bound to the top-level Zope object)
  app
 Application instance at 40dc1bf0
  app.aq_chain  # not wrapped
 Traceback (most recent call last):
   File stdin, line 1, in ?
 AttributeError: aq_chain
  app.manage_addFolder( 'testfolder' )
  tf = app.testfolder
  pr = tf.getPhysicalRoot()
  pr.aq_acquire
 Traceback (most recent call last):
   File stdin, line 1, in ?
 AttributeError: aq_acquire
  pr = pr.__of__(tf)
  pr.aq_chain
 [Application instance at 40dc1bf0, Folder instance at 40e51d10, 
Application instance at 40dc1bf0]
  pr.aq_acquire
 CMethod object at 0x40dfac60
  manage_permission(View, [Owner])
 

The real fix would be to rip out the use *anywhere* of 
'self.aq_acquire';  the 'aq_aquire' function from Acquisition will 
always do the Right Thing (TM) if the object is not yet wrapped (the 
difference parallels using 'aq_base', 'aq_parent', 'aq_inner' as 
functions rather than methods).

Tres.
--
===
Tres Seaver[EMAIL PROTECTED]
Zope Corporation  Zope Dealers   http://www.zope.com
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Using a truely revision based storage for Zope ?

2004-04-08 Thread Shane Hathaway
[EMAIL PROTECTED] wrote:
I've started looking at the ZODB and APE packages to try and get some
understanding of how the whole storage interaction works, but it'll take me
some time to figure it all out ... So I thought I'd get feedback on the idea
first ...
Sounds great!  If I were you, I would start by replacing 
Ape/lib/apelib/fs/fileops.py with something that uses the Subversion 
module.  It's a very simple module and you should be able to get pretty 
far that way.

Shane

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: [Zope3-dev] proposal: serving static content faster

2004-04-08 Thread Paul Winkler
On Wed, Mar 24, 2004 at 12:57:00PM -0500, Chris McDonough wrote:
 moved to zope-dev only
 On Wed, 2004-03-24 at 09:28, Shane Hathaway wrote:
  This sounds useful for serving content from the filesystem.
  
  However, I'm a little concerned about this because producers must not 
  read from the object database. 

FWIW, I've added your warning to the IStreamIterator docstring.
If I hadn't vaguely remembered this thread, I probably would have
tripped over it sooner or later!
 
-- 

Paul Winkler
http://www.slinkp.com
Look! Up in the sky! It's COBRA DAMN LAGOMORPH!
(random hero from isometric.spaceninja.com)

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: [Zope3-dev] proposal: serving static content faster

2004-04-08 Thread Paul Winkler
On Wed, Mar 24, 2004 at 01:32:18PM -0500, Shane Hathaway wrote:
 Jeremy has suggested that object pre-fetching could be added to ZODB.

This is much on my mind currently.
Any thoughts on what an API for pre-fetching might look like?

The use case that most concerns me is:
If you have an Image or File object with a very long Pdata chain,
you're likely to A) thrash the ZEO client cache if it's not big enough,
and B) spend a long time waiting for all the objects in the chain
to load.  At least, this is my theory on what's slowing things
down - I will be trying to verify this today. See the bottom of this page:
http://www.slinkp.com/code/zopestuff/blobnotes

-- 

Paul Winkler
http://www.slinkp.com
Look! Up in the sky! It's RETRO-NEWBIE Z!
(random hero from isometric.spaceninja.com)

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: [Zope3-dev] proposal: serving static content faster

2004-04-08 Thread Shane Hathaway
On 04/08/04 12:16, Paul Winkler wrote:
On Wed, Mar 24, 2004 at 01:32:18PM -0500, Shane Hathaway wrote:

Jeremy has suggested that object pre-fetching could be added to ZODB.


This is much on my mind currently.
Any thoughts on what an API for pre-fetching might look like?
Well, thinking about it some more, it seems like it might be just as 
easy to run a special thread just for fetching blobs.  One queue should 
tell the blob thread what to get, and another queue should return the 
results.  I think that would be a nice design.

The use case that most concerns me is:
If you have an Image or File object with a very long Pdata chain,
you're likely to A) thrash the ZEO client cache if it's not big enough,
and B) spend a long time waiting for all the objects in the chain
to load.  At least, this is my theory on what's slowing things
down - I will be trying to verify this today. See the bottom of this page:
http://www.slinkp.com/code/zopestuff/blobnotes
In fact, Zope puts large files (the threshold is around 256K - 512K) 
into a temporary file before serving them, to free up application 
threads.  It's a tremendous handicap.

Here's a relevant tangent.  Twisted and asyncore like all I/O to be 
event-driven, but ZODB is fundamentally not event-driven.  In ZODB, any 
attribute access can incur I/O time, and you can't control that.  This 
is why Stackless is interesting for ZODB.  With Stackless, you might be 
able to switch stack frames at the moment ZODB needs to do some I/O and 
get some useful work done.  Without Stackless, ZODB can not guarantee 
any particular response time and therefore should not operate inside a 
time-critical event loop.  Threads can also solve this problem, but the 
global interpreter lock hurts performance.  There's also POSH.  With 
POSH, you can take advantage of multiple processors (which you can't do 
with Stackless nor threads)... that seems like a really good thing. 
Some careful coding might make Zope + POSH scream.

Shane

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: [Zope3-dev] proposal: serving static content faster

2004-04-08 Thread Paul Winkler
On Thu, Apr 08, 2004 at 08:02:05PM -0400, Shane Hathaway wrote:
 On 04/08/04 12:16, Paul Winkler wrote:
 On Wed, Mar 24, 2004 at 01:32:18PM -0500, Shane Hathaway wrote:
 
 Jeremy has suggested that object pre-fetching could be added to ZODB.
 
 
 This is much on my mind currently.
 Any thoughts on what an API for pre-fetching might look like?
 
 Well, thinking about it some more, it seems like it might be just as 
 easy to run a special thread just for fetching blobs.  One queue should 
 tell the blob thread what to get, and another queue should return the 
 results.  I think that would be a nice design.

ok... i'll mull that over.
 
 The use case that most concerns me is:
 If you have an Image or File object with a very long Pdata chain,
 you're likely to A) thrash the ZEO client cache if it's not big enough,
 and B) spend a long time waiting for all the objects in the chain
 to load.  At least, this is my theory on what's slowing things
 down - I will be trying to verify this today. See the bottom of this page:
 http://www.slinkp.com/code/zopestuff/blobnotes
 
 In fact, Zope puts large files (the threshold is around 256K - 512K) 
 into a temporary file before serving them, to free up application 
 threads.  It's a tremendous handicap.

If you're referring to the code in ZServerHTTPResponse.write:
AFAICT, since Image.File calls RESPONSE.write() for each Pdata
in its chain, we'll actually get a temporary file for each Pdata 
rather than one big one for the whole file.
There may be (probably are) performance issues in there, but
right now I'd be really happy to get ClientStorage a bit closer to the 
speed of FileStorage for this Pdata gunk.

-- 

Paul Winkler
http://www.slinkp.com


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Fwd: [ZPT] Making PageTemplate's edit pages Unicode aware

2004-04-08 Thread Stuart Bishop
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
I havn't received much feedback on the ZPT mailing list, so I
thought I'd bring it over here to a wider audience (thread is at
http://mail.zope.org/pipermail/zpt/2004-March/005218.html ).
Begin forwarded message:

From: Stuart Bishop [EMAIL PROTECTED]
Date: 29 March 2004 6:13:06 PM
To: Dieter Maurer [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: [ZPT] Makeing PageTemplate's edit pages Unicode aware
On 27/03/2004, at 9:57 PM, Dieter Maurer wrote:

Stuart Bishop wrote at 2004-3-25 12:27 +1100:
Currently, if you enter non-ascii text into the title or contents
fields on a PageTemplate's edit page, the data ends up stored as
an encoded string (using management_page_charset, if it is set. 
Unknown
encoding if it is not).

This should be easy to fix using the foo:charset:ustring notation
to have Zope convert the encoded strings to Unicode. However, the
file upload  feature is more problematic. Should the file upload
try converting the file to Unicode from UTF-8 and raise an exception
if this is not possible? I personally feel this is preferable to
ending up with arbitrarily enncoded document source, with no idea
of the character set used.
I do not think that Zope should convert when it does not know the
encoding. I am unaware that a missing management_page_charset
can be interpreted as UTF-8. If this were the case, converstion
to unicode might be correct. By the way: the HTML specification
says that uploaded files should come with a content-type 
declaration.
In this case, the charset specified there (if any) should be used
to determine the encoding.
Yes - A missing management_page_charset should probably be interpreted
as either US-ASCII or ISO-8859-1. US-ASCII is probably more correct,
but I would guess that most browsers will be configured to use
ISO-8859-1 as their default (and this might be specified in the HTML
spec?)
I guess using the charset type the browser tells us for file uploads
means we can blame the browser. I don't know how this could be reliable
(since text files themselves don't encode their character set unless
they happen to be UTF-16 or have a BOM). I am wondering if having a
file upload  function is incompatible with a Unicode aware page
templates product.
If management_page_charset is not set, it is unknown what charset
is being used. The only way of knowing the character set of data that
has been submitted is to know the character set of the form that it
was submitted from. All other mechanisms do not work due to
incompatibilities in how the browsers work.
Currently, if you create a page template that contains non-ASCII
characters, any tal:content or tal:replace expressions that return
Unicode will now raise a Unicode error. This can be demonstrated
simply:
html
  divMy 2¢/div
  div tal:content=python:u'My 2\N{CENT SIGN}'My 2¢/div
/html

These are the things I think need to be fixed in Zope's Page Templates
implementation to make them Unicode aware. There may be more (?):
- It should be possible for the actual page template source to
be stored as a Unicode string. Currently, there is an assert
ensuring it is a traditional string.
	- The title property should be a Unicode string.

- PageTemplateFile should grow an optional charset parameter,
  defaulting to US-ASCII.
- PageTemplate.write(text) should raise an exception if text
  is not either a Unicode string or an ASCII string.
- The ZopePageTemplate edit page should use Zope's
  :charset:ustring notation so Unicode strings get passed
  to its handler.
- The file upload widget needs to either be removed, or grow
  a charset box. I don't think either of these solutions are
  ideal :-(
Note that when I say 'Unicode string', we can still store ASCII
text using a traditional string to save space.
My application is currently using a ZopePageTemplate subclass that
has been modified to use Unicode strings for the document source
and title, and it seems to be functioning just fine. Does anyone
know if that assert type(text) == type('') in PageTemplate.write
is there for a reason?


- --  
Stuart Bishop [EMAIL PROTECTED]
http://www.stuartbishop.net/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.3 (Darwin)

iD8DBQFAdfPfAfqZj7rGN0oRAkBuAJ0WLSC3V2eL+zNzkQqBqjJ2bl5degCfe2SB
DlT7NTsieQlDhVgEnHYaXp8=
=6XPE
-END PGP SIGNATURE-
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


RE: [Zope-dev] Re: [Zope3-dev] proposal: serving static content faster

2004-04-08 Thread zope
 In fact, Zope puts large files (the threshold is around 256K - 512K) 
 into a temporary file before serving them, to free up application 
 threads.  It's a tremendous handicap.

I'm working on a product which serves files from the filesystem. The
data retrieval method is the usual:

def pushData(self, f, outstream):
finished = False
while not finished:
block = f.read(blocksize)
if len(block)  blocksize:
finished = True
outstream.write(block)

f is the file on the filesystem, outstream is the request object.

Testing with a 1Mbyte file (ab -n 12 -c 4), I get ~4.4 req/sec -
~4.7Mbyte/sec after a few iterations (the os caches the file).

Now, I change this method to the following:

def pushData(self, f, outstream, data='0'*65536):
for i in range(16):
outstream.write(data)

The test results are the same.

Now, if I disable the temporary file thing in ZServer/HTTPResponse.py
the performance goes up to around 6.9 req/sec - ~7Mbyte/sec. If I
restore my pushData method to it's original form it can still do ~6.2
req/sec - ~6.6Mbyte/sec.
In this case, practically every disk operation was served from the
cache.

It seems from these results, that ZServer's tempfile strategy causes
some (~20% if everything is cached) performance hit, but I think that
there should other bottleneck(s) beside this.

Regards,
Sandor


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: [Zope3-dev] proposal: serving static content faster

2004-04-08 Thread Chris McDonough
I'm pretty sure that the tempfile penalty is unrelated to the results
Paul sees in his tests, at least for smaller files.

If the content-length header indicates that the body of the item is
smaller than 128k, it does not appear to be spooled from a tempfile at
all.  This also may be why there isn't much of a difference between
normal product results and Paul's patched product results at his 12K
file size as opposed to the roughly 60% difference in speed at at 324K
and 1216K (I cannot explain the large difference in results at 38322k
across products; it appears to be product-specific).  Also, for this
file size, wrapping the data in a producer is actually just more
overhead; products would be better off to do a direct RESPONSE.write if
the total file size was under 128k (as evidenced by the fact that the
iterator-based products are slower at small file sizes).

It's a bit of a mystery about why retrieving data over ZEO is an order
of magnitude slower than retrieving it from a local file storage when
the ZEO server is actually on the same physical machine.   I'm sure
there are lots of optimization strategies for making this faster that
involve a lot of heavy lifting and hard thinking in a dark room. ;-) 
While that might be fun, in the meantime, for the hoi-polloi who just
need to get stuff done, I believe that one simple answer to this is, as
usual, to do very aggressive caching and avoid ZODB access altogether
for large blobs.  The ZEO caching algorithm is largely agnostic about
what kinds of objects it caches and doesn't allow for setting a
cache-eviction policy other than whatever its implementors consider
optimal for the most general case.   This would be fairly difficult to
change in the best-case circumstance for someone with deep ZODB
knowledge.  A more effective strategy for normal people might be to
contribute to the hardening of something like FileCacheManager, which
prevents ZODB access in the first place and allows for fine-grained
policies on an application level by caching blobs on client filesystems.

- C


On Thu, 2004-04-08 at 12:16, Paul Winkler wrote:
 On Wed, Mar 24, 2004 at 01:32:18PM -0500, Shane Hathaway wrote:
  Jeremy has suggested that object pre-fetching could be added to ZODB.
 
 This is much on my mind currently.
 Any thoughts on what an API for pre-fetching might look like?
 
 The use case that most concerns me is:
 If you have an Image or File object with a very long Pdata chain,
 you're likely to A) thrash the ZEO client cache if it's not big enough,
 and B) spend a long time waiting for all the objects in the chain
 to load.  At least, this is my theory on what's slowing things
 down - I will be trying to verify this today. See the bottom of this page:
 http://www.slinkp.com/code/zopestuff/blobnotes


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )