[Zope-dev] Documentation

2004-10-05 Thread Toby Dickenson
Im a little out of touch with Zope development these days; Ive been focussing 
on non-zope projects for the last year or so...

I have some Zope documentation on http://www.zope.org/Members/htrd that I am 
no longer able to maintain, but wich generates a steady stream of emails with 
questions and change suggestions. In particular, my notes on using squid with 
zope, and the changes I originally drafted for the Zope Developers Guide on 
using unicode in Zope.

Is there now a more approporiate place to hold this documentation where it can 
be maintained?

Thanks in advance,

-- 
Toby Dickenson
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Reporting X-FORWARDED-FOR into the log ( REMOTE_ADDR )

2004-07-23 Thread Toby Dickenson
On Thursday 22 July 2004 12:06, Egon wrote:
 I have a Zope server behind a proxy server, this proxy
 enables de X-FORWARDED-FROM that contains the real ip
 of the client that is connected.
 I want to log this variable but I cannot find the way
 to do it.

Check out trusted-proxies in the zope configuation file.

and you mean X-FORWARDED-FOR, not -FROM, right?

-- 
Toby Dickenson
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] any limits on object number?

2004-07-14 Thread Toby Dickenson
[resend]

   We are builiding a large portal using Zope.  We need to create a large
   number of objects.
  
  How large?
 
 in one case, we want the metadata of all the wikipedia contents, that
 will be around 300,000 objects.  
 
The data component of the objects is small, but each
   object carries lots of metadata.  My question is: Is there any limit on
   the number of objects in a given folder?
  
  No, but you should use BTreeFolder2 in any case and don't use the standard
  Zope folder!

'Folders' provide an easy gui for managing sub-items one at a time, and a 
single interface for accessing sub-items that enables things like the tree 
view (in the left pane of the management interface) to work with different 
folder implementations.

Im not sure any of that is a benefit to you, and Folders arent the only kind 
of ZODB container.  I suspect you would be better off directly using a BTree 
of some kind. 

-- 
Toby Dickenson
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Re: [ZODB-Dev] RFC: Multi-Databases

2004-05-05 Thread Toby Dickenson
(reply-to set to zodb-dev)

 In the long term, this will allow is to *finally* support cross database
 references.

Your proposal describes an unfamiliar kind of reference. In unix filesystem 
terms, it mixes characteristics of hard links and symbolic links.

Like hard links it directly references the inode/oid. The reference doent need 
to have a filename/object reference from the root of the referenced 
filesystem/database.

Like symbolic links it does not prevent garbage collection from freeing the 
inode/oid. So there does need to BE a filename/object reference from the root 
of the referenced filesystem/database it needs to exist, but the 
reference doesnt need to have it.

(by 'object reference' above I mean a chain of normal object references like 
we have in normal python and zodb today)


Right now I cant think of a use case for this new type of reference (either in 
a filesystem or zodb database).  Can you give some examples?

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Unicode in Zope 2 (ZMI, Archetypes, Plone, Formulator)

2004-04-26 Thread Toby Dickenson
On Monday 26 April 2004 09:27, Bjorn Stabell wrote:
 While we're all waiting for Zope 3 and Plone 3, I'd like to know what the
 standard practice way of using Unicode with Zope 2.  In particular, we'd
 like to store all text as Unicode in the ZODB, and have Zope do the
 encoding/decoding as automatically and transparently as possible.

I added most of the Unicode support in core Zope in 2.6. Transparency was my 
goal too.

For the applications I was working on at the time this goal was achieved. 
Naturally I stopped without converting all the components that I dont use, 
and sadly noone has picked up this work in a consistent manner.

 * in PropertyManager you can add ustrings, but since it doesn't add
 :ENCODING to the field names, you get a Unicode error when trying to save
 since it tries to decode the text assuming ASCII (big problem)

It works for me in 2.6. Note that it only adds the :ENCODING after adding your 
first unicode property. This is ugly, but necessary for legacy applications 
using 8-bit string properties to hold pre-encoded text.

 I suggest this way of dealing with Unicode right now in Zope 2:

good advice.

 None of the above components handles Unicode in this way, but it seems to
 be how the Unicode support in Zope 2 was meant to be used.  

That is correct.



-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] performance tuning of ZODB

2004-04-22 Thread Toby Dickenson
On Thursday 22 April 2004 10:42, Syver Enstad wrote:
 I've used the cache_size paramter to the constructor of the DB to good
 effect. Are there any further gotchas for ensuring that the ZODB stays
 in memory as much as possible?

Memory usage scales proportional to the number of threads.  Reducing the 
number of worker threads lets you increase the per-thread cache size.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] performance tuning of ZODB

2004-04-22 Thread Toby Dickenson
On Thursday 22 April 2004 11:43, Syver Enstad wrote:

 cache_deactivate_after sounds interesting. Since I am running ZODB in
 a web server I don't want the data to timeout and disappear from
 memory since every object I have should be loaded at all times.

Thats why ZODB ignores that parameter now ;-)

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Display Files from directory

2004-03-16 Thread Toby Dickenson
On Monday 15 March 2004 12:59, Darshan Preet Singh Manku wrote:

 file = 'C:\tmp'

try 'C:/tmp' or 'C:\\tmp'. unless your file name really does contain a tab 
character ;-)

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

2004-03-11 Thread Toby Dickenson
On Thursday 11 March 2004 12:19, Chris McDonough wrote:

  What if you want to make a change in the error handler?

 If anyone had been depending on this behavior in their current error
 handlers, they would have seen weirdnesses in their applications like
 those being reported now.

I have applications that send emails in error handlers. MailHost isnt 
transactional today, but Ive always wanted it to be.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Proposal (was How to make Zope fail nicely under high load?)

2004-02-19 Thread Toby Dickenson
On Thursday 19 February 2004 11:18, Bjorn Stabell wrote:
 I wrote up a simple proposal for how to change Zope to work better under
 heavy load:

 http://zope.org/Members/bjorn/proposals/MakeZopeFailNicelyUnderHeavyLoad

 The solution is a bit different from what we discussed on the list
 before; basically do away with the Medusa request queue altogether and
 use listen()'s backlog instead.

Last week you were keen that medusa keep processing requests so that it can 
return a 503 error quickly. I guess your view on that requirement has 
changed? 

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] How to make Zope fail nicely under high load?

2004-02-12 Thread Toby Dickenson
On Thursday 12 February 2004 00:57, you wrote:

 One of the optimization we're thinking of is storing results of ZCatalog
 searches (e.g., number of replies to postings) in volatile variables so
 we don't have to run the catalog search at all.  We'd like to use memory
 space shared between threads for this.  Using ZEO would require us to
 store this in the ZODB.

Storing that in ZODB would be a bad idea. Theres no reason to think that this 
cache would be faster than ZCatalog.

I dont see why you would be *required* to store in ZODB... just dont share 
your cache between publisher threads on different Zope instances.

(my apologies if this is obvious)
-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Zope, forking and cpu-affinity

2004-02-12 Thread Toby Dickenson
On Thursday 12 February 2004 08:26, Dario Lopez-Kästen wrote:
 Dieter Maurer wrote:
  Zope itself does not fork (unless you use system, popen or things
  like that; in these cases, you do not want to bind the children to the
  same processor).

 Well, I had that impression as well, but here is the output of ps on my
 machines (edited for cruft-removal):

 UID  PID   PPID  C STIME TIME CMD

 zope 32538 14379 0 Feb09 00:02:32 python2.1 z2.py -X -w 
 zope 32539 32538 0 Feb09 00:00:00 python2.1 z2.py -X -w 
 zope 32541 32539 0 Feb09 00:00:09 python2.1 z2.py -X -w 
 zope 32542 32539 0 Feb09 00:00:11 python2.1 z2.py -X -w 
 zope 32543 32539 0 Feb09 00:01:52 python2.1 z2.py -X -w 
 zope 32544 32539 0 Feb09 00:01:41 python2.1 z2.py -X -w 

 that is more than 2 processes for a single instance...

one of them is the supervisor process that restarts the main zope process if 
it crashes.

the others are all threads of one process. one main medusa thread, and 4 
publisher threads.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] How to make Zope fail nicely under high load?

2004-02-11 Thread Toby Dickenson
On Wednesday 11 February 2004 09:16, Jamie Heilman wrote:
 Bjorn Stabell wrote:

  Basically, when the load gets high, Zope has a huge backload of work
  (several minutes of requests), making the average latency for each
  request many minutes.  What are effective ways to do this kind of
  overload management so that the backlog of work doesn't get that big?

Zope's ZServer manages a queue of requests that have been recieved over http, 
but not dispatched to the publisher. This is handled in 
PubCore/ZRendezvous.py. I suspect this queue will be holding your backlog. 

You might get some benefit from capping the length of that queue.

 The idea is that you want tweak it so
 some of your users get everything, and some get nothing, instead of
 everybody getting partials.  This means you have to track sessions
 though... 

Before creating a session, check the size of the ZRendezvous backlog. might 
work.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] cPickleCache endless loop...

2004-01-27 Thread Toby Dickenson
On Tuesday 27 January 2004 19:08, Tim Peters wrote:

 Maybe Toby remembers which release(s) of ZODB the
 current cache implementation first appeared in

Zope 2.6

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] cPickleCache endless loop...

2004-01-27 Thread Toby Dickenson
On Tuesday 27 January 2004 09:39, Mario Lorenz wrote:

 Given that this property is not that widely published (in the various
 tutorials etc.), I wonder if it might be a good idea to improve that loop
 check code, and walk through the ring not more than once, using a counter,
 and not the comparison vs. the ring start, which, as we have seen, may be a
 target that moves too quickly.

You got me curious

The attached patch to 2.6 uses an extra temporary ring node to mark where 
scanning should stop. Normally this will be right next to the home node, and 
behaviour is unchanged. Any of these troublesome objects that move themselves 
to the top of the LRU list *during* the scan will end up *after* the scan 
terminator node, and therefore do not get rescanned.

This works for me on 2.6, which is the only branch Im set up to use at the 
moment.

-- 
Toby Dickenson
? diff
? diff2
Index: cPickleCache.c
===
RCS file: /cvs-repository/Zope/lib/python/ZODB/Attic/cPickleCache.c,v
retrieving revision 1.68.10.3
diff -c -4 -r1.68.10.3 cPickleCache.c
*** cPickleCache.c	30 Apr 2003 17:03:34 -	1.68.10.3
--- cPickleCache.c	27 Jan 2004 23:02:06 -
***
*** 216,224 
  #endif
  
  
  static int
! scan_gc_items(ccobject *self,int target)
  {
  /* This function must only be called with the ring lock held,
 because it places a non-object placeholder in the ring.
  */
--- 216,224 
  #endif
  
  
  static int
! scan_gc_items(ccobject *self,int target,CPersistentRing *terminal)
  {
  /* This function must only be called with the ring lock held,
 because it places a non-object placeholder in the ring.
  */
***
*** 265,281 
  return -1;
  }
  #endif
  
! 	/* back to the home position. stop looking */
! if (here == self-ring_home)
! return 0;
  
  /* At this point we know that the ring only contains nodes
! 	   from persistent objects, plus our own home node. We know
! 	   this because the ring lock is held.  We can safely assume
! 	   the current ring node is a persistent object now we know it
! 	   is not the home */
  object = OBJECT_FROM_RING(self, here, scan_gc_items);
  if (!object) 
  	return -1;
  
--- 265,288 
  return -1;
  }
  #endif
  
! /* reached the end position. stop looking */
! if (here == terminal)
!   return 0;
! 
! /* back to the home position. should not happen */
! if (here == self-ring_home) {
! PyErr_SetString(PyExc_RuntimeError,
!   reached home before terminal, in scan_gc_items);
! return -1;
! }
  
  /* At this point we know that the ring only contains nodes
! 	   from persistent objects, plus our own home node, plus the
!terminal node. We know this because the ring lock is held.
!We can safely assume the current ring node is a persistent
!object now. */
  object = OBJECT_FROM_RING(self, here, scan_gc_items);
  if (!object) 
  	return -1;
  
***
*** 325,332 
--- 332,343 
  
  static PyObject *
  lockgc(ccobject *self, int target_size)
  {
+ int error;
+ CPersistentRing terminal;
+ 
+ 
  /* This is thread-safe because of the GIL, and there's nothing
   * in between checking the ring_lock and acquiring it that calls back
   * into Python.
   */
***
*** 338,350 
  if (IS_RING_CORRUPT(self, pre-gc)) 
  	return NULL;
  ENGINE_NOISE();
  self-ring_lock = 1;
! if (scan_gc_items(self, target_size)) {
! self-ring_lock = 0;
! return NULL;
! }
  self-ring_lock = 0;
  ENGINE_NOISE(\n);
  if (IS_RING_CORRUPT(self, post-gc)) 
  	return NULL;
  
--- 349,376 
  if (IS_RING_CORRUPT(self, pre-gc)) 
  	return NULL;
  ENGINE_NOISE();
  self-ring_lock = 1;
! 
! /* insert a placeholder just before the head node. This will be used to
!  * terminate the scan_gc_items loop. Items touched during the
!  * deactivation run will get inserted between the head and this terminal
!  * node, and will not get re-scanned. */
! terminal.next = self-ring_home;
! terminal.prev = self-ring_home.prev;
! self-ring_home.prev-next = terminal;
! self-ring_home.prev = terminal;
! 
! error = scan_gc_items(self, target_size, terminal);
! 
! /* unlink the placeholder from the ring */
! terminal.next-prev = terminal.prev;
! terminal.prev-next = terminal.next;
! 
  self-ring_lock = 0;
+ if (error)
+ return NULL;
+ 
  ENGINE_NOISE(\n);
  if (IS_RING_CORRUPT(self, post-gc)) 
  	return NULL;
  
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists

Re: [Zope-dev] cPickleCache endless loop...

2004-01-26 Thread Toby Dickenson
On Friday 23 January 2004 17:08, Tim Peters wrote:

 It looks like ghostifying your self triggers self.__del__().  Then the
 __del__ method unghostifies self, which has the side effect of moving self
 to the MRU end of the ring, which in turn means the list traversal will
 visit self *again*.  When it does, same thing happens all over again, ad
 infinitum.

Not necessaralily ad infinitum. It will only run forever if the number of 
__del__-resurrecting objects in the cache is larger than the cache target 
size. Does that fit with your scenario?

 2. If you need a __del__ method (it's hard to imagine why, since it
will get called whenever the object is ghostified, and has nothing
to do with the object's actual lifetime), don't reference any
persistent objects (and esp. not self) within it.

or 2b as jeremy suggested, put your __del__ on a non-persistent sub object.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZODB load state error

2004-01-26 Thread Toby Dickenson
On Monday 26 January 2004 12:08, Thyb wrote:

 What could be the problem?

That indicates that you are out of memory. If that doesnt seem right, maybe 
your storage is corrupt in a manner that the unpickler interprets as 
something with a huge memory footprint.


-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] cPickleCache endless loop...

2004-01-26 Thread Toby Dickenson
On Monday 26 January 2004 17:22, Tim Peters wrote:

 It's actually that the number of __del__-resurrecting objects *plus* the
 number of non-ghostifiable objects in cache is larger than the cache target
 size, right?

Yes, Right. That is more achievable than I thought.


-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] 2.7 management_page_charset cannot be callable anymore

2004-01-16 Thread Toby Dickenson
As the originator of management_page_charset, I should have jumped into this 
thread earlier. My appologies for my late arrival.

On Thursday 15 January 2004 17:22, Martijn Faassen wrote:

 it assumes the only place
 management_page_charset can be coming from is a property, which is
 not true.

Ive missed the use case that needs this.

management_page_charset was added as a transition aid for people who already 
had sites where their use of the management interface assumed something other 
than latin-1 was used for plain string properties. That is, a transition 
towards using unicode objects for storing all natural language strings.

I wouldnt like to see management_page_charset grow into something bigger than 
this - its too ugly.

 Considering the amount of rewriting necessary to make this saner and the
 scariness of doing that just before a Zope 2.7 release, I'll skip it
 and live with my Formulator hack solution.

Can you point me to a desicription of this hack please.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZServer HTTP 1.1 support

2003-12-11 Thread Toby Dickenson
On Thursday 11 December 2003 21:39, Paul Winkler wrote:

 In particular, some people on my team asked me if zserver
 supports persistent connections, and I didn't know how
 to answer. I'm looking now at HTTPServer.py and the docstrings
 suggest that it does ... yes? no?

If you have a squid proxy in front of your zope, then squid will manange both 
HTTP 1.1 persistent connections to your browsers, and persistent connections 
to your Zope.

The squid-to-zope persistent connections are not in the style of HTTP 1.1 
though. Squid uses slightly different headers - I forget the details.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: Zope 2.7.0 b3 regressions

2003-12-09 Thread Toby Dickenson
On Monday 08 December 2003 21:21, Paul Winkler wrote:
 On Mon, Dec 08, 2003 at 08:24:04PM +0100, Dieter Maurer wrote:
  Maybe, my contribution has not been read. Thus, I try again:
 
 / + absolute_url(1) should implement the notion
 of absolute-path URL reference (see RFC2396 section 5).
 
 This means, that the receiving browser should be able
 to retrieve the object correctly given this URL reference.

 Yup.
 But while we're on the subject...
 Why doesn't absolute_url(1) include a leading slash?
 I don't think I've ever seen a use of absolute_url(1)
 that didn't have to add the slash. What was the rationale
 originally?

Because

dtml-var BASEPATH1/dtml-var absolute_url(1)

looks nicer than without the slash

?

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: Post-mortem [Was: Zope 2.7.0 b3 regressions]

2003-12-09 Thread Toby Dickenson
On Tuesday 09 December 2003 16:24, Evan Simpson wrote:
 Summary: absolute_url(1)

Looks good.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: Zope 2.7.0 b3 regressions

2003-12-08 Thread Toby Dickenson
On Monday 08 December 2003 11:35, Stefan H. Holek wrote:

 Note that this is one of my main points: It will be of little use to
 document usage of BASEPATH1+absolute_url(1) when '/'+absolute_url(1)
 appears to work (until it is far too late).

We can fix this social problem by providing an easy way for product developers 
to run their development zope server with the virtual path equivalent to an 
inside-out hosting configuration. easy means not needing apache/squid.

Our staging server is uses an inside-out virtual host configuration (to 
simplify ssl certificate management) so we hit all these problems early 
enough to fix the damage cheaply.


-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Unicode id's and utf8 url encoding

2003-11-11 Thread Toby Dickenson
On Tuesday 11 November 2003 14:23, Tres Seaver wrote:
 On Mon, 2003-11-10 at 22:13, Stuart Bishop wrote:
  Hi. I think the last thread on this issue was:
  http://aspn.activestate.com/ASPN/Mail/Message/zope-Dev/1843637
 
   From my reading of RFC2396 and RFC2277, the character set encoding for
  URLs is UTF8. This is confirmed by
  http://www.w3.org/International/O-URL-code.html .
 
  Does this settle the issue on how to handle non-ascii strings in URLs?

if only browser always worked that way :-(

  If so, is their anything stopping us allowing Unicode ids in Zope? This
  would involve patching OFS.ObjectManager.checkValidId to accept strings
  not in [0-9a-zA-Z\$\-_\.\+!\*'(),], and writing replacement
  urllib.quote and urllib.unquote methods for use by HTTPRequest.py.

+1 on starting a prototype now, but I cant see this landing before 2.8.

This is going to break every use equivalent to getattr(some_object, id). Its 
not obvious to me how this can be cleanly resolved in zope 2.


-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] _p_deactivate() and _v_ variables?

2003-10-23 Thread Toby Dickenson
On Thursday 23 October 2003 08:07, Chris Withers wrote:
 Toby Dickenson wrote:
  Apart from the most trivial cases, it would allow _v_ attributes to
  disappear at random. Its a similar problem to the one that makes it hard
  to write an optimiser for python code, and I am unconvinced that this is
  sane.

 Which, unfortunately, then leaves us with the problem of how to stop Zope
 using up an undeterminable amount of memory...

No, we just exclude objects with _v_ attribute from mid-transaction 
deactivation. There arent many objects in that category, but they do need 
protection.

But, your proposal means we would improve the situation for transactions that 
read from an undeterminable number of persistent objects. It does not help 
for transactions that touch an undeterminable number of non-persistent 
objects, or transactions that change an undeterminable number of persistent 
objects. Is the gain big enough to justify the effort?



-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] cvs.zope.org down

2003-10-23 Thread Toby Dickenson
On Tuesday 21 October 2003 18:08, Jens Vagelpohl wrote:
 Just a quick heads-up:

 Then we will start restoring the data
 from the old drives.

That makes me nervous. How will you know that the sources in cvs havent been 
compromised? 

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Re: _v_ variables and cache garbage collection

2003-10-23 Thread Toby Dickenson
On Thursday 23 October 2003 18:52, Chris Withers wrote:

 What in the ZODB cache or other ZODB code could be causing _v_ variables to
 stick around after they've been set to None in their containing objects?

reference cycles

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Making Zope accept international characters in Id's

2003-10-20 Thread Toby Dickenson
On Monday 20 October 2003 15:01, Lennart Regebro wrote:

 The fact that litmus complains about Zope's inability to accept UTF-8
 filenames is something I view as a bug.

does webdav specify how utf-8-encoded dav filenames correspond to file names 
used in URLs?

 Either the already existing, but seemingly undocumented setting
 management_page_charset could be used to check that the given Id is
 acceptable in that charset.

That feature is intended only as an aid for the transition to unicode for 
users who have content that is stored in an encoded form.

 OR, a separate setting could be used, maybe allowed_id_charset?

If we need to support unicode ids, then the right approach is to allow them to 
be of unicode type. It looks like OFS has the beginnings of supporting this 
(as your proposed code fragment pointed out), but Im not aware of it being 
used.

 I'm gonna go with that last one if nobody complains.

consider this a complaint. character encodings should be applied at zopes 
boundary, not in the middle of its processing.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] _p_deactivate() and _v_ variables?

2003-10-15 Thread Toby Dickenson
On Wednesday 15 October 2003 12:47, Chris Withers wrote:
 Casey Duncan wrote:
  I would argue that a better plan would be to only use _v_ vars for
  completely disposable data only. The application should expect that this
  values will be gone at any random time, not just at transaction
  boundaries.

 Agreed. Are there any situations, apart from the already discussed CMF
 skindata, where this currently isn't the case?

every database adapter?

(I guess, but havent checked)

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] _p_deactivate() and _v_ variables?

2003-10-15 Thread Toby Dickenson
On Wednesday 15 October 2003 14:53, Casey Duncan wrote:

  Agreed. Are there any situations, apart from the already discussed
  CMF skindata, where this currently isn't the case?
 
  I'm a bit puzzled - of what use is a variable which may disappear at
  any random time?

 It's not exactly random. It would happen when the object was deactivated
 (removed from cache).

The proposal earlier in the thread was aiming towards allowing objects to get 
deactivated at any time if the cache was overfull, not just at transaction 
boundaries. This is desirable from a cache management point of view.

Apart from the most trivial cases, it would allow _v_ attributes to disappear 
at random. Its a similar problem to the one that makes it hard to write an 
optimiser for python code, and I am unconvinced that this is sane.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] _p_deactivate() and _v_ variables?

2003-10-15 Thread Toby Dickenson
On Wednesday 15 October 2003 15:51, Casey Duncan wrote:

 As it is, the hard cache
 implementation, although beneficial from a memory management perspective
 cause loaded servers to do alot more work because they are constantly
 pruning the cache and then reloading objects again immediately thereafter.

If you are seening that then I think you need a bigger cache. And possibly 
fewer publisher threads.


-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-CMF] Re: [Zope-dev] _p_deactivate() and _v_ variables?

2003-10-12 Thread Toby Dickenson
On Friday 10 October 2003 18:34, Dieter Maurer wrote:
 Toby Dickenson wrote at 2003-10-10 07:54 +0100:
   ...
   A while ago there was a discussion on zodb-dev about _v_-like attributes
   that would be automatically cleared at the end of a transaction. Do we
   need something similar that guarantees it will _not_ be cleared until
   the end of the transaction?

 This definitely is necessary for the _v_ attributes that hold
 connections to a relational database.

 If such a _v_ attribute is flushed, the next access to the DA
 (in the same request) reopens the database. As this is a new
 connection, it does not see the changes made by the previous
 connection (in the same request).

Thats how alot of code works today, and how it had to be done in the past.

Today we have the 'transaction participant' interface. That would be a better 
place to hold these things, allowing the DA object itself to be deactivated 
if necessary. 

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] _p_deactivate() and _v_ variables?

2003-10-10 Thread Toby Dickenson
On Thursday 09 October 2003 14:01, Florent Guillaume wrote:

  I agree with this. How do we go about find code that uses the assumption
  that _v_ stuff won't change unless it's at a transaction boundary?

 Note that we had a problem related to this with a client recently: In
 CMF, skin data is stored in portal._v_skindata, and is actually needed
 for the whole request, but in some ZEO setting this go cleared by a
 get_transaction().commit(1) which was unexpected and led to breakage
 because in that batch treatment we used some skin methods too...

Something after the subtransaction commit must be tickling the cache garbage 
collector. Thats generally what subtransactions are used for.

A while ago there was a discussion on zodb-dev about _v_-like attributes that 
would be automatically cleared at the end of a transaction. Do we need 
something similar that guarantees it will _not_ be cleared until the end of 
the transaction?

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope] Re: [Zope-dev] Using 2.3.2 for Zope 2.7

2003-10-03 Thread Toby Dickenson
On Friday 03 October 2003 15:33, Chris McDonough wrote:
 Jim is keen to get an audit going quickly before a 2.7
 final release, and the audit would be performed against Python 2.3.2.

Does anyone else have an interest in blessing Zope 2.6.x with Python 2.2/2.3 ?



-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Re: [ZODB-Dev] ZODB 3.2 feedback

2003-09-30 Thread Toby Dickenson
On Monday 29 September 2003 22:25, Christian Reis wrote:
 On Mon, Sep 29, 2003 at 05:06:52PM -0400, Jeremy Hylton wrote:
  Has anyone had a chance to test ZODB 3.2b3?  We're hoping to do a
  release candidate tomorrow, and it would be good to know if anyone has
  tried to the beta release.  If you've got a current CVS checkout of Zope

 I'm unable to test because of the Python 2.2 requirement :-(

Im in the same position, also :-(


-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] _p_deactivate() and _v_ variables?

2003-09-26 Thread Toby Dickenson
On Friday 26 September 2003 09:32, Chris Withers wrote:
 Toby Dickenson wrote:
  On Thursday 25 September 2003 11:51, Chris Withers wrote:
 Hmmm, does _p_deactivate() clear the contents of the object's _v_
 variables?
 
  Yes

 Then given your earlier comment that _v_ variables are supposed to last at
 least teh length of the request, John's idea of using _p_deactivate() to
 reduce memory usage for large ZCatalog results sets could be seen as
 playing with fire, right?

He had code that would only deactivate objects at the end of a loop if they 
were not active at the start of the loop. Its not entirely safe, but I think 
it mitigates the most serious risks here.

ZCatalog (and others) have used this same technique for a while.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] _p_deactivate() and _v_ variables?

2003-09-25 Thread Toby Dickenson
On Thursday 25 September 2003 11:51, Chris Withers wrote:

 Hmmm, does _p_deactivate() clear the contents of the object's _v_
 variables?

Yes

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: Catalog performance

2003-09-11 Thread Toby Dickenson
On Thursday 11 September 2003 03:03, John Barratt wrote:

 I think ghosts
 are only 'removed' after a restart, 

fyi, ghosts are removed from memory using reference counting.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Cache growing during single REQUEST

2003-09-05 Thread Toby Dickenson
On Wednesday 03 September 2003 12:15, Chris Withers wrote:

 If the object load would cause the cache to go above it's maximum number,

*Number* isnt the the right parameter to control here. We need to limit 
the total amount of RAM. Objects are of variable size, and the largest ZODB 
objects are very much bigger than the average.

One task I would like to find time for is making the cache aware of this. For 
example, this would fix the problem where the current cache unfairly 
penalises ZCatalog operations because its BTree nodes are so small.

 then boot an object out of the cache in order to make room for the new one.

That would have a bad effect on ReadConflictErrors. Cache purging should only 
happen on transaction boundaries for storages where ReadConflictErrors are 
possible.

 control the amount of memory Zope
 actually uses and other requests would stand a chance of beind processed
 normally.

I havent seen a mention of ulimit or autolance earlier in this thread They 
are mostly adequate protection against the work problems.


-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Cache growing during single REQUEST

2003-09-05 Thread Toby Dickenson
On Friday 05 September 2003 14:21, Chris Withers wrote:

 Hmmm, maybe not... could we make a note of the pickles size when the data
 is loaded and update that size when it's comitted? Is this the same as the
 in-memory size?

Yes, I hope its close enough for most purposes. It would be nice to have a way 
for the object to override that in the cases where it is badly wrong.

 then boot an object out of the cache in order to make room for the new
  one.
 
  That would have a bad effect on ReadConflictErrors.

 Don't follow, can you explain?

You get a ReadConflictError when loading an object if it has been modified 
since the start of the transaction. This exception therefore becomes 
increasingly likely as time progresses since the start of the transaction.

Today you can minimise the probability of a read conflict by touching the 
objects you need (at least the ones likely to change) near the start of the 
transaction. This works because, once loaded, the objects stay in memory 
until the end of the transaction.


Another problem I didnt mention is that _v_ attributes are supposed to last at 
least until the end of the transaction.

  I havent seen a mention of ulimit or autolance earlier in this thread
  They are mostly adequate protection against the work problems.

 Do they kill the thread or the whole Zope?

whole zope

of course there isnt a great overhead in using twice as many zopes, each with 
half as many publisher threads. (assuming you already have zeo, of course)

 I'm also keen for users to nto
 get MemoryErrors, but to just have their request take much longer ( cache
 thrashing and the like...)

use squid, and it will retry the request.

-- 
Toby Dickenson


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] UndoSearch Timeout

2003-08-07 Thread Toby Dickenson
On Wednesday 06 August 2003 22:05, Brian Brinegar wrote:
 Hello,

 We have a ZODB which is about 12 - 14 gigs when packed. We use
 FileStorage, and when anyone clicks Undo eventually times out. This is
 better now that the undoLog does not lock the database.

 However if there are not enough transactions to fill the batch Undo is
 useless. So, everytime we upgrade to a new version of Zope I patch
 FileStorage.py to add a timeout. Used to be 3 seconds when the ZODB
 locked, now I've bumped it up to 10 seconds.

 Has anyone else had this problem? Would adding the timeout, possibly set
 by an environment variable, be considered by the maintainers?

DirectoryStorage has always provided a timeout, set by config file. There have 
been occasions where I wanted to wait for the full detail, and a global 
timeout setting was inconvenient. Is it worth adding it as a new parameter to 
these functions?

-- 
Toby Dickenson - http://www.geminidataloggers.com/people/tdickenson

Want a job like mine?  http://www.geminidataloggers.com/jobs for Software
Engineering jobs at Gemini Data Loggers in Chichester, West Sussex, England


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] segfaults in cPersistence under 2.6

2003-07-25 Thread Toby Dickenson
On Friday 25 July 2003 08:30, Anthony Baxter wrote:
 More information: I don't see the failure with a fresh Data.fs. The
 Data.fs in question is my 2.5 one.

Do you have any custom persistent extension classes stored in that Data.fs 
that might not have been updated to the 2.6 persistence C API?

-- 
Toby Dickenson - http://www.geminidataloggers.com/people/tdickenson

Want a job like mine?  http://www.geminidataloggers.com/jobs for Software
Engineering jobs at Gemini Data Loggers in Chichester, West Sussex, England


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: FHS, zopectl, #925, Re: [Zope-dev] 2.7 installation

2003-06-19 Thread Toby Dickenson
On Thursday 19 June 2003 15:58, Shane Hathaway wrote:

 Has anyone tried
 extracting Gentoo's build system and using it for partial software
 distributions?  That might be the way to go.

I am now using Gentoo on all my servers and on this workstation.

I find value in being able to manage Zope stuff as a non-root user. Portage 
was designed to allow this, but I doubt it is well exercised. Trying this has 
been on my to-do list for a while.

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] version status

2003-06-17 Thread Toby Dickenson
On Tuesday 17 June 2003 09:01, Oliver Bleutgen wrote:

 I don't quite understand the nature of this DOS attack after the patch.
 You do requests with REQUEST['Zope-Versiom'] == big string.
 If I understand your code correctly (it was bash and perl afterall ;))
 you create version i with a version name str(i)*50.
 It seems (to me) that the sole cause for this DOS is that zope stores
 the version names in memory, that means you get a memory consumption for
 all version name strings of 10*50 + 90*50*2 which is 95.000.000
 bytes, which is roughly the 90M you reported.

The connection cache will also store a cached connection for each version. The 
connection is opened to *read* from the storage; no writes are needed.

A more 'efficient' attack would be to use a tiny (but unique) Zope-Version 
string to request a page that loads alot of zodb objects into the connection 
cache, for example as a seach page.

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZMI tweak

2003-06-16 Thread Toby Dickenson
On Saturday 14 June 2003 16:53, Christian Theune wrote:
 When catching up with the mail in the mailinglist, I remembered a small
 annoyance I'm having with the ZMI. I really appreciate the errorlog and
 am using it frequently. But if you don't put it it's own window (or tab)
 you always need two clicks to reach it. I once put it's icon in the
 upper ZMI bar to move me to the servers central error log.

 Is this something that others would like too?

I have a number of objects that provide site-global information like the error 
log. I have occasionally considered developing a mechanism to allow them to 
hook into the global gui; either the top bar or Control_Panel.


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] ZMI tweak

2003-06-16 Thread Toby Dickenson
On Monday 16 June 2003 08:05, Toby Dickenson wrote:
 On Saturday 14 June 2003 16:53, Christian Theune wrote:
  When catching up with the mail in the mailinglist, I remembered a small
  annoyance I'm having with the ZMI. I really appreciate the errorlog and
  am using it frequently. But if you don't put it it's own window (or tab)
  you always need two clicks to reach it. I once put it's icon in the
  upper ZMI bar to move me to the servers central error log.
 
  Is this something that others would like too?

 I have a number of objects that provide site-global information like the
 error log. I have occasionally considered developing a mechanism to allow
 them to hook into the global gui; either the top bar or Control_Panel.

To throw in an extra idea...  For a long time I have been using a patch that 
puts BASE0 and the title of the root folder in the ZMI top bar. The reasons 
for wanting this should be appreciated by anyone who has ever had two browser 
windows open for two different zope sites, and made a change in the wrong 
one.

The patch is in toby-zmi-title-branch (the branch tag is only on the 
lib/python/App/dtml directory)

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] version status

2003-06-16 Thread Toby Dickenson
On Sunday 15 June 2003 08:11, Jamie Heilman wrote:
 Whats the status of versions for 2.6.2 and 2.7?  Have there been any
 decisions reached?  I saw Jim's code get checked in but it won't
 stop the DoS I posted.

Ive not tested Jims code, but it looks to me like it *should* stop that 
attack. Have you tested it?

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: small summary and big plea was:(Re: [Zope-dev] Versions: should they die?)

2003-06-10 Thread Toby Dickenson
On Friday 06 June 2003 21:28, Jamie Heilman wrote:

 Quick way to add 100 zodb connections and ~90M to the memory footprint
 with relatively little clue of who is responsible assuming traditional
 logging; presumeably one would get much trickier if they really wanted
 to obfuscate the source of attack, slowly crawling the site, changing
 the user-agent string, etc. 

Attached is a temporary patch to block this denial of service attack. This 
patch applies cleanly to the trunk and the 2.6 branch. This patch works by 
blocking all access to versions in the publisher, so dont apply it if you 
cant afford to stop using versions

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickensonIndex: lib/python/ZODB/ZApplication.py
===
RCS file: /cvs-repository/Zope/lib/python/ZODB/ZApplication.py,v
retrieving revision 1.13
diff -c -2 -r1.13 ZApplication.py
*** lib/python/ZODB/ZApplication.py	8 Apr 2003 18:48:22 -	1.13
--- lib/python/ZODB/ZApplication.py	9 Jun 2003 22:58:43 -
***
*** 42,49 
  
  def __bobo_traverse__(self, REQUEST=None, name=None):
! db, aname, version_support = self._stuff
! if version_support is not None and REQUEST is not None:
! version=REQUEST.get(version_support,'')
! else: version=''
  conn=db.open(version)
  
--- 42,53 
  
  def __bobo_traverse__(self, REQUEST=None, name=None):
! # Disable nasty insecure version support. Thanks to
! # Jamie Heilman and everyone one zope-dev
! #
! # db, aname, version_support = self._stuff
! # if version_support is not None and REQUEST is not None:
! # version=REQUEST.get(version_support,'')
! # else: version=''
! version = ''
  conn=db.open(version)
  


Re: small summary and big plea was:(Re: [Zope-dev] Versions: should they die?)

2003-06-10 Thread Toby Dickenson
On Tuesday 10 June 2003 09:32, Jamie Heilman wrote:
 Toby Dickenson wrote:
  ! # Disable nasty insecure version support. Thanks to
  ! # Jamie Heilman and everyone one zope-dev

 Unless you're damning me with faint praise for posting an exploit,
 (which is fine)

No criticism was implied public exploits are valuable part of the security 
process.

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: small summary and big plea was:(Re: [Zope-dev] Versions: should they die?)

2003-06-06 Thread Toby Dickenson
On Friday 06 June 2003 15:04, Shane Hathaway wrote:

 I think 2.6 ought to fix this by disabling recognition of the
 Zope-Version cookie

Setting this individually for each http port would better support existing 
happy users of this feature. (Im sure there must be some ;-)

Being able to set up a port that ignored cookies but always used a specific 
Version would avoid many of the existing problems too.

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Versions: should they die?

2003-06-05 Thread Toby Dickenson
On Wednesday 04 June 2003 17:40, Brad Clements wrote:
 Sorry if is OT.

 I'd like ZODB and Zope to support Revisions. That is, historical copies
 that do not get removed when ZODB is packed.

DirectoryStorage allows you to designate classes that should have all their 
history kept indefinitely. That may not have the flexibility that you need...

 Does the Version mechanism contribute to this kind of functionality? I'd
 like to be able to tag revisions of wiki pages and other documents, and
 then be able to diff them later without having to create a special kind of
 product for each object type that I'd like to use revisions for.

The accepted wisdom is that feature like this should be implemented in the 
application, above zodb. This has come up a few times on zodb-dev list.


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Re: [ZCM] [ZC] 869/ 5 Comment Broken transaction handling in case of exceptions

2003-04-04 Thread Toby Dickenson
Lets take this back to the list Ive trimmed the cc list to people I think 
might be interested.

http://collector.zope.org/Zope/869

 I am not in line with your principle the error report is
 part of [the output of] the first transaction. 

The error report tells you what went wrong. You can only generate an accurate 
report if you see the objects in the error state.  Zope is saying to your 
application: your transaction is doomed. tell me why.

As you said:
 Drawback: you may see newer data in the error report than
 has been seen by the erroneous transaction.
I think thats a *major* drawback. The risk of zodb badness is a major drawback 
too. Small risks add up over time.

 But, even with Zope, when the error report should do
 something persistently, it must do it in a new transaction
 as the old state is inconsistent and can not be committed.

I agree that it is unacceptable to commit changes made during the error report 
without starting a clean new transaction. I agree my proposal is inadequate 
if we need to allow this.

Currently Zope doesnt allow the error report to do something persistently, and 
the use case for this is not obvious to me. Can you explain the use case? I 
am hoping that there may be a better solution that avoids the problems I 
raised.

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: [ZCM] [ZC] 869/ 5 Comment Broken transaction handling in case of exceptions

2003-04-04 Thread Toby Dickenson
 The view has access to the original request that ended in the error. The
 view is looked-up and rendered in a new transaction.

A new transaction means that *arbitrary* changes may have occurred to the 
application state in between:
1. the transaction that raise the error, and
2. the transaction that reports it

In principal it is posssible to delete the database state and import a whole 
new application between these two points. The object containing the 
originally published method might no longer exist. I guess that might not 
matter if your error page just says sorry an internal error has occurred - 
if you want your users to be shielded from the details of any problem. I have 
some applications at the other extreme, where the users are all capable of 
debugging any problem and I want the error page to be a core dump of 
application state. 

 Can you give an example of the kind of situation where you'd need access
 to objects in a doomed transaction in their doomed state, in order to
 make an error report?

Access to the doomed state at the end of the transaction is what zope 2 gives 
us today. Access to the objects restored to their state as they were at the 
beginning of the failing transaction would be ok too (maybe even better) but 
zodb cant do that today. Either of these two are necessary for easily 
providing an error report that displays application state explaining why the 
error occurred. 


Under the Zope 3 model I think I would need to take a non-persistent copy of 
the application state in an exception handler towards the end of the first 
transaction. The second, error-reporting transaction would dump the copy, 
rather than dump what it sees to be the current world state. 

It is important that the objects passed from one transaction to the next are 
non-persistent, otherwise bad zodb-level things can happen. Does Zope 3 guard 
against this? 

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Session Errors (read conflicts)

2003-03-18 Thread Toby Dickenson
On Tuesday 18 March 2003 3:24 pm, Chris McDonough wrote:

  Our live sessions code uses the sessions about half to two-thirds of the
  way through the transaction. Given what can happen in that first half,
  there is easily plenty of time for read conflicts. I think I might be
  able to move our session use to the beginning of the transaction (just
  storing stuff in the REQUEST object for later usage) and hopefully fix
  our issue.

Depending on what your sessioning involves, you might not need to change when 
you do all the work. Possibly just touching (an attribute of) the 
persistent session object will be enough.

 But I'm not sure I understand why touching an object at the start of a
 transaction would fix the consistency problem.  If a object A is read at
 the beginning of a transaction that uses the data from object A during
 write of object B and later object A changes and is invalidated by
 another transaction (before the first finishes), no conflict errors will
 be raised.  Isn't there the potential for the first transaction to write
 inconsistent data into object B?

Today you are guaranteed that your transaction reads a consistent initial 
state of A and B. That is, there is no chance that you only see half the 
changes of a recent transaction that modified both.

Yes, it is possible for one transaction to modify A and a concurrent 
transaction to modify B. This is what Deiter describes as not safe in that 
zodb-dev thread. IMO, it is only not safe if you are not aware of it. And it 
is critical for performance with concurrency - consider these two 
transactions occuring on different zeo nodes.

  In my mind, this is the same thing as
 using a low conflict connection (ie. you are essentially turning off
 read conflicts, albeit by circumstance rather than by code).

There is a significant difference. The low conflict connection permits the 
possibility of seeing half the changes of a recent transaction that modified 
both objects.

low conflict connection is a misleadnig name. The conflicts are still there 
- just not reported as exceptions. low consistency connection would be more 
accurate ;-)



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Session Errors (read conflicts)

2003-03-18 Thread Toby Dickenson
On Tuesday 18 March 2003 6:12 pm, Chris McDonough wrote:
 I'm
 thinking that I also may just need to move the housekeeping duties to a
 separate scheduled thread that only happens when the system is not
 busy (e.g. when the asyncore poll select timeout is reached maybe) in
 order to reduce the potential for conflicts.

Thats hard to define, when zeo is installed.

   But I'm not sure what else
 to do, and I still don't understand what is causing the KeyErrors in the
 storage code.  Sigh.

Ive never looked at nor used your Sessioning support, but I am interested in 
things that can cause (POS)KeyErrors in Storages.

This might not be a storage bug. There are several known corner cases in zodb 
that mean semi-legitimate applications can cause KeyErrors. DirectoryStorage 
tries to guard against them, and will raise a DanglingReferenceError rather 
than commit a transaction that will cause a POSKeyError when read. Can the 
problem be reproduced using DirectoryStorage? (without the low consistency 
connection)

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Session Errors

2003-03-17 Thread Toby Dickenson
On Saturday 15 March 2003 5:40 pm, Chris McDonough wrote:

 - The second is an unidentified (yet) bug in TemporaryStorage. 

As a cause of the KeyError exception? Could be I have seen equivalent 
exceptions with plain FileStorage in applications that do not use sessions, 
so I suspect there might be a hard-to-hit zodb bug that can cause this too. 
(In FileStorage the equivalent exception is POSKeyError, a subclass of 
KeyError)

 If I find and fix the TemporaryStorage bug I will let you know. In the
 meantime, you can probably work around this by using a different
 non-undoing storage to put your session data in (e.g. Berkeley, or maybe
 DirectoryStorage?).

It would be interesting to hear if this exception is repeatable with the 
session state stored in a FileStorage too.

   You may see many more conflicts with this running.  But maybe the data
   structures will not become desynchronized.
 
  You weren't kidding about the increase in conflict errors.

Read conflicts occur if a change is committed in between the start of a 
transaction, and the transaction needing to load the object. A workaround to 
reduce the number of read conflicts is to touch the objects that are likely 
to change early in the transaction.


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] How (in)secure is Zope?

2003-03-13 Thread Toby Dickenson
On Thursday 13 March 2003 5:21 am, Shane Hathaway wrote:

  The only vulnerability would involve
 trusted users who want to vandalize Zope.  So even though there have
 been many hotfixes, they are irrelevant--Zope is still secure. (Unless
 you can't trust your trusted users, which is a different problem.)

Of course you cant *completely* trust your trusted users. Thats why we have 
seperate user accounts, and seperate roles.

IMO:
Zope is sufficiently vulnerable to abuse from trusted users to justify 
concern. The normal zope development model is to consider normal python code 
as trusted - normal python code can do anything without security checks. Zope 
has many normal python methods that can be called by through-the-web code 
(after permission checking). In unix terms this is equivalent to having many 
setuid root programs. IMO concern can be justified without needing to find a 
specific exploit. From this point of view, Jamies advocacy of using Unix 
mechanisms to restrict this 'trusted' python code is valuable.


On Thursday 13 March 2003 1:58 am, Jamie Heilman wrote:

 I will go on record as saying that, recently, response times to
 security related issues in the Zope2 tree have been disapointing.
 Construe from that what you will.

It is hard to find time for security work among the feature rush of the cvs 
trunk, and without compromising the stability of the maintenance branch.

Would there be any interest from other developers in addressing these 
potential security issues in a *fork* starting with the 2.6 maintenance 
branch?

(reply-to set to zope-dev)

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] How (in)secure is Zope?

2003-03-13 Thread Toby Dickenson
On Thursday 13 March 2003 9:25 am, Lennart Regebro wrote:

 5. Protecting yourself against denial of service:
 Zope does not seem to crash if you send random data to it, and I have in
 logs seen attemps to overflow buffers and the like that obviously are
 attempt to crash or break in to other (MS) servers, without this
 affecting Zope at all.

There is evidence that this is not true.

 If you don't trust Zope in this, you can put
 Apache in front of it.

 In this sense Zope is again VERY secure.

Zope is insecure

Zope+Squid(or Apache or Pound)+OS resource limits+careful choice of products  
is secure

(Note that  I dont consider this a flaw in Zope.)
-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] How (in)secure is Zope?

2003-03-13 Thread Toby Dickenson
On Thursday 13 March 2003 2:33 pm, Chris McDonough wrote:
 Hi Jamie,

 I'm wondering if you might consider applying for checkin privileges.

Thats a good idea

 The host header issue

Good  catch on that bug btw,

 that you've uploaded several patches for is a
 bonafide problem for some users, but I think that most people with
 checkin privs feel that it isn't sufficiently dangerous to the majority
 of users to take the time out to review all of your patches 

I suspect most people with checkin privelidges dont know about this problem, 
because it wont have been sent to the public mailman list.

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope] Re: [Zope-dev] Proposed installation changes for review

2003-03-12 Thread Toby Dickenson
On Tuesday 11 March 2003 10:48 pm, Jamie Heilman wrote:

  You'd probably still want a single master config file for the whole
  thing, and a tool to check the configuration is valid separate from
  the process that uses the file to configure itself.

 Not I.  Large applications with a master config file are to be held
 with suspicion.  Their longevity inevitably suffers because they are
 difficult to adapt to new situations.

Im not sure the big config file approach is necessarily less adaptable than 
the big /etc directory approach. It is the details that make the difference 
- both approaches can be done well.

 Another way to ease configuration is to make things modular so its
 easier to visualize the flow of data.

There is no amount of reconfiguration that can improve this in Zope2. Zope3 
promises to fix this, but with modular python components rather than modular 
unix components. I would be interested in your thoughts on whether this makes 
a difference.

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Prevayler and Object Prevalence

2003-03-03 Thread Toby Dickenson
On Monday 03 March 2003 3:48 pm, John Ziniti wrote:
 Any ZODB developers read this article at /. and the accompanying
 one at developerWorks?  Would anyone care to comment about
 the relevance/comparisons of prevalence to the concepts of
 persistence used in ZODB and/or ZEO?

I had looked at this (and the python port) briefly last year 

The main advantage of ZODB over a relational approach is that it can 
*automaticly* add ACID conditions to application logic objects, without major 
code changes. It is possible to think about your application logic and ignore 
the ODB aspects. Until recently, using a relational store for objects meant 
adding intrusive persistence code and breaking encapsulation.

As I understand it, prevayler preserves encapsulation, but still requires the 
intrusive code changes. It relies on separating appliction logic from the 
object store. Application logic can only read and write to the object store 
using remote method calls, which prevayler calls persistent  command  
objects. Commands are processed  by a single threaded server process. 

Command objects have to be persisitent because durability is provided by 
writing them to a log file, and replaying commands on recovery. There is no 
way to apply transactional semantics to application logic, except by cramming 
all the work into one big command object.

(I hope that helps. Appologies in advance for any inaccuracies)

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] support for low level HTTP Response logging?

2003-02-25 Thread Toby Dickenson
On Tuesday 25 February 2003 12:08 pm, Romain Slootmaekers wrote:

 We need this in our 3th line support, were we want to be able to follow
 all ins and outs for singled out user (fi filtered on cookie).
 We also need to be able to turn this on/off at runtime.

I would do this in the front-end proxy. Set up a squid acl to catch this one 
user, and send his requests via tcpwatch (or similar) rather than direct to 
zope.


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: Bare except dangerous to ZODB?

2003-02-20 Thread Toby Dickenson
On Thursday 20 February 2003 5:17 pm, Jeremy Hylton wrote:

  There are application-level reasons to mark a transaction as doomed, and
  I would like to keep *that* code looking similar ;-). The transaction
  states approach would work in that context too, right?

 Here's a late answer:

 If an application needs to mark a transaction as doomed, it is supposed
 to call get_transaction().abort().  If a transactional resource manager,
 like a database connection, needs to mark a transaction as doomed, it:

   - returns False from prepare() -- the ZODB4 spelling
   - raises an exception in tpc_vote() -- the ZODB3 spelling

Thanks for that idea.

Normally in the Zope context, transaction and request boundaries are 
co-aligned. This assumption runs deep in zope. I can see problems because an 
application calling abort wont stop the publisher calling commit at the end 
of the request.

Having application state revert mid-request to the pre-transaction state seems 
like a bad idea. Commiting application changes made in the second half of the 
request seems bad too.

(All from theory - I ve not tested this)


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Trying to understand the nuances of a VerboseSecurity report ..

2003-02-20 Thread Toby Dickenson
On Thursday 20 February 2003 11:38 am, Jean Jordaan wrote:

 All I can think is that that object 'basic' lacks context for
 some reason (i.e. lacks an acquisition wrapper?). I can't think
 why it lacks context .. we use that idiom all over the app, and
 don't normally get any problems.

You need to look inside newItem. I guess it is returning the newly created 
object in the form it was created; without acquisition wrapper.

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] How can I tell what a Zope Process is doing?

2003-02-18 Thread Toby Dickenson
On Tuesday 18 February 2003 4:04 pm, Chris Withers wrote:
 Hi,

 I have a Zope instance whose memory usage is ballooning every so often (to
 800Mb or so :-S) Now, I can see using top that one Zope process is using a
 lot of CPU and I'm guessing it's doing some silly iteration that's dragging
 huge numbers of objects into memory.

If you have got a live one then Control Panel / Debug Information will 
show current  requests.

 As a related question, does the line get written to Z2.log before or after
 serving the request has been completed?

You mean the log that has a field for the number of bytes sent back to the 
client?

;-)

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Re: Bare except dangerous to ZODB? was Re: [Zope-dev] Accept-Charset hearders causing 500 internal server error.[correct but not lenient]

2003-02-12 Thread Toby Dickenson
On Tuesday 11 February 2003 11:04 pm, Leonardo Rochael Almeida wrote:
 On Tue, 2003-02-11 at 13:19, Shane Hathaway wrote:

  I'm thinking the veto should be added by Connection.commit() and
  Connection.setstate() whenever a conflict is about to propagate.  What
  do you think?


 Maybe I don't understand all the transaction code correctly, but
 shouldn't it be set in ConflictError.__init__()?

No. ConflictError exceptions can be created by the storage, but swallowed by 
conflict resolution code in the connection.

Shane is correct; we assert a veto at the point where the conflict is 
terminal.


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Re: Bare except dangerous to ZODB?

2003-02-12 Thread Toby Dickenson
On Tuesday 11 February 2003 5:21 pm, Jeremy Hylton wrote:
 On Tue, 2003-02-11 at 12:10, Shane Hathaway wrote:

 I'd like to do the transaction states, because it would keep the code in
 zodb3 and zodb4 similar. 

There are application-level reasons to mark a transaction as doomed, and I 
would like to keep *that* code looking similar ;-). The transaction states 
approach would work in that context too, right?

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )



Re: Bare except dangerous to ZODB? was Re: [Zope-dev] Accept-Charset hearders causing 500 internal server error.[correct but not lenient]

2003-02-11 Thread Toby Dickenson
On Monday 10 February 2003 8:47 pm, Shane Hathaway wrote:

  Great, so there's at least 133 things to examine to see if they
  could catch a ConflictError.  And I only wrote about 15 of those.
  The rest could be very time-consuming to audit.

 tal:on-error also catches all exceptions.  It could be made to catch
 all exceptions except ConflictError, but I don't feel like that's the
 right solution. 

A while ago I tracked down a bug in one of our products to a case where a 
mutator method failed half way. It raised an exception and left the object in 
an invalid state. This would be safe except for a dtml-try that swallowed the 
exception. 

Recently I have been experimenting with this attached module to ensure that an 
application-level object can veto a transaction. I guess it would be 
applicable to ConflicctErrors too.


from zLOG import LOG, WARNING, ERROR

class _veto:
def __init__(self,msg):
self.msg = msg

def sortKey(self):
return repr(self)

def _do_nothing(self,*args):
pass
tpc_finish = tpc_abort = abort = abort_sub = _do_nothing

def _veto(self,*args):
try:
LOG('Veto', ERROR, self.msg)
except:
pass
raise TranactionVetoError(self.msg)
__getstate__ = tpc_begin = commit = tpc_vote = commit_sub = _veto

class TranactionVetoError(Exception):
pass

def veto(msg=None,ob=None):
Call this if your object is in an invalid state,
it is about to raise an exception, and you want to be sure that
the current transaction does not complete even if something
swallows that exception.
msg  - An error message describing the problem. optional.
ob   - The object which is now invalid. optional. If specified,
   the transaction will only abort if this object is
   persisted.

m = 'This transaction had been vetoed following a previous error'
if ob is not None:
try:
m += ' in '+str(ob)
except:
m += ' in unprintable object'
if msg is not None:
try:
m += ':\n'+str(msg)
except:
m += ':\nunprintable error'
print 'nnn',m
if ob is not None:
try:
ob._transaction_veto_attribute_that_can_not_be_persisted = _veto(m)
except:
pass
else:
return
# Drop through if we can not set our veto attribute on that object,
# or if no object was specified
get_transaction().register(_veto(m))



Re: [Zope-dev] [Bug] Zope's transaction behaviour flawed

2003-02-03 Thread Toby Dickenson
On Sunday 02 February 2003 3:40 pm, Dieter Maurer wrote:

 This is flawed as error handling is done outside of a transaction.

Excellent analysis. A futher problem is that this could cause dangling 
references, and a subsequent POSKeyError, since persistent objects can be 
passed from one transaction to the next inside the exception and traceback.

The same applies to your prorosed fix. Is there a need to allow the error 
handling transaction to commit? I propose it always be aborted.

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] [Bug] Zope's transaction behaviour flawed

2003-02-03 Thread Toby Dickenson
On Sunday 02 February 2003 3:40 pm, Dieter Maurer wrote:
 Zope's current transaction behaviour is essentially:

 1   ## request starts
 2   transaction.begin()
 3   try:
 4object= REQUEST.traverse(...)
 5mapply(object,...)
 6transaction.commit()
 7   except:
 8transaction.abort()
 9handle_error()
 10  ## request ends


 This is flawed as error handling is done outside of a transaction.

Potential changes during the error handling spill over
uncontrolled into another request and are there
either committed or aborted as part of this request.

A visit to the dentist has given me some time to think about this, and I think 
there is a flaw in this explanation. Surely any changes that leak out from 
one request will will be be aborted before they can do any damage, during the 
call to transaction.begin() on line 2.

The doc string from transaction.begin is:

'''Begin a new transaction.

This aborts any transaction in progres.
'''

(I should have a chance to experiment with this tomorrow)

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )



[Zope-dev] Re: [Zope-Coders] FYI: Zope 2.6.1 b2 scheduled for friday

2003-01-30 Thread Toby Dickenson
On Wednesday 29 January 2003 8:50 pm, Brian Lloyd wrote:

 (I saw Toby had committed some things, and it looks
 like that merge is complete).

Yes, that is complete


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] product refresh and random number generation.

2003-01-28 Thread Toby Dickenson
On Tuesday 28 January 2003 11:31 am, Romain Slootmaekers wrote:

 I'm having trouble understanding what is happening:

You are storing an instance of the Random class in ZODB, but not telling ZODB 
about changes to its internal state.

  def next(self):
  
   returns next random number.
  
   self._p_changed = 1
  return self.__random.random()



-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2003-01-28 Thread Toby Dickenson
On Sunday 26 January 2003 7:25 pm, Kazuya FUKAMACHI wrote:

 Have you decided which version the patch should be included?

 I combined 4 of the patches and Yousei Tahara's patch into single patch
 and published for japanese users on Jan 15.
 http://www.atransia.co.jp/home/ZenKai/Members/kafka/patch/Zope261b.patch

Thanks to everyone who has helped on this. This patch is in cvs ready for 
2.6.1 beta 2.

Please give the beta a good work out.



-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2003-01-27 Thread Toby Dickenson
On Sunday 26 January 2003 7:25 pm, Kazuya FUKAMACHI wrote:
 On Tue, 14 Jan 2003 10:33:40 +

 Toby Dickenson [EMAIL PROTECTED] wrote:
  I am currently looking at getting this into 2.6.1 or 2.6.2. I would
  appreciate confirmation that you are all happy with this combination of
  patches.

 Have you decided which version the patch should be included?

I am trying to get confirmation of the release schedule for 2.6.1 beta 2. If 
only I had known it would be delayed this long back in December...

 I've announced the patch by Japanese  Zope Users Group Mailing list,
 which  has more than 1000 subscribers.
 More than 100 people downloaded it, and any no complaints heard.
 Many said they were happy with the patch.

Thats good news. Thankyou.

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] 2.6.1b2?

2003-01-27 Thread Toby Dickenson
On Saturday 25 January 2003 1:07 am, Guido van Rossum wrote:
  Guido van Rossum wrote:
   If only it were that easy.  Everybody wants no new features except
   for the one feature they desperately want to have without upgrading to
   the next feature release. :-(
 
  Still, we're back to the situation where it's been weeks since the
  last release, and there's nothing we (the public) can do about that
 
  :-(

 If you want to make a rogue release, that will be available
 until Brian deems the time ready for the next release, be my guest.

Is there an estimate for this?

afaict, the public schedule since December has been that this is imminent any 
day now. I have some important bug fixes that I would like to include in 
2.6.1, and I would like some assurance that beta 2 will not be released 
mid-merge.


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope reference counts

2003-01-22 Thread Toby Dickenson
On Wednesday 22 January 2003 11:43 am, Ivo van der Wijk wrote:
   Do these
   references point to the same object instances, or are these copied
   as well?
 
  They are independant copies

 Why exactly is this? concurrency issues?

Yes.

  Is there a
 way to optimize this?

1. Obviously, use fewer threads. Do you have a component that otherwise 
benefits from many threads? 

2. If you have large immutable state that does not hold references to 
persistent objects, use application-specific code to share this state between 
threads. 

  I assume it's an issue on the ZODB level?

yes

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope reference counts

2003-01-22 Thread Toby Dickenson
On Wednesday 22 January 2003 3:01 pm, Ivo van der Wijk wrote:

 It soudns asif a further upgrde is reasonable.

I think you will have much more joy with 2.6

  Also, the 'activity' tab
 is new to me (looks interesting ;), and caching settins seem to be ignored
 mostly until 2.6, right?

it worked in a way that lead to strange behavior. 2.6 is more predictable, and 
understandable.


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope reference counts

2003-01-21 Thread Toby Dickenson
On Tuesday 21 January 2003 4:14 pm, Ivo van der Wijk wrote:

 Does this mean each thread keeps it's own cache pool of objects?

yes

 Do these
 references point to the same object instances, or are these copied
 as well?

They are independant copies

 I have the impression the memoryconsumption grows with the number of
 threads - is this true?

yes

 Cheers

I hope this helps

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope and Python compatibility

2003-01-14 Thread Toby Dickenson
On Monday 13 January 2003 7:46 pm, Casey Duncan wrote:

 I personally feel like Zope should run on 2.2.x whether required or not.
 Requiring 2.2.x does have the advantage of limiting the possible blessed
 configurations on which Zope will run, thereby making development, support
 and bug fixing that much easier.

Im not sure thats true. Having less strict version requirements means that 
there will be *more* *people* doing the development, support, and bug fixing. 

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2003-01-14 Thread Toby Dickenson
On Saturday 21 December 2002 10:54 pm, Kazuya FUKAMACHI wrote:

 I applied Toby's patch + Yousei's patch, and it works.

 Thanks to your neat piece of work,
 I can start to work on Zope 2.6.0 now, and go farther.

I am currently looking at getting this into 2.6.1 or 2.6.2. I would appreciate 
confirmation that you are all happy with this combination of patches.

thanks,

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] zope and python compatibility

2003-01-12 Thread Toby Dickenson
On Saturday 11 January 2003 4:06 pm, Jamie Heilman wrote:
 Guido van Rossum wrote:
   Without python 2.2 zope will continue to harbor remotely exploitable
   zlib-based memory exhaustion attacks. FWIW

There are workarounds that work in 2.1.x, and I dont think any uses in Zope 
are performance-critical. I will be happy to push through any patches for any 
other Zopes uses.

  Can you explain?  Where does Zope even use zlib?

 dtml-tree for one,

That is the least of your worries for dtml-tree. Please try the patch at:
http://www.zope.org/Members/htrd/tree.diff

This has a chance for 2.6.2 if I get enough positive feedback from people who 
actually use dtml-tree. (and a sufficently paranoid review would be nice 
too.)

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] zope and python compatibility

2003-01-12 Thread Toby Dickenson
On Sunday 12 January 2003 8:16 pm, Guido van Rossum wrote:
  On Saturday 11 January 2003 4:06 pm, Jamie Heilman wrote:
   Guido van Rossum wrote:
 Without python 2.2 zope will continue to harbor remotely
 exploitable zlib-based memory exhaustion attacks. FWIW
 
  There are workarounds that work in 2.1.x, and I dont think any uses in
  Zope are performance-critical. I will be happy to push through any
  patches for any other Zopes uses.

 I expect I'll be releasing Python 2.1.4 in the next month or so.  Can
 you submit a patch and assign it to me?

The fix in python 2.2 is in rev 2.44 of zlibmodule.c.  It involves an API 
addition to the zlib module, which I understand is discouraged for a Python 
bug fix release?

I think the workarounds in Zope are a better solution for Zope 2.6.x. We can 
gradually migrate these to the new zlib API once Zope has other dependencies 
on Python 2.2

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Python method wrapping around DTML method

2003-01-03 Thread Toby Dickenson
On Friday 03 January 2003 10:13 am, Bjorn Stabell wrote:

 Right now, I'm struggeling because I cannot get the
 namespace to be passed to PlainFunc, therefore PlainFunc cannot pass the
 namespace on to DTMLFunc.  Any clues?

There isnt one true namespace. You can create your own as easy as { }, and it 
will be as good as any other.

If you want an authentic ZPublisher namespace then you need

http://www.zope.org/Members/htrd/howto/FunctionTemplate

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-12-21 Thread Toby Dickenson
On Friday 20 December 2002 6:26 pm, Yusei TAHARA wrote:

 1. browsers autodetection make a mistake.

Not a browser mistake - this is *my* mistake. thanks.

 2. management_page_charset_tag really needs?

Yes, I agree with your analysis.

 I made a patch for fix those problems. It looks good work on my zope.

Good work. This definitely looks viable for 2.6.2

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Re: Collector 697 should be critical!!!

2002-12-20 Thread Toby Dickenson
from ages ago

   In OFS/dtml/properties.dtml I replaced
  
 select name=xyz:utf8:list multiple
  
   with:
  
 select name=xyz:utf8:list:string multiple
  
   It works for me, but I'm not sure, if this is a good approach on this
   problem (think of the integers as items for the multiple-select).
 
  I havent looked in detail yet, but my first impression is that this is a
  good approach.

 Well, I think that this is just a workaround, but not a bugfix ;)

This patch was committed on bug day

 Assume that ulines property is used to define the set of possible values
 for a multiple selection property. In this case one would expect to get
 a ustring vaule back.

 And as Maik already mentioned, it is even possible to use something like
 [1, 2, 3,4] as the possible values of the multiple selection . OK, this
 case does not work very well with older version of Zope too, but I think
 that this shows the general problem: Currently (including older Zope
 versions), there absolutely no type conversion for multiple selections.

Correct. That means that multiple selection can never work 100% with ulines 
without risking breaking old code.

Something like your suggested improvement would be one way to improve this. I 
think this would make a good fishbowl proposal

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-12-20 Thread Toby Dickenson
On Wednesday 11 December 2002 2:15 am, Kazuya FUKAMACHI wrote:

  5) Toby's proposal

 I hope +1.
 I'm not satisfied with (1)-(4).
 So, I would like to wait for Toby's implementation.

Ive not had as much time for this as I hoped, so please excuse any rough edges 
in these patch. All comments gratefully accepted.

http://collector.zope.org/Zope/737

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-12-20 Thread Toby Dickenson
On Friday 20 December 2002 2:22 pm, Kazuya FUKAMACHI wrote:

 if all of them are included in Zope 2.6.1.

The final beta is due today. If that goes to plan then it is already too late. 
2.6.2 is a realistic target.


-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-12-10 Thread Toby Dickenson
On Tuesday 10 December 2002 12:03 am, Kazuya FUKAMACHI wrote:
 On Mon, 09 Dec 2002 13:18:26 -0800

 Heiichiro NAKAMURA [EMAIL PROTECTED] wrote:
2. Whenever any experimental enhancements to the ZMI which rely on
using Unicode is to be integrated, create an new tab and put these
features in that page in order:

 Sounds reasonable.

 USA is #1 in Internet Users with 160M.
 http://www.etforecasts.com/pr/pr1202.htm

 At least 21.91% of Internet users are CJK users.
 (Japan + China + South Korea + Taiwan)
 Since using UTF-8 for their pages is still not in common
 in these area, it is preferable to accomodate such workaround.

I dont follow the logic here. Using unicode in the server doesnt force you to 
use utf8 in the browser.





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-12-10 Thread Toby Dickenson
On Monday 09 December 2002 9:18 pm, Heiichiro NAKAMURA wrote:
 On Mon, 9 Dec 2002 10:13:16 +

I agree with everything, except

   This is a tentative limitation on UI design, and should be removed
   after the Unicode Support gets matured enough to handle Unicode object
   with all languages/encodings.

Thats not right. The core unicode support has been mature in this area for 
over a year. The incompatability with pre-encoded strings is a design choice 
- not something that will improve with time.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-12-09 Thread Toby Dickenson
On Monday 09 December 2002 2:20 am, Heiichiro NAKAMURA wrote:
 On Sun, 8 Dec 2002 21:58:16 +

 Toby Dickenson [EMAIL PROTECTED] wrote:
  On Thursday 05 December 2002 9:36 pm, Toby Dickenson wrote:
   Yes, I have an idea. I hope to find time to flesh it out early next
   week.
 
  I propose:

 Here is my understanding of your proposal:



 1. management_page_charset property for non-Latin-1
 
 Affected application: ZMI only
 Limitation:
- Standard Objects in Zope (such as title property) must be
  Non-Unicode.

Yes, if using management_page_charset. 

- All objects must be Non-Unicode in order to work in ZMI.

To be precise, all objects rendered on the management page must be plain 8bit 
strings.

- If there is even one Unicode object there, all data will be
  assumed as 'Latin-1' and non-Latin-1 data cannot be used in
  any object.

yes

- REQUEST.set('management_page_charset','UTF-8') does not work
  when 'management_page_charset' is set as global property.

That is the current, broken behaviour. I propose that REQUEST.set overrides 
the global property. Note that no standard ZMI page will do so, except the 
'properties' tab explained below.

 Homework (new issues):
- define the safe steps of implementation of Unicode Support

- find a way of smooth migration of MBCS-to-Unicode

Yes. There is nothing zope-specific about this being hard ;-)

 


 2. Behaviour of handling of text in ZMI

Not all of ZMI - this is just the 'Properties' tab.

 
 When UnicodeType Properties are contained:

  - internal representation: 'UCS-2/UTF-16' if UnicodeType(ex. ustring)
 'Latin-1' if not UnicodeType(ex. string)
  - encoding of output data: 'Latin-1'

No, output over http will be utf8.

  - encoding of input data: 'Latin-1'

If you mean input from the browser when a property changes, no. it will use 
utf8.

 When UnicodeType Properties are NOT contained:
  - encoding of input data: any

yes, the same as all other ZMI pages. latin-1 by default, and can be overidden 
by management_page_charset.

  - internal representation: same as input (no conversion)
  - encoding of output data: same as internal (no conversion)

 Limitation:
  - Non-Latin-1 value cannot be used when creating the first Unicode
Property.
 

 (Any correction is welcomed)

 I'll ask power users of Japan-Zope-User-Group-ML about opinions
 regarding to the proposal.

  but I
  suggest that compatability with this feature should not hold back any
  future enhancements to the ZMI which rely on using unicode)

 I guess this statement is somewhat ambiguous. Probably we could say
 something like that? :
 
   1. Any future enhancements to the ZMI which rely on using unicode
   should be carefully defined, examined, evaluated and feasibility-tested
   so that all issues/limitations can be clarified and the consequent
   impact of these issues/limitation can be evaluated in order to
   avoid hassle implementation integrated into the official version.

yes

   2. Any experimental enhancements to the ZMI which rely on using unicode
   must be integrated into the official version in the manner that
   it doesn't affect the behaviour of 'legacy' applications, 

I disagree. I dont want to block a good new feature from Zope 2.x (x=7) 
simply because it uses unicode in a way that is incompatible with the 
management_page_charset property. Thats what 'legacy' means.

 until
   the Unicode Support gets matured enough to handle Unicode object
   with all languages/encodings.

Yes. There are several places in the ZMI that could benefit from 
'unicodization'. At the moment I suspect it will be difficult to implement 
these improvements while retaining support for management_page_charset. 



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Non Bloating Page Counter

2002-12-08 Thread Toby Dickenson
On Wednesday 06 November 2002 8:33 am, Janko Hauser wrote:

 Just an idea from the 'could-be-done-if-needed-department'. Generally
 there is quite often the need to have non-undoable properties in
 different objects of a site. There is one way to store it in the
 temporary storage, which is ram based, and not shared between ZEO
 instances. But if a second non-undoable storage is mounted it could be
 stored there and could be shared between ZEOC.

 To make this more transparent, one could perhaps implement something
 like a special propertysheetobject, which stores its properties in this
 mounted storage.

 Is there anything obviously wrong with this idea?

the problem is that cross-storage references are tricky.

Mixing undoable and non-undoable *objects* in the same storage is on the to-do 
list for DirectoryStorage, http://dirstorage.sourceforge.net


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-12-08 Thread Toby Dickenson
On Thursday 05 December 2002 9:36 pm, Toby Dickenson wrote:
 On Thursday 05 December 2002 8:41 pm, Heiichiro NAKAMURA wrote:
  Does anyone have any other idea for the Collector 623 issue?
  I hope better ideas will be posted..

 Yes, I have an idea. I hope to find time to flesh it out early next week.

I propose:

1. Currently you can set a management_page_charset property to override the 
global assumption that legacy ZMI pages are latin1. This is a hack that 
happens to mostly work by accident, provided your ZMI pages do not encounter 
a unicode object. I propose promoting this to a standard documented feature.

Is there anyone who actually uses this feature who can contribute some 
documentation?

(Note that feature will never work with pages that do encounter unicode 
objects. This means Zope should try to avoid gratuitous unicode usage, but I 
suggest that compatability with this feature should not hold back any future 
enhancements to the ZMI which rely on using unicode)


2. The documented way of setting a ZMI-page-specific encoding is 
REQUEST.set('management_page_charset','UTF-8'). Currently 
management_page_charset as a global property will override this page specific 
setting. This is a bug that needs to be fixed. 


3. I propose changing the encoding used by the standard 'Properties' ZMI page. 
Currently it uses UTF8 always, and assumes that 8bit string properties are 
latin1. I propose that this policy is used only if a unicode property has 
already been defined on this object. If it has not, it uses the same encoding 
policy as every other ZMI page - that is the value of the 
management_page_charset property, or latin-1 if it has not been set.

A disadvantage of this change is that it will not be possible to set a 
non-latin-1 initial value when creating the first unicode property. I think 
this is just about acceptable.


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] bug day

2002-12-05 Thread Toby Dickenson
On Thursday 05 December 2002 5:14 pm, Chris McDonough wrote:

 Developers and any other interested parties should gather via IRC on the
 #zope-dev channel on irc.openprojects.net at the time and date above.

I think its worth saying that the most important (IMO) outcome from the last 
two bugs days has been *concensus* about which bugs can be forgotten about, 
or rejected, and which are most important. You dont need to be a hardened 
zope developer to make a valuable contribution.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-12-05 Thread Toby Dickenson
On Thursday 05 December 2002 8:41 pm, Heiichiro NAKAMURA wrote:

 Does anyone have any other idea for the Collector 623 issue?
 I hope better ideas will be posted..

Yes, I have an idea. I hope to find time to flesh it out early next week.



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-12-03 Thread Toby Dickenson
On Tuesday 03 December 2002 1:16 pm, Yusei TAHARA wrote:
 Hi.

   I made monkey patch for myself, when management_page_charset is not
   UTF-8, this patch remove :utf8: from non unicode type input field.
   because if input values are not latin-1, then unicode error raised.
  
   I think that every encoding can be used it satisfactory:-)
  
   Please consider whether it can merge into Zope2.6.1.
 
  There are some details missing from your explanation, but hopefully not
  from your patch. where do I find it?

 What thing is concretely some details?

The fix to this situation is more complicated than removing a :utf8: from 
somewhere that it shouldnt be. Im sure you know this.

 I'm very interested in Zope development, especially i18n.
 So I would like to contribute something about it:-)

The patch you mentioned?



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Bugfix release?

2002-12-02 Thread Toby Dickenson
On Sunday 01 December 2002 3:36 pm, Brian Lloyd wrote:
   FYI I'd like to have a 2.6.1 beta out next week. Jeremy is still
   looking at a few ZODB bug reports - as soon as he's done we'll
   make the beta.
 
  Im not sure this is a good plan.
 
  Jeremy's sortKey changes look like they deserve a longer beta
  testing period
  than I would have thought the demand for a DateTime fix would allow.

 My understanding was that objects would not be required to implement
 sortKey (and wouldn't be adversely affected for not implementing it)
 unless they cared about the specific issue that it addresses. I'll
 verify this w/Jeremy tomorrow, but last I heard ordering should only
 actually be an issue for Connection objects - other objects that
 play with the transaction mgr will generally not be subject to the
 deadlock issue that the sorting is designed to prevent (the deadlock
 is due to connection interaction w/a storage server).

Yes, Jeremys changes should be entirely backwards compatible. I have commented 
on zodb-dev how well this has been handled. I am entirely confident that it 
can go into a 2.6.x fairly soon. However it is still a major change to a 
critical component, with some indications that things are not 100% right:

http://collector.zope.org/Zope/701
http://lists.zope.org/pipermail/zodb-dev/2002-November/003810.html

I am not normally shy of the bleeding edge, but I am still running the cvs 
dated 2002-11-11 in production. This one change makes me nervous.

A bug in the transaction manager would be really bad news. I would have 
thought it appropriate to resolve these problems and leave at least a few 
weeks of beta test before release.



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Re: Collector 697 should be critical!!!

2002-12-02 Thread Toby Dickenson
On Monday 02 December 2002 4:11 pm, Maik Jablonski wrote:

 I don't have much experience with the unicode-converters too... that's
 my problem for this issue...:-(

I had seen this bug on the list, but hant spotted that it was a 
unicode-related. I will try to take a look sometime this week.

 Abel Deuring told me a patch, which solves the problem in the meanwhile:

 In OFS/dtml/properties.dtml I replaced

   select name=xyz:utf8:list multiple

 with:

   select name=xyz:utf8:list:string multiple

 It works for me, but I'm not sure, if this is a good approach on this
 problem (think of the integers as items for the multiple-select).

I havent looked in detail yet, but my first impression is that this is a good 
approach.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-11-30 Thread Toby Dickenson
On Saturday 30 November 2002 5:39 am, Yusei TAHARA wrote:

 2. ustring can not join or replace to 8-bit strings other than ascii.

Yes, it is painful to work in a mix of pre-encoded 8 bit strings and unicode 
strings. Thats why Zope's unicode policy was intended to be entirely optional 
- at least in the short term - and provided you can avoid unicode objects 
entirely.

  Zope's approach is that your application (excluding the ZMI for now)
  should work as in 2.5, provided you dont use any unicode values. Can you
  confirm that this is working OK?

 yes.
 If I input values by  PythonScript without using property tab, it works
 well.

Hurrah.

  One area where this went wrong for you is the properties page, which is
  explicitly set to utf8 to allow unicode properties to be set. Im not yet
  sure on the best way to fix this for you, but I agree it needs fixing.

 I made monkey patch for myself, when management_page_charset is not
 UTF-8, this patch remove :utf8: from non unicode type input field.
 because if input values are not latin-1, then unicode error raised.

 I think that every encoding can be used it satisfactory:-)

 Please consider whether it can merge into Zope2.6.1.

There are some details missing from your explanation, but hopefully not from 
your patch. where do I find it?




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



Re: [Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

2002-11-29 Thread Toby Dickenson
On Friday 29 November 2002 5:44 am, Yusei Tahara wrote:
 Hi.

  The right approach is to make it possible to change the title property to
  a unicode string. All my custom products have this already, but it is a
  deficiency in the standard Zope types such as 'Folder' that their titles
  type can not be changed.

 This approach is not useful for me.

 I often use zope with RDB like MySQL.
 generally japanese encoding text is in RDB.

 so if object properties are unicode string,
 I always encode it before publishing.

 title tal:content=python:here.title.encode('euc-jp')TITLE/title


 because, it always mix in RDB data(japanese) and
 properties(python unicode string) in one page.

 certainly I will be faced with this ustring problem everytime.
 Japanese charactor is not ascii,
 so if I do not encode ustring before publishing,raise unicode error.

The unicode changes were designed to cope with your scenario, so I would like 
to explore the limits of where this approach has gone wrong.

As I understand it, you dont want to do anything in unicode. You store, 
manipulate, process, and output strings as 8-bit pre-encoded strings. You  
are making an assumption that these 8-bit strings use some specific character 
encoding. The scope of this assumption is quite broad. It has to cover:
1. All strings stored by your ZODB
2. All strings stored by your RDB.
3. All processing performed by Zope products. (must follow your assumptions, 
or be encoding neutral)

Anything that beaks these assumptions will need special treatment.

Am I right so far?


Im sure you are aware of the limitations of this approach - they are the 
limitations that unicode was designed to break (not Zope's unicode, but 
unicode in general). Im not sure your approach is workable long term because 
an increasing number of Zope products will naturally use unicode so that they 
can store properties containing text across a range of scripts. Even if this 
is not workable long term, it was supposed to work in 2.6.

Zope's approach is that your application (excluding the ZMI for now) should 
work as in 2.5, provided you dont use any unicode values. Can you confirm 
that this is working OK?

Zope 2.5 left ZMI character encoding down to browser autodetection, and as a 
result most ZMI-controlled properties are encoding-neutral. For compatability 
with unicode pages, which are seen as the long term future, the ZMI has to 
specify *some* character encoding. By default in 2.6 this is latin-1, a 
change which I think was announced mid-way through the 2.5 development cycle. 
I understand that some users are very happy overriding this with a 
management_page_charset property on their root folder. Ive never used this, 
and it wasnt designed to work this way, but it looks like it works due to 
happy coincidence. Note that this doesnt work for management pages which 
explicitly set their own character encoding, or management pages which 
'accidentally' encounter a unicode string.

(Although this management_page_charset feature was not planned, I guess I will 
be supporting this feature if it is being used)

One area where this went wrong for you is the properties page, which is 
explicitly set to utf8 to allow unicode properties to be set. Im not yet sure 
on the best way to fix this for you, but I agree it needs fixing.

As far as I can tell this is the only area which should be causing you 
problems. Am I missing something?



(The 'title' problem I mentioned in a previous post is also a problem that 
needs fixing, but not one that affects you if you dont want to use unicode.)





___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



[Zope-dev] Re: [Zope] post-publishing hook

2002-11-29 Thread Toby Dickenson
On Friday 29 November 2002 12:07 pm, Carlo Giomini wrote:
 Dear all,
 I need a sort of post-publishing hook (so to say). I need Zope to call a
 function of mine as the very last action of publishing a request, i.e.
 after having built the response (and sent it to the browser possibly), but
 before starting publishing a new request. My Zope server is running as a
 single-threaded app.
 Does Zope support this any way?

Yes, but its nasty. Look at ZPublisher.BaseRequest._hold



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )



  1   2   3   4   5   >