[Zope-dev] Proposed interface for application-level conflict resolution in ZODB

[Jim Fulton]

Zope takes a pretty optimistic approach to dealing with conflicting 
database changes. Sometimes, there may be "hot spots" in an application
that get a lot of simultaneous writes. These hot spots can often be designed
away. An alternative is to provide application-level logic for sorting
out the changes made by conflicting writes. The proposed interface at:
http://www.zope.org/Members/jim/ZODB/ApplicationLevelConflictResolution
provides a way to provide this application logic.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] ZEO on NT? update

[Jim Fulton]

Messages about ZEO should be sent to the [EMAIL PROTECTED],
http://lists.zope.org/mailman/listinfo/zope-zeo

julio dinis wrote:
> 
> Ok. :-)
> 
> Continuing with the ZODB.POSException.Conflit...
> 
> Now I know the problem with Zeo on NT is with the ClientStorage
> and not with the ServerStorage.
> You ask why? Because Ive tested different setup to come to this
> conclusion.
(snip)

This is very helpful. Our *current* focus is on Unix.
I've seen this problem on Windows, but haven't had time
yet to look into it, and probably won't very soon.
I certainly haven't done the sort of test you've done.

I don't think that the problem it pickle-related, since pickles
are platform independent. It appears to be some strangosity
with the async/networking code. 

Let me know if you figure anything out. :)

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: [Python-Dev] ExtensionClass and __radd__()?

[Jim Fulton]

Greg Ward wrote:
> 
> Hi all --
> 
> looks like ExtensionClass doesn't recognize/implement the '__radd__()'
> protocol.  Speculation below; first, a demonstration.  Normal case: a
> regular class Number that knows how to add itself to other number-like
> things:

(demonstration snipped)

> Speculation time: I'm guessing that this is similar to the problem with
> 'isinstance()' and ExtensionClass that I found several months ago, which
> was heroically debugged by Barry.  To recap, it's a mutual
> finger-pointing bug: Python (Guido) can claim that it's up to
> ExtensionClass (Jim) to emulate the full semantics of Python
> classes/instances, but ExtensionClass can claim that Python should be
> more relaxed in what it accepts as a "class object" or "instance
> object".
> 
> I think the relevant code in Python is in Objects/abstract.c,
> specifically 'PyNumber_Add()' and the BINOP macro:
> 
> #define BINOP(v, w, opname, ropname, thisfunc) \
> if (PyInstance_Check(v) || PyInstance_Check(w)) \
> return PyInstance_DoBinOp(v, w, opname, ropname, thisfunc)
> 
> [...]
> PyNumber_Add(v, w)
> PyObject *v, *w;
> {
> PySequenceMethods *m;
> 
> BINOP(v, w, "__add__", "__radd__", PyNumber_Add);
> [...]
> 
> My guess is that PyInstance_Check() returns false for ExtensionClass
> instances.  Two possible fixes: loosen up PyInstance_Check(), or loosen
> up BINOP.
> 
> Well, it's a nice theory.  It doesn't explain why '__add__()' works for
> ExtensionClass while '__radd__()' does not; perhaps ExtensionClass
> implements that much of Python's class semantics, but doesn't go as far
> as '__radd__()'.

I'd love to see __radd__ added. ;) I don't remember why it's not there.
Maybe I was just lazy.  It may be fairly hard to add. I haven't looked
in quite a while. As anyone whos looked at ExtensionClass sources may
be able to tell, ExtensionClass has to play quite a few tricks to:

- Try to sanely bridge the quite different semantics of Python
  "types" and "classes" (e.g. there's no radd for "types").

- Try to overcome the fact that the interpreter special-cases 
  InstanceType and ClassType pretty liberally. To (try to and
  mostly succeed to) provide instance semantics I have to do 
  alot of weird indirection. This is especially hard for 
  numeric things.

  Your analysis of the code demonstrates this issue. ExtensionClass
  instances are not of type InstanceType. In fact, each ExtensionClass
  is a separate type and instances of different ExtensionClasses have
  different types.

  Note that I just got around to responding to your earlier 
  post "Comparison inconsistency with ExtensionClass".
  This has a similar root cause: the special-case treatment
  of InstanceType.

I think that the *real* solution to this problem is to get rid
of ExtensionClass. ;) To do this, I need to get the features
I have in ExtensionClass into Python. I guess there's some hope
for that in Python 3K (love that name :).

In the mean time, I don't have time to fix the radd problem
myself, but would be willing to advise someone who wanted
to try to take it on, especially if we could work out some
phone or face-to-face sessions.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: Aquisition, in, == and is

[Jim Fulton]

Chris Withers wrote:
> 
> Chris Withers wrote:
> > Steve Alexander wrote:
> > > > Do you know if objects in PARENTS are acquisition wrapped?
> > > I'm pretty sure that they are.
> >
> > They are indeed, in fact, pretty much everything is :(
> > The only way to check if o is in PARENTS appears to be:
> 
> > if o.aq_base in map (lambda o : o.aq_base,PARENTS):
> 
> Hmm, is there anyway the Acquisition ExtensionClass(right thing? I'm not
> too hot here ;-) could be altered such that, if r1 and r2 are two
> different acquisition wrappers for the same object that:
> 
> r1 == r2 and r1 in [r2,x,y]
> 
> would return true?

Yes. I think that this would be a great idea.

Note that if r1 implements __cmp__, then it is called.
This means that you could implement the proposed behavior now
with:

  def __cmp__(self, o):
 return cmp(
id(getattr(self, 'aq_base', self)), 
id(getattr(o, 'aq_base', self))
)

It also means that (with the proposed change) that '=='
will only be equivalent to an identity comparison if
an object doesn't define comparison.  In most cases, however,
this shouldn't be a problem.

> It might be best to leave (r1 is r2) returning false as it does now so
> that you can actually tell if things aren't what you expected ;-)

Good, because there's no hook to override 'is'.

> PS: This question is still worrying me :(
> 
> > 2. Is there any case where a Zope object isn't going to have a .aq_base
> >attribute?

Sure. There is no guarentee that all objects are wrapped.

A common idiot for dealing with this is getattr(o, 'aq_base', o).
So your example above would become:

  if getattr(o, 'aq_base', o) in map (
     lambda o : getattr(o, 'aq_base', o),
 PARENTS)  

I've considered implementing the aquisition attributes in
the Implicit and and explict base classes, which would probably
be a good idea.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Request for Comment: Zope API naming convention

[Jim Fulton]

Jim Fulton wrote:
> 
> There is a proposal to adopt a naming convention for all
> new Zope API methods at:
> 
>   http://dev.zope.org/Wikis/DevSite/Proposals/APINamingConvention
> 
> Comments are gratefully accepted at:
> 
>   http://dev.zope.org/Wikis/DevSite/Proposals/APINamingConventionDiscussion
> 
> Please make comments by Wednesday April 16.

Sorry, make that August 16.

Jim


--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] remote procedure calls to manage functions

[Jim Fulton]

Michel Pelletier wrote:
> 
> Chris Withers wrote:
> >
> > andrew wrote:
> > > Have been delving into ZPublisher.Client to make remote procedure calls
> > > under Zope 2.2.1.
> >
> > Did someone slip 2.2.1 without mentioning it?!
> >
> > I thought XML-RPC was now favoured over ZClient?
> 
> It's not really favored, both are quite useful.  xml-rpc is more for
> when you want two different system to interoperate.  ZClient is very
> zope specific and probably gives you a bit more functionality than
> xml-rpc (because it has the 'remote object' abstraction).

I don't think ZClient is really Zope specific. At least 
it's not supposed to be.  It does have a richer API, including
features like authentication and header support.

>  Also, ZClient
> is much faster i've found, probably due to the marshalling/unmarshalling
> necessary for xml-rpc.

That's interesting..Hm.  ZClient has to marshal.  I suspect that
xml-rpc wants some sort of optimization.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Weird Data.fs corruption...

[Jim Fulton]

Carl Robitaille wrote:
> 
> Hi everybody,
> 
> I looked around the Zope site and mailing lists, but I wasn't able to
> find an answer to my problem.

That's because your problem only existed for a few hours. Sigh.

> I'm using the CVS version of Zope2.

We try to keep this stable, but no guarantees.  Some changes were
checked into CVS on Monday night that caused the problem you observed.

> I did
> an update yesterday.

Unlucky you. :/

> Since then, there's been some commits.
 
(snip)

Copy/paste (export/import) were broken and led to database corruption.
I checked in the fix to this yesterday.

> 
> I then logged to Zope web site to get information on how to get my
> Data.fs fixed. I read the
> http://www.zope.org/Members/itamar/CorruptedZODB document and downloaded
> the tranalyzer.py python program. I then understood what the backtrace
> already told me. The first 4 bytes of any ZODB files should be something
> like "FS21". Instead, I have garbage in my Data.fs:

Right. The bug caused the file pointer to get incorrectly set at 0, so
transactions after the copy/paste errors were written at the beginning 
of the file.
 
> [zope@petechie var]$ python tranalyzer.py Data.fs | more
> Traceback (innermost last):
> error: Not a .fs file (at least, not a kind I know about)
> magic =
> 6TÆ
> 
> Just for the record, Data.fs.in is read easily by tranalyzer.py.
> 
> How can this happen? Any idea?

I think I answered that.

> My only "guess" is that it has to do
> with the Data.fs file being used by ZServer over an NFS partition on
> Linux. This is only a guess, but I really can't see any other thing I
> could have done wrong.

It was a software bug, which has been fixed.
 
> This doesn't really bug me for the moment as I only had a couple of
> hours put in my site, but I would like to prevent it from happening
> again yes I do backups, but only once a day ;-)

It shouldn't happen again.
 
Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Request for Comment: Zope API naming convention

[Jim Fulton]

There is a proposal to adopt a naming convention for all
new Zope API methods at:

  http://dev.zope.org/Wikis/DevSite/Proposals/APINamingConvention

Comments are gratefully accepted at:

  http://dev.zope.org/Wikis/DevSite/Proposals/APINamingConventionDiscussion

Please make comments by Wednesday April 16.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Permission

[Jim Fulton]

iew permission' in 'Root', then RoleA has the view permission in
> 'private' (acquisition, not?).

You cant set the "acquire permission settings" check box for roles.
You can only set it for permissions.

If the acquire permissions settings is set for the permission "View"
in folder "private" and "RoleA" has the "View" permission in the root, then
"RoleA" has the permission setting implicitly in folder "private".

> However the checkbox at 'view permission'
> for RoleA is not checked.

Because it is not set here.

> I it possible to put name next to this checkbox
> which tells that permission to 'view' for RoleA is acquired from 'root', or
> just a link which brings us directly to the security view of 'root'.

That would be useful, as long as people understand that the decoration
could change when someone changes a setting up above.
 
> I'm thinking that this would help us a lot when we want to know from where
> the permission is granted. Then we don't have to check all security views
> of the parent-containers.

Yes, I agree, that this would be useful. Maybe the security display
wouldn't give the source directly but maybe a symbol with
alt text that gives the source location and some sort of hyperlink
to the same.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] OFS.objectManager checking object Ids

[Jim Fulton]

"David C. Kankiewicz" wrote:
> 
> "Jay, Dylan" wrote:
> >
> > As I said, when the url is quoted there is no problem.
> >
> > http://azonia.auslabs.lucent.com/Auslabs/AuslabsFAQ/Where%20is%20X%3f
> >
> > causes no problems. The above object is sitting happily inside Zope with a
> > question mark at the end. I had to let the bad_id check let it pass but is
> > there any reason why it shouldn't?
> 
> bad_id=ts_regex.compile('[^a-zA-Z0-9-_~\,\. ]').search #TS
> 
> The .search was .match
> Changing it back restores the behavior, who changed it and why is the
> question?

I don't know who changed it, but the why is obvious.
'match' was clearly a bug, since it only found an 
invalid character if it was at the beginning of an id.

I think that it's a bad idea to allow '?'s in ids
and am sorry if it was allowed. In general, I don't
like to see characters in ids that need to be quoted.
I'm not happy that ' ' was added, although 
I understand why.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] remote procedure calls to manage functions

[Jim Fulton]

Itamar Shtull-Trauring wrote:
> 
> Jim Fulton wrote:
> 
> > > It's not really favored, both are quite useful.  xml-rpc is more for
> > > when you want two different system to interoperate.  ZClient is very
> > > zope specific and probably gives you a bit more functionality than
> > > xml-rpc (because it has the 'remote object' abstraction).
> >
> > I don't think ZClient is really Zope specific. At least
> > it's not supposed to be.  It does have a richer API, including
> > features like authentication and header support.
> 
> The problem I have with ZClient is that the result of method calls is always
> a string - while with XML-RPC I can get lists/dictionaries/integers, if
> that's what the method returned.

I didn't claim that it was better for RPC, but that it simply wasn't
Zope specific and has some features that XML-RPC lacks. Frankly, I'd
very much like to see the missing features in CML-RPC or SOAP.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] remote procedure calls to manage functions

[Jim Fulton]

Michel Pelletier wrote:
> 
> Jim Fulton wrote:
> >
> > Michel Pelletier wrote:
> > >
(snip)
> > >  Also, ZClient
> > > is much faster i've found, probably due to the marshalling/unmarshalling
> > > necessary for xml-rpc.
> >
> > That's interesting..Hm.  ZClient has to marshal.  I suspect that
> > xml-rpc wants some sort of optimization.
> 
> I made a test script to make about 300 xmlrpc calls to various manage_
> methods.  I had to kill it after about a half-hour cuz it was takin so
> long. 

This sounds pretty fishy. Are you saying that you made less than
300 calls in half an hour? This doesn't seem right at all.


> Here's the profiler output:
> 
>ncalls  tottime  percall  cumtime  percall filename:lineno(function)
> 702883/2151  108.9800.000  288.5800.134
> DT_InSV.py:379(__getitem__)
>  7704   79.0500.010  243.9000.032 xmllib.py:201(goahead)
>  3852   42.3000.011   43.1600.011
> FileStorage.py:706(_finish)

Hm. This says that the FileStorage _finish method was called
3852 times.  This says that 3852 transactions were committed.
Since transactions only get committed on write requests, then
this suggests 3852 write requests. There should have been 
way more than 3852 requests to have this many write requests.


> 158831/158827   38.0100.000   70.1400.000 DT_Var.py:258(render)
>732178   37.5800.000   49.4500.000 re.py:112(match)
> 495239/51912   36.6900.000  307.2000.006
> HTTPRequest.py:741(__getitem__)
>586827   29.0400.000   29.0400.000 re.py:335(group)
>499291   22.9000.000   30.5300.000 re.py:95(search)
>875347   19.5000.000   19.5000.000 re.py:290(__init__)
> 45981   14.0900.000   39.0000.001
> xmllib.py:539(parse_starttag)
>748592   10.6800.000   10.6800.000 exceptions.py:65(__init__)
>5419249.4200.0009.4200.000 re.py:297(start)
>4562739.1900.0009.1900.000 re.py:306(end)
>1078318.5200.0008.5200.000 urllib.py:898(quote)
> 127647/107577.4000.000  300.0800.028 :0(?)
> 491137.1600.0008.2400.000
> User.py:140(getRolesInContext)
> 127647/43026.3500.000  303.9100.071 DT_Util.py:325(eval)
> 459815.3600.000   19.8700.000
> xmllib.py:615(parse_endtag)
>1112585.2400.0007.0300.000
> xmllib.py:699(handle_charref)
>  38565.1000.001   17.0200.004
> BaseRequest.py:224(traverse)
>2845154.5100.0004.5100.000 xmlrpclib.py:387(data)
> 
> I think you're right about optimizing, why is it spending most of it's
> time in DT_InSV?

First, it's not spending most of it's time in DT_InSV __getitem__, although
it's spending more time in that method than in any other single method.
The reason it spends so much time in that method is that the method 
gets called 700 thousand times. That is, it's used alot, because alot
of in-tag special variables are used in your test. This is also a
pretty non-trivial method, so it's reasonable that it might take
a while.

All of this seems to me to be somewhat beside the point.

The original comment was regarding ZClient vs XML-RPC.
It would be more interesting to:

 - Compare ZClient and XML-RPC for some realistic tests, like yours,

 - Compare them for simple Python methods that don't do much but
   return some data, something like:

   def test(x,y,z):
  return 'hello world' * 100

 - If there is a big difference, use the profiler to try to figure out
   why XML-RPC is taking so long.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Weird Data.fs corruption...zope-dev@zope.org

[Jim Fulton]

Dieter Maurer wrote:
> 
> Martijn Pieters writes:
>  > Ai, NFS! There is a known no-no about using NFS for Data.fs storage, it leads
>  > to data corruptions. Zope and NFS do not mix.
> Huch, why that?
> 
> I use Data.fs (occasionally) over NFS and did not yet have had
> problems.
> And appending to a file should work over NFS.
> Locking may be a bit more problematic, but this is not an
> issue provided I ensure that only on Zope process runs at a
> time.

Locking was the main reason why I recommended against using
NFS. This is much less of an issue with ZServer than it was with
the old PCGI publisher, which tried to guess when a site was 
down and automatically restart it when it was.

I also generally don't trust NFS with something as important
as data Data.fs file.

Note that the subject corruption error was due to a software
bug that appeared briefly in the Zope CVS.

Jim


--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Permission

[Jim Fulton]

Tom Deprez wrote:
> 
> Hi,
> 
> I have this q'n already a long time in my head and I still don't know the
> answer to it. It has to do with the security-view. I hope that someone can
> shed a light on my brains.
> 
> Why is their an Acquire permission (referred to as acq_perm in following
> text) checkbox column?

It allows you to honor permission settings made in containing objects.

> In order to change a permission for a certain role, we have to check or
> uncheck this permission at the roles permission checkbox.

I don't understand this.  You can grant permissions to 
a role locally, regardless of whether you acquire permission
settings.  If you don't acquire, then the role only gets the permissions
you set, otherwise, it *may* get permissions from above.

> However, for this
> to work we have to uncheck the acq_perm column (because if this one is
> checked, the permission is taken from parent-folder-objects anyway, the
> value of the role checkbox doesn't counts at that moment).

You don't have to ncheck the acq_perm column if you simply want to
grant a permission to a role. You only need to uncheck the
acq_perm column if you want to prevent containing objects from
granting a permission to other roles.

> As a drawback,
> this means that all roles have to be checked/unchecked, because they don't
> acquire the permission anymore from one of its parents-folder-objects.

So don't uncheck the acq_perm column.
 
> Now, why couldn't this be implemented like the following? Isn't this easier
> to grasp?
> 
> The Acq_perm column doesn't exists. Assume that acquisition of a permission
> is always Active. The checkboxes of the role show their actual value
> (according to the acquisition). Thus if permissionA is checked in the
> parent-object, permissionA is also checked. If you want, for a certain role
> that this permission is not given, you uncheck permissionA. The subobject
> will show an unchecked permissionA box (because it acquires from its
> parent). In order to give the subobject again the permission, we only have
> to check permissionA.

The problem with this is that it doesn't recognize changes made to containers
later.
 
> A) In the first approach we've to uncheck the acq_perm checkbox and have to
> set the checkbox of every role according to how we want the permission to
> be handled by that role.

You don't need to uncheck the acq_perm checkbox unless you want to
prevent containers from granting permissions.

> B) In the second approach we've to uncheck the permission checkbox of the
> role(s). Other roles are not effected.
> 
> Am I missing something here? Is my explenation somewhere wrong? Why does
> Zope uses the first approach? Which, sounds for me, a little bit more
> complicated.

By acquiring permission settings you are allowing containers to 
grant a permission to a role today or sometime in the future.
This allows someone to control permissions in a centralized fashion.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] inheritedAttribute

[Jim Fulton]

Steve Alexander wrote:
> 
> In the latest Zope source, I've noticed calls to inheritedAttribute in
> the Python code.
> 
> I've found the C source code in ExtensionClass.c, but it doesn't help me
> understand when I should use it from Python, and why.
> 
> *reads more source*
> 
> Ah! About 70 lines further on in ExtensionClass.c:
> 
>"inheritedAttribute(class,name) -- Get an inherited attribute\n\n"
>"Get an attribute that would be inherited if the given (extension)\n"
>"class did not define it.  This method is used when overriding\n"
>"inherited methods.  It provides 2 advantages over accessing\n"
>"\n"
>"attributes directly through a superclass:\n"
>"\n"
>"1. The superclass need not be known,\n"
>"\n"
>"2. The superclass may be a Python class.  Without this method, it
> would\n"
>"   be possible to override methods inherited from python classes
> because\n"
>"   unbound methods gotten from Python classes cannot be called with
> \n"
>"   extension class instances.  \n"
> 
> Is there some documentation of this on zope.org or dev.zope.org that
> I've missed?

Probably not.  This is documented in the ExtensionClass
documentation:

  http://www.digicool.com/releases/ExtensionClass/

which is also included in the release at:

  lib/Components/ExtensionClass/ExtensionClass.stx.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]
Technical Director   (888) 344-4332  Python Powered!
Digital Creationshttp://www.digicool.com http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] ANNOUNCE: OracleStorage 1.0.0 beta 1

[Jim Fulton]

The initial beta release of an Oracle-based storage for ZODB
is available at: 

  http://www.zope.org/Products/OracleStorage

The Oracle storage provides full ZODB storage capabilities,
including undo and versions, by storing serialized objects
and meta data in Oracle tables.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: ANNOUNCE: OracleStorage 1.0.0 beta 1

[Jim Fulton]

Jim Fulton wrote:
> 
> The initial beta release of an Oracle-based storage for ZODB
> is available at:
> 
>   http://www.zope.org/Products/OracleStorage
> 
> The Oracle storage provides full ZODB storage capabilities,
> including undo and versions, by storing serialized objects
> and meta data in Oracle tables.

Chris McDonough pointed out that I had left some specific
table names in some SQL statements rather than using 
"prefix" specific names. This would lead to pack failures
(unless you chose the prefix "jim_" foy your Oracle objects. :)

Thanks Chris.

I've fixed this and made beta 2 available at the location above.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] not being set under some circumstances

[Jim Fulton]

Steve Alexander wrote:
> 
> Zope 2.2.1
> 
> Zope is not setting the  tag in the html header under some
> circumstances.
> 
> To get a reproducable example of this, create a folder "test" under the
> zope root object.
> 
> Inside "test" create a new dtml document called "body_html". Leave the
> content of body_html as the default.
> 
> Now, view the page, and view its source. You'll see no  element.
> 
> Is this intentional?

Yes. This should be a FAQ.

> Does Zope now not bother with the body element if
> it doesn't think it is needed?

Yes.

> This would seem to be the rationale
> behind the request attribute "request._hacked_path".
> 
> Comments?

This is needed because sometimes Zope does hack the path,
for example when index_html or :method form types are used.
The base href is needed in these cases to make sure that
relative URLs are treated correctly.

Jim


--
Jim Fulton   mailto:[EMAIL PROTECTED]
Technical Director   (888) 344-4332  Python Powered!
Digital Creationshttp://www.digicool.com http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] REPOST: Cut down on the Available Objects list.

[Jim Fulton]

Erik Enge wrote:
> 
> Hi, all - someone suggested to me that I should try the zope-dev list
> instead of the zope list, here goes.
> 
> When I create ZClasses I can make the Available Objects shrink to just
> a couple of available objects via subclasses and so forth.  How would
> I do this with a Python Class in a Python Zope Product?  (Let's say an
> OFS.Folder.Folder class, which could normally contain all available
> objects.)

You would override the method all_meta_types. See
this method in lib/python/OFS/ObjectManager.py.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]
Technical Director   (888) 344-4332  Python Powered!
Digital Creationshttp://www.digicool.com http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] ZODB optimization

[Jim Fulton]

[EMAIL PROTECTED] wrote:
> 
> I'm wondering, is the ZODB easily 'fragmented' or .. un-optimized ?
> If so, are there any tools/functions that will 'unfragment' / optimize
> it?

This really depends on the undelying storage used.  

> Does the 'pack database' in the Administration menu do this? 

Packing does two things:

  - It removed records for objects that are no longer referenced
(as of the pack time), or, IOW, it doesm garbage collection.

  - For storages that support undo (and time travel), it also
removes records that were not current as of the pack time.

Both of these, especially the latter are a little bit similar to 
defragmentation or optimization. In addition, the particular storage
implementation may have other issues/tools. For example, if the
Oracle storage is used, them whatever fragmentation or optimization
issues or adminstration tools provided by Oracle apply.

> If so,
> it could be a Good Thing (tm) to have separate optimize and packing
> (history deletion etc.) functions (buttons).

This doesn't really apply. 
 
> Is there any way to make an object not keep history on itself or on
> contained objects ?

Currently, the only way to do this is with a non-versioning storage, 
such as the Berkely DB storage.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] HiperDOM & xmlc

[Jim Fulton]

[EMAIL PROTECTED] wrote:
> 
(snip)
> If one is really interested in a separation of content and
> presentation (which I feel is good), then the "standard_html_*"
> were wrong in the first place. It forces the
> content author to prepare for presentation.

Here's a historical note on this decision. At the time that
standard_html_* were introducted, DTML was used soley for creating
templates to display content in Python objects. The expansion for
DTML is Document *Template* Markup Language. The mistake was 
not introducing components, but introducing the use of DTML
as content.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Shared Libraries and Persistent Objects

[Jim Fulton]

Monty Taylor wrote:
> 
> Hey guys,
> 
> I've got a question about how Zope deals with shared libraries. Say
> I have a 2Meg shared library that I've generated with SWIG. (I know it's
> big, but for sake of argument) Say then that I create a Python Class
> that imports this library and provides a front-end for each of its
> exported methods. Then, I instantiate this class 2000 times within a
> Zope instance. My question is, the library code is loaded into memory
> once, right?

Right. In fact, if you were running multiple Zope (or Python)
processes using the library, then there would only be one copy in 
memory shared among the multiple processes.

> That seems to be what should happen, but I want to make
> sure before I start selling people here on SWIGing a few utilities and
> bundling them into one of our Zope objects instead of doing fork/execs
> to call an executable. I'm not shifting the burden from loading time to
> storage size, am I?

No.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]
Technical Director   (888) 344-4332  Python Powered!
Digital Creationshttp://www.digicool.com http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] ZSQL Methods from Python?

[Jim Fulton]

Monty Taylor wrote:
> 
> Hey, is there any doco about using ZSQL Methods from Python Products?

Not that I'm aware of. You simply call them pretty much like
any other Python callable thing. The signature is:

  someMethod([mapping, **other_arguments])

  where:

 mapping is a mapping object that provides SQL method arguments, and
 can include other information. (The actual name of the positional
 argument may be "REQUEST" ;).  This argument is optional.

 You can also provide arguments as named keywords.  

 


> Also, does anyone know of any work done to extend ZSQL Methods to allow
> stored-procedure calls?

No, but I'd love to see someone tackle it. The semantics
of stored procedures varies so widely accross databases, that
I doubt that it would be easiliy generalizable. I think, at
least for starters, a form of stored procedure support for
Oracle would make alot of sense.

> I know I can do an Oracle Procedure call from
> DCOracle that returns a cursor, but I'd love to combine that with the
> caching/pluggable brains of ZSQL Methods. Is there anything out there on
> this or am I going to have to roll my own?

I think you'd have to roll your own. If you want to 
do something reusable that other people could use, I'd be
happy to provide whatever advice and support I can.

Note that one of the things I like about Oracle's stored procedures
is that they allow me to avoid screwing with cursors in the common case
that I'm getting one row of data.  I can just get the data I need through 
a straight function call.  The DCOracleStorage uses stored procedures
almost exclusively.

Jim


--
Jim Fulton   mailto:[EMAIL PROTECTED]
Technical Director   (888) 344-4332  Python Powered!
Digital Creationshttp://www.digicool.com http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] ZSQL Methods from Python?

[Jim Fulton]

Monty Taylor wrote:
> 
> Jim Fulton wrote:
> 
(snip)
> > Note that one of the things I like about Oracle's stored procedures
> > is that they allow me to avoid screwing with cursors in the common case
> > that I'm getting one row of data.  I can just get the data I need through
> > a straight function call.  The DCOracleStorage uses stored procedures
> > almost exclusively.
> >
> >
> 
> I've been looking through that code (we've started using DCOracleStorage on
> the backend.) For some reason, though, I can't get the stored procedure stuff
> to work like you do. Check out the following:
> 
> Oracle8i Enterprise Edition Release 8.1.6.0.0 - Production
> With the Partitioning option
> JServer Release 8.1.6.0.0 - Production
> 
> SQL> desc pgm140_api;
> PROCEDURE SELECT_TARGETS
>  Argument Name  TypeIn/Out Default?
>  -- --- -- 
>  PP_NAMEVARCHAR2IN
>  PP_RESULT  REF CURSOR  IN/OUT
> RECORD  IN/OUT
>  AMSM_IDNUMBER(12)  IN/OUT
>  NAME   VARCHAR2(80)IN/OUT
> PROCEDURE SELECT_TECHNOLOGIES
>  Argument Name  TypeIn/Out Default?
>  -- --- -- 
>  PP_CATEGORYVARCHAR2IN
>  PP_RESULT  REF CURSOR  IN/OUT
> RECORD  IN/OUT
>  ET_ID  NUMBER(12)  IN/OUT
>  NAME   VARCHAR2(80)IN/OUT
> 
> *
> Then from python, with the same connection string, I do:
> Python 1.5.2 (#1, Feb 14 2000, 18:27:27)  [GCC 2.95.1 19990816 (release)] on
> sunos5
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>> import DCOracle.oci_
> >>> conn=DCOracle.Connect('')
> >>> pgm140_api=getattr(conn.procedures,'pgm140_api')
> >>> sql_select_targets=getattr(pgm140_api, 'select_targets')
> Traceback (innermost last):
>   File "", line 1, in ?
>   File "/apps/zope/lib/python/Products/ZOracleDA/DCOracle/ociProc.py", line
> 324, in __getattr__
> oci.error: no usable procedure named pgm140_api.select_targets
> 
> I've tried using all caps on one or both as well to no avail. Any thoughts?

I suspect that the problem is the RECORD argument type. I don't
remember off-hand what this is, but I'm pretty sure, the DCOracle
procedure code doesn's handle this type. We (you;) would need to
either work around this or fix it.  Unless the fix is easy though,
we might want to wait for an OCI8-based Oracle interface that
someone here is rumored to be working on. :) (Actually, I *know*
that someone's working on an OCI 8 interface, but I'm not sure
what the status or priority is.)

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]
Technical Director   (888) 344-4332  Python Powered!
Digital Creationshttp://www.digicool.com http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] ZSQL Methods from Python?

[Jim Fulton]

"Phillip J. Eby" wrote:
> 
> At 08:43 AM 9/22/00 -0400, Jim Fulton wrote:
> >> Also, does anyone know of any work done to extend ZSQL Methods to allow
> >> stored-procedure calls?
> >
> >No, but I'd love to see someone tackle it. The semantics
> >of stored procedures varies so widely accross databases, that
> >I doubt that it would be easiliy generalizable. I think, at
> >least for starters, a form of stored procedure support for
> >Oracle would make alot of sense.
> >
> 
> Ty and I have put together a Stored Procedure method for Sybase; it
> requires a minor patch to ZSybaseDA, however, to allow for the status code
> return.  I'm not sure how useful it would be to anyone else, though, since
> all we ever do with stored procedures is process the return value, and
> ignore any actual SELECT results.

I think it would be useful. Can you get multiple return values?

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] ZSQL Methods from Python?

[Jim Fulton]

"Phillip J. Eby" wrote:
> 
> At 12:49 PM 9/22/00 -0400, Jim Fulton wrote:
> >"Phillip J. Eby" wrote:
> >>
> >> Ty and I have put together a Stored Procedure method for Sybase; it
> >> requires a minor patch to ZSybaseDA, however, to allow for the status code
> >> return.  I'm not sure how useful it would be to anyone else, though, since
> >> all we ever do with stored procedures is process the return value, and
> >> ignore any actual SELECT results.
> >
> >I think it would be useful. Can you get multiple return values?
> >
> 
> By "return value", I mean the "return status" code which is an integer
> returned by the Sybase "RETURN" statement.  Sybase also can generate its
> own, negative-numbered return statuses, which indicate a Sybase error.
> These error codes can include ones that mean the current transaction has
> already been aborted.
> 
> Apart from the "return value", a Sybase stored procedure can execute
> SELECT's which result in rows being returned.  We have never used this
> capability, so I'm not sure what would need to be done for it to be
> practical.  We ordinarily use stored procedures only to perform inserts,
> updates, deletes, and other things that only need a integer result if
> anything.  (Eg: get next counter value to assign an object identifier.)

Ah, too bad. Can Sybase *only* return values through selected rows?

Oracle's stored procedures are much more like traditional procedures
and therefore extremely useful even when no results are returned.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Do I really understanding caching?

[Jim Fulton]

Andy McKay wrote:
> 
> We have been looking at caching in Zope as a way of tweaking performance.
> Heres an example of what I think happens:
> 
> - Supposing I have a 1,000 object catalog. If one person changes an catalog
> aware object, that instance of the catalog will be pulled out of the ZODB
> and changed. It will then be written to the ZODB. That last copy will stay
> in the cache for as long as the "Cache Parameters" are set to allow.
> 
> - If somebody changes another catalog aware object, that will repeat the
> above process.
> 
> - However simply accessing the catalog (no changes) will pull the object
> from the cache.
> 
> - What would be really nice is if the object only got written to the cache
> when it is no longer used, that way every time the catalog changed it didnt
> write to ZODB instead it changed the cached version. Of course that does
> bring up the recovering from disaster problem.
> 
> So do I understand it correctly?

Not exactly.  Have you seen the ZODB paper?

  http://www.zope.org/Members/jim/Info/IPC8/ZODB3

You can think of the normal ZODB cache as being a tool that
manages the objects in memory.  Whenever an object has been
accessed, it is in the cache. The cache, together with the
database are responsible for moving objects, and their state
in and out of memory. An object can be in serveral in memory 
states. Two that are of interest, from a caching perspective, 
are the "ghost" state and the up-to-date state.  The up-to-date
state is a state in which the object and it's state are loaded.
In the ghost state, the object is in memory, but it has now state.

The cache manager tries to release an object's state and 
remove the object from memory when it is not used.  An object's
state is released when it hasn't been accessed in a long time.
It can be removed when no other objects (not counting the
cache itself) refer to it.

Note that the cache parameters are only guides for an algorithm
that is somewhat adaptive and a bit more complicated than
the parameters suggest. For example, the target cache size is a 
target, not a limit. The cache sizes are usually mich higher than
the target.

Another thing to be aware of is that large objects are generally 
designed to spread their state over many subobjects so that
they can make effective use of the cache.  A catalog may have hundreds
of sub-objects, only a small fraction of which are kept in memory
at a time.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]
Technical Director   (888) 344-4332  Python Powered!
Digital Creationshttp://www.digicool.com http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Testing Zope applications (was Re: [Zope-ZEO] Advice)

[Jim Fulton]

Note that this conversation hasn't had anything to do
with ZEO for some time, so I'm moving it over to zope-dev.

Toby Dickenson wrote:
> 
(snip)
> >I think it is really much easier to use ZPublisher/Test
> >(which is also available as Zope.debug:
> >
> >   import Zope
> >   Zope.debug(url)
> >
> >This provides a much thinner and more easily
> >controlled environment.
> 
> ... but not a complete replication of the deployed environment. For
> example, you cant test RESPONSE.write.

This sounds like a bug that needs to be fixed, not a limitation in the
approach. Would you mind submitting a collector item on it?

> From a philosophical point of view, using Zope.debug is wrong because
> the whole purpose is an integration test of the interactions of your
> pre-tested components with the Zope environment (depending on the
> complexity of your glue, you may also think of it as a unit test of
> that glue).

But Zope.debug uses the Zope environment.  Unless you consider ZServer
a critical part of the Zope environment.  It is extremely
rare for the difference between something like ZServer and
ZPublisher.Test to have any noticable impact on the application
behavior.
 
(snip)
> >> The key to this is (as always) ensuring that your design process
> >> considers the testability of the glue.
> >
> >What do you mean by glue?
> 
> The layer which allows Zope-ignorant code to be used as a Zope
> product. ZCatalog.py is an example of glue for Catalog.py.

I think that ZPublisher.Test provides a pretty good test harness
for anything below the publisher level, such as ZCatalog.

(snip)
> >How does this make anything untestable?  You certainly can decide
> >correct and incorrect behavior. I think there is much more possibility
> >of problems due to UI changes or dynamism that make analysis of
> >test results difficult.
> 
> I suspect our difference in opinion is the scope of what we are
> testing. Is it a bug if a product can be broken by a Manager using the
> management interface of an instance of a different product?
> 
> Should this 'bug' be tested for?
> 
> If so, then a test plan must verify that every dtml-namespace and
> every acquisition name lookup can never be subverted by an
> ObjectManager or PropertyManager instance, no matter how the instances
> are arranged in containers.
> 
> The alternative is that the Manager take responsibility for not
> creating any objects with a conflicting name - but who decides which
> names are disallowed. How do you test that this list is complete?

First, as an aside, you can't prove that non-trivial software 
is correct via testing. :)

Second, part of a test is deciding what the interface to the tested
software is. One way to limit the scope of testing is to limit the
promises made, for example to leave the behavior undefined under
some circumstances.
 
> I dont see a way to test this constraint, and it has proven impossible
> to avoid the problems using design rules. I recently checked some of
> our recent products using strategically placed debugging __getattr__
> hooks - with initially horrifying results.

It sounds like this is a problem that needs to be addressed.
It's not really a testing problem, but an architecture problem

I'm not sure exactly what problem you are refering to. It sound's
like an issue of depending on a specific acquired name and having
the name overridden with something bogus. Is that it?

There have been a couple of recent developments that I think could
help this:

  1. It's much easier than it used to be to name objects using 
 physical paths and converting the physical paths to objects.
 This would allow you to refer to a particular object using an 
 absolute reference.

  2. There will be a new interface in Zope 2.3 that will allow
 you to prevent a name from being used lower in a containement
 hierarchy. This change is documented at: 
 http://dev.zope.org/Wikis/DevSite/Proposals/ReplaceableProperty.
 The work has already been done and checked into CVS. I've asked Shane,
 the author, to update the interfaces wiki to capture this change.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] aq_inner: don't call it!

[Jim Fulton]

Chris Withers wrote:
> 
> After more playing I've discovered that calling the resutl of aq_inner()
> appears to be a very bad idea ;-)
> 
> The line:
> print self.aq_inner()
> 
> in the __call__ method of a python product caused the python process to
> dump an 'unknown software exception'.
> 
> Should that be so?

No. Sound's like a bug. Would you mind submitting a collector entry?

> Also, I thought aq_inner gave the contaiment context (presumably as a
> list?) but it just returns a single object.

It returns the innermost wrapped objects. IOW, it stips
off all levels of wrapping except one.


> So, how can I get the containment context of an object as a list?

See:

http://www.zope.org/Members/michel/Projects/Interfaces/AcquisitionModuleInterface

In particular:

  Acquisition.aq_chain(ob,1)

More acquisition pointers can be found at:

  http://www.zope.org/Members/michel/Projects/Interfaces/Acquisition

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]
Technical Director   (888) 344-4332  Python Powered!
Digital Creationshttp://www.digicool.com http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Testing Zope applications

[Jim Fulton]

Toby Dickenson wrote:
> 
(snip)
> >I'm not sure exactly what problem you are refering to. It sound's
> >like an issue of depending on a specific acquired name and having
> >the name overridden with something bogus. Is that it?
> 
> There are two related issues that conspire to make the problem hard:
> 
> A. What you descibe above, that looks like it will be fixed in part
>by NO_SUBOBJECTS_OVERRIDE (which looks great). The outstanding
>issue is what happens when a new version of a product wants to
>add a new NO_SUBOBJECTS_OVERRIDE name (when objects of that
>name may already exist in old subobjects).

I think that there should be some discussion of this design
pattern. Specifically, I'm not sure I like the idea of an application
that depends on fixed names in a hierarchy. In fact, I know I
don't. :)  I think it would be a good to have a discussion of
this pattern. In particular, I'd like to see:

  - Problems it solves

  - Alternative approaches

*I* would like to see this discussion happen in a Wiki,
but I won't insist. :)  FWIW, it will be much more likely
for me to make comments in a wiki, especially if someone
sends me the wiki link when they are ready for comments.
 
>In many cases these to-be-protected names are part of a
>product's API.
>http://dev.zope.org/Wikis/DevSite/Proposals/APINamingConvention
>suggests a naming convention for new API attributes. I suggested
>in the discussion Wiki that it should not be possible to create
>subobjects with an id of the form reserved for API attributes -
>but it seems to have got lost in the Wiki noise :-(

I thought, until a couple of days ago, that this proposal had been rejected.
I found out recently that it hadn't. I'll look at your comments when
I move the proposal forward.


On Wikis, while they need improvement (see
http://dev.zope.org/Wikis/DevSite/Proposals/WikiNG)
I personally find them far less noisy than mail
lists. Hopefully, we'll make the wiki's provide
the best of both worlds soon.



 
> >  2. There will be a new interface in Zope 2.3 that will allow
> > you to prevent a name from being used lower in a containement
> > hierarchy. This change is documented at:
> > http://dev.zope.org/Wikis/DevSite/Proposals/ReplaceableProperty.
> > The work has already been done and checked into CVS. I've asked Shane,
> > the author, to update the interfaces wiki to capture this change.
> 
> B. A common mistake in DTML authoring is to look up a name in the
>DTML namespace when the context expected to contain the name is
>not at the top of the namespace stack. The bug goes undetected
>if the context at the top of the stack does not contain that
>name.

But if the name isn't found in the inner (aka top ;) namespace,
then the search should proceed further in.
 
>I have found that this happens too frequently even for
>skilled authors, and is rarely detected in review. In summary:
>one namespace isnt enough.
> 
>I suspect that this is the problem percieved by Andrew Kuchling
>(who started this thread back on zope-zeo). Our solution
>to this (successful so far) has been to replace DTML in any use
>other than Document Templates.

I think that there is broad agreement that DTML should be
uses soley for templates and shouldn't contain complex logic.
 
>(For me, the greatest horror is the name 'DTML Method'. The name
>'method' implies a tight coupling to the instance that it is a
>method of, however it is possible for a DTML Method to execute in
>a context where all of it's instances attributes are masked by
>those of other objects. Urgh)

This is a great point. The intent was that DTML Methods would be bound
most tightly to the instances they were accessed in, but this is not what's
done.
 
Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]
Technical Director   (888) 344-4332  Python Powered!
Digital Creationshttp://www.digicool.com http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: more __call__ ...

[Jim Fulton]

Chris Withers wrote:
> 
> (moving back onto list, hopefully my head/wall contact will save others
> braincells ;-)
> 
> Jim Fulton wrote:
> > > On an (almost) related matter, are there any docs (other than the code
> > > ;-) for how __call__ is handles WRT to ?
> > > I had a look at DTMLMethod.py and PythonMethod.py and ended up very
> > > confused...
> >
> > Currently, DTML looks at something that it is going to render
> > and tries to decide if it's callable. If it is, it checks to
> > see if it has a true isDocTemp attribute. If an object is callable
> > and has a true isDocTemp, then it is called with None and the DTML
> > namespace. Otherwise, if an object is callable, it is just called.
> 
> So, if I give my product a class attribute of isDocTemp=1, what
> signature should I give my product's __call__ method so it picks up the
> DTML namespace?

  def __call__(self, ignored, md): ...

> > We are going to provide an alternate interface, so that if
> > an object has a special method, then this will be called instead.
> 
> okay... which proposal/project is this in?

I don't remember. Evan?
 
> > > I think the 'client', rather than 'self' is what I'm after, but how can
> > > I get it inside my product's __call__ method?
> 
> Hmmm... I don't suppose there's a nice definition of what the 'client'
> is, what's likely to be in the namespace stack and what self is likely
> to be in these circumstances lying around anywhere, is there? ;-)
  
client is just a namespace, defined by it's attributes. It's not really
in the namespace. Neither is self.
 
> > You can't really.  You should be able to infer it from the acquisition
> > context. For example, 'client' should be aq_parent.
> 
> def __call__(self):
> """ """
> print self.aq_parent.id
> 
> This is called by a  in standard_html_header which is
> also called with a dtml-var in, lets say, the DTML documnet: myDocument.
> 
> The above code prints the id of this product instance's container.

That's what I expect. 'client', in DocumentTemplate is usually
the object the template was invoked on.

> I was
> expecting it to print the id of myDocument. Is this me misunderstanding
> what the 'client' is,

Yes.

> or is something else going on?
> 
> In either case, what do I need to go to get hold of what would be
> PARENTS[0-1], if you see what I mean?

I don't see what you mean.  
 
> > Note that subtemplates
> > are not invoked on the objects they are found in, but in the namespace
> > they are called in. This is a bit weird and not really intensional. :(
> 
> I think I'm probably going to be relying on this, and I _think_ it makes
> sense to me. I presume subtemplate calls are things like doing  myObject>?

That's fine.

> In which case it seems sensible to keep playing in the
> namespace and just popping myObject on top as an instanceDict. I'm note
> quite sure how you think this area should work adn what difference it
> would make, could you elaborate? :-S

First, I have no intention of changing anything in a backward
incompatible way. For that matter, I don't have much interest in
changing DTML at this point, given:

http://dev.zope.org/Wikis/DevSite/Projects/XHTMLTemplate

I think that if you say:

  

that foo's namespace should take precedence over
the DTML namespace, but, in fact, foo's namespace
is ignored. I think that this should at least
happen optionally.  Otherwise, how could you think
of bar as a method?

 
> > > PS: In python product terms, whats' the difference between implementing
> > > __call__, __str__ or render?
> >
> > render?
> 
> Yeah, that's what I thought, but a coupla people have mentioned it.
> Maybe they're thinking of DTML tags?

Well, a number of people have suggested that there should
be a separate render (ie call as subtemlpate) interface. 
Maybe that's what what you meant.

Evan, help me out here. :)
 
> > __call__ implements the () operator (in C++ terms). It is the method
> > you implement to make your object callable.
> >
> > __str__ is the method you implement to control how your object is
> > converted to a human readable string.
> 
> So what's the difference in terms of doing  and
> http://site.com/myobject, if there is any?

The difference is clearer if you have 

  http://site.com/foo/bar

vs 

  

in http://site.com/zoo/splat.

In the former case, foo's namespace is 
at the top of the stack and, in the second case, 
it's not on the stack.

> PS: What I&

Re: [Zope-dev] more __call__ ...

[Jim Fulton]

Chris Withers wrote:
> 
> Jim Fulton wrote:
> > > So, if I give my product a class attribute of isDocTemp=1, what
> > > signature should I give my product's __call__ method so it picks up the
> > > DTML namespace?
> >
> >   def __call__(self, ignored, md): ...
> 
> Right, now if I call other DTML methods from my __call__ method, can I
> just call them with:
>
> self.nav_header(self, ignored, md)?

No: self.nav_header(None, md)
 
> In any case, what is self in __call__(self, ...) and what should it be
> in nav_header(self, ...)?

Python method signatures always begin with an argument that is the instance
to which they are bound.
 
(snip)
> > > In either case, what do I need to go to get hold of what would be
> > > PARENTS[0-1], if you see what I mean?
> >
> > I don't see what you mean.
> 
> http://a.site/folder/object/myobject
> 
> myobject contains 
> standard_html_header contains 

Is myobject a DTMLMethod? Some kind of container?
I don't see how it can contain a var tag if it's not
DTML.  If it is DTML, I don't see where mynavigator
fits in.
 
> __call__ is a method of the Navigator product, of which mynavigator is
> an instance.
> 
> Anyway, in that __call__, self.REQUEST.PARENTS =
> [zope-app,folder,object] (roughly)
> 
> By PARENTS[0-1], I meant the myobject object, itself.

Uh OK. 
 
> Hurm, I'm finding this hard to explain clearly, let me know if I'm
> getting closer :-S

OK, you've snipped the orignal question, so I'm off the hook. :)
(I don't really know if you want me to comment on anything
else here.)
 
> > I think that if you say:
> >
> >   
> >
> > that foo's namespace should take precedence over
> > the DTML namespace,
> 
> Okay, I see what you mean now :-)
> 
> > > render?
> >
> > Well, a number of people have suggested that there should
> > be a separate render (ie call as subtemlpate) interface.
> > Maybe that's what what you meant.
> 
> Nope... DT_Var.py line 258:
> def render(self, md):
> ...is what I think the people who mentioned it to me were talking about.

Ok, well that's that's part of the DTML tag interface, which I assume
you aren't interested in.
 
(snip)
> > > If there's an easier way to get this from within the __call__ method of
> > > a python product, someone PLEASE tell me! ;-)
> >
> > aq_chain(x, 1)[-1:] should give this to you, I believe, or
> > be very close. :)
> 
> Won't that just return x? (hmm... does aq_chain return
> [child,...,parent] or [parent,...,child]?)

[child ... parent]
 
> I think the hard part fo the problem I'm trying to solve is finding out
> what x is from within the __call__ method.
> 
> Any help muchly appreciated, sorry this is takign such a long time to
> sort out in my head... ;-)

The Zope debugger is your friend. It's a good way to 
find out some details like this.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]
Technical Director   (888) 344-4332  Python Powered!
Digital Creationshttp://www.digicool.com http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] CORBA-ZODB: Is replacing global get_transaction() the only way...

[Jim Fulton]

"John D.Heintz" wrote:
> 
> Hi all,
> 
> I am about to embark on integrating ZODB with CORBA,

Woo hoo!

> and am in the deep
> thinking phase of the endeavor.  ;-)
> 
> What I want to do is _explicitly_ manage connections and transactions
> without being able to depend on what thread is running.  I know that
> this is _not_ the way that Zope works,

That depends on your point of view.  The Zope publisher (ORB) largely
automates transaction management for application programmers, so they
don't have to deal with identifying transaction boundaries. So,
from an application point of voew, these details are hidden.

>From a system point of view, Zope does manage per-thread transaction data.
This is something a CORBA interface could provide as well.

IMPORTANT NOTE: The ZODB-provided transaction manager automatically
manages per-thread transaction information. 

Were you aware of this? This sounds like what you need.

> but if I want to use standard
> CORBA I think I have only two choices:
> 1) Explicitly associate Connections with Transactions in my CORBA
> layer.  What I need to do is allow any thread to change a Persistent
> object from some Connection,

This would be nice. Basically, you want to open a ZODB database
connection and begin a transaction at the beginning of the request.
At the end of the request, you want to commit or abort the transaction, 
depending on whether an error occurred, and close the connection.

> and make sure that the right transaction is
> called for "register".

If I understand your requirements, then the standard transaction
machinery should do this for you.
 
> 2) Build a complete wrapping adapter layer that does thread
> synchronization to the actual Persistent objects and proper thread for
> the transaction.
> - Lots of overhead and redundant coding - tricks with ExtensionClass
> might make this adaptation simpler to code, but still it won't be easy.

I'm not sure what this is, but I think it probably isn't necessary.
 
> Obviously number one is my preferred choice.  In order to accomplish
> that, I see only two ways:
> 1) Modify ZODB to maintain a Connection to Transaction link, and modify
> cPersistence.c to use that link in the changed() function instead of
> relying on the standard get_transaction() thread index.

Is there a one-to-one relation between threads and connections
(or, more precisely, is it the case that there is never more than 
one simultaneous connection per thread)? If so, then get_transaction()
should do the right thing.
 
> 2) Replace the get_transaction() in globals to return the appropriate
> Transaction regardless of thread.

It currently returns a different transaction depending on the thread.

> Again, my preference is number one.  After going over the ZODB code, I
> _think_ that a Connection is always associated with one Transaction.  If
> this assumption is true, then it should be safe to make the change I'm
> proposing. If not, then I need to understand why so that I can rethink
> how to solve my problem. ;-)
> 
> I hope that I've made my problems and ideas for solutions clear, if not
> please let me know. 

I dunno. You'll probably be able to tell from my response. :)

> Also, I think that I should be able to make the
> changes to ZODB myself within the next few week, but only if there is
> the _possibility_ of folding them back into the main Zope codebase.

I sincerely hope that no ZODB changes are necessary.
 
> Thanks for any help,
> John D. Heintz
> 
> CORBA Threading description:
> CORBA defines the POA, which has two standard threading policies:
> ORB Controlled Model
> Single Threaded Model
> 
> The POA is basically a controller for requests to one or more
> distributed objects, with thread policy as one of its parameters.
> 
> The first threading policy means whatever the ORB gives you - single or
> multi, and you have to deal with all synchronizations.
> 
> The second I consider a mis-nomer because there can be many threads, but
> only one at a time will access objects for a given POA.  (This is the
> model that I perceive being used by default for ZODB object from a
> specific Connection.)

No. Multiple threads can be accessing the same logical object.
Each thread has it's own copy (and DB connection and transaction).

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, 

Re: [Zope-dev] RFClarification: Security on Product Attributes

[Jim Fulton]

Chris Withers wrote:
> 
> Hi,
> 
> If I have the following lines in a Python Product:
> 
> def __init__(self, id):
> """initialise a new instance of product"""
> self.id = id
> self.title = 'Title!'
> self.anInt = 0
> self.aString = 'testing'
> 
> Are these attributes protected by the security machinery?
> If so, how so?

Yes. There are two ways of protecting objects, depending on
whether they (can) have a __roles__ attribute. None of the
values above can have a __roles__ attribute, so they are covered 
by assertions made in their containers.

See http://www.zope.org/Members/michel/Projects/Interfaces/ZopeSecurityPolicy

Note that if you can't adequately control something that
can't have __roles__, you can provide an access function
(e.g. getAnInt), which you can control

> Can I read them? I think the answer is yes for anInt and no for aString.

Probably, if you can get at one, you can get at the other.

> Don't know the mechanics of title and id, I'm guessing they're going to
> be special cases whatever...

Nope, except that we may provide separate accessors (e.g. getId).
 
> Are they protectable by permissions? I do hope so although my experience
> is that, at best, it's not necessary, which is contrary towhat I thought
> the new security policy was.

The goal of the new security policy was to:

  - Centralize authorization policies

  - Begin the tightening of access to attributes/sub-objects
that can't have roles.
 
> Should they have to be protected by permissions? Probably...

If you need the sort of control that permissions provide, 
you should consider providing accessors that can play with 
permissions.
 
Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] RFClarification: Security on Product Attributes

[Jim Fulton]

"Phillip J. Eby" wrote:
> 
> At 12:27 PM 10/4/00 -0400, Brian Lloyd wrote:
> >
> >I've verified (any of my previous comments to the contrary) that
> >simple attributes (python types) do not really play in the
> >permissions machinery. The canonical way to expose such things
> >for now is to expose them through method calls (which can play
> >in the permissions scheme).
> >
> 
> IIRC, this stuff got broken by the switch to the new security machinery.
> ZopeSecurityPolicy doesn't check 'foo__roles__' on the parent object the
> way ZPublisher does/did.

It never did.  Before the switch to the new policy machinery, 
most attributes that don't have roles were unprotected.
Now, we at least have a way to make some assertions.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: [Zope-ZEO] How to get rid of the cPickle and asyncore dependencyhassleshassles

[Jim Fulton]

Toby Dickenson wrote:
> 
> [cc to zope-dev.]
> 
> > > Ive been using Zope only with Python 2.0 recently.
> >
> > That's great to hear. I'm frankly a bit surprised that
> > this works.
> 
> 2.0 is a suprisingly conservative upgrade from 1.5.2
> 
> > Did you have to make many changes to Zope?
> 
> All my critical patches are already merged into the Zope cvs, or the Python
> cvs ;-)
> 
> There are also some non-critical bug fixes related to Unicode awareness,
> bundled up with my other Unicode-In-Zope patches at
> http://www.zope.org/Members/htrd/wstring. These mostly relate to the fact
> that Zope does not defend against objects that raise exceptions when
> converted to a string using str() - and Unicode objects do this frequently.
> 
> (At the moment these extra patches are not 1.5.2 compatible... Ill work on
> this if DC feel the need for a transition period during which Zope supports
> both 1.5.2 and 2.0.)
> 
> > I heard that ExtensionClass didn't work with Python 2.0.
> 
> It looks like a little work is needed to support 2.0's garbage collection of
> cyclic trash - but thats not enabled by default. Apart from that, everything
> works very well. (In some ways, better than 1.5.2)

Wow! That's swesome.  I'll get back to you on some of this.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] More almost __call__ ;-)

[Jim Fulton]

Chris Withers wrote:
> 
> Toby Dickenson wrote:
> > <http://www.zope.org/Members/htrd/howto/FunctionTemplate>
> >
> > you would use
> >
> > def a_method(self,md):
> > do_stuff_with(md['param1'],md['param2'])
> > a_method = FunctionTemplate(a_method)
> 
> That looks like it'll do the trick... I wonder if there's any way you
> can role it up into a Product so that I don't need to have
> FunctionTemplate.py in each folder of a product that needs to use it?

Is it *really* the case that you often want to method to be called directly
via the Web *and* and from DTML?  This doesn't seem right to me.

In any case, if you *just* want to get your function to be
called from DTML (and your class is an extension class), you can give
it the isDocTemp attribute like so:

  class MyClass( include some extension class ...):

some_methodisDocTemp=1
def some_method(self, ignored, md):


Methods in extension classes can have (read-only) attributes
by defining an attribute whose name is the concatenation of the
method name and the attribute name.  Note that the method is
also callable from the web as usual. If you want it to be called
the same way as a template, you'd need to use:

  class MyClass( include some extension class ...):

some_methodisDocTemp=1
def some_method(trueself, self, REQUEST, RESPONSE=None):


In this example, 'trueself' is bound by Python and is the traditional
python self. The 'self' argument gets bound by the publisher
to the object the method was invoked on.  

Note that if you get called from the web, RESPONSE will
not be none. This is a handy way to decide if you
are called from the web or as a sub-template. Of course,
someone could pass a non-None RESPONSE argument directly.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] More almost __call__ ;-)

[Jim Fulton]

Chris Withers wrote:
> 
> Jim Fulton wrote:
> 
> > In any case, if you *just* want to get your function to be
> > called from DTML (and your class is an extension class), you can give
> > it the isDocTemp attribute like so:
> >
> >   class MyClass( include some extension class ...):
> >
> > some_methodisDocTemp=1
> > def some_method(self, ignored, md):
> 
> great :-) that did the trick, thanks...
> 
> I can't help but feel this woudl all be so much simpler if it was
> documented somewhere for idiots like me :-S

Well, it *is* documented somewhere. (In the ExtensionClass
documentation), but not somewhere where you were likely to find it.

Jim


--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] More almost __call__ ;-)

[Jim Fulton]

Shane Hathaway wrote:
> 
> Jim Fulton wrote:
> >
> > Chris Withers wrote:
> > >
> > > Toby Dickenson wrote:
> > > > <http://www.zope.org/Members/htrd/howto/FunctionTemplate>
> > > >
> > > > you would use
> > > >
> > > > def a_method(self,md):
> > > > do_stuff_with(md['param1'],md['param2'])
> > > > a_method = FunctionTemplate(a_method)
> > >
> > > That looks like it'll do the trick... I wonder if there's any way you
> > > can role it up into a Product so that I don't need to have
> > > FunctionTemplate.py in each folder of a product that needs to use it?
> >
> > Is it *really* the case that you often want to method to be called directly
> > via the Web *and* and from DTML?  This doesn't seem right to me.
> 
> As I understand it, that's not the issue.  Chris wants to receive the
> DTML namespace implicitly.
> 
> > In any case, if you *just* want to get your function to be
> > called from DTML (and your class is an extension class), you can give
> > it the isDocTemp attribute like so:
> >
> >   class MyClass( include some extension class ...):
> >
> > some_methodisDocTemp=1
> > def some_method(trueself, self, REQUEST, RESPONSE=None):
> > 
> 
> That's a good idea, I wish I'd thought of it. :-)
> 
> However, the new calling convention (__render_with_namespace__, or
> something like that) might break "isDocTemp" code.

I thought of that, but I don't think so. There are obviously 
enough people using the isDocTemp thing that we'll probably need to
keep supporting it.  The use of a separate render method won't be
convenient for iplementing simple methods. Not that that is necessarily
that critical.

> > Note that if you get called from the web, RESPONSE will
> > not be none. This is a handy way to decide if you
> > are called from the web or as a sub-template. Of course,
> > someone could pass a non-None RESPONSE argument directly.
> 
> This convention is helpful but not at all documented, so products (ZWiki
> is a good example) often break this convention.

Which convention? The passing of arguments with names meaningful to
the publisher is certainly well documented.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] More almost __call__ ;-)

[Jim Fulton]

Shane Hathaway wrote:
> 
> Jim Fulton wrote:
> >
> > Shane Hathaway wrote:
> > >
> > > Jim Fulton wrote:
> > > > Note that if you get called from the web, RESPONSE will
> > > > not be none. This is a handy way to decide if you
> > > > are called from the web or as a sub-template. Of course,
> > > > someone could pass a non-None RESPONSE argument directly.
> > >
> > > This convention is helpful but not at all documented, so products (ZWiki
> > > is a good example) often break this convention.
> >
> > Which convention? The passing of arguments with names meaningful to
> > the publisher is certainly well documented.
> 
> No, I was referring to the convention of RESPONSE being passed only if
> ZPublisher is calling the method.  It is not documented and often
> broken.

The fact that ZPublisher will pass it is documented.  Whether
or not something else calls it is up to the something else.
The method designer specified an interface that includes RESPONSE.
If it requires RESPONSE, by not specifying a default, then it 
requires the clients to oass it. If the clients don't, then they
are not satisfying the interface.  Are you saying that objects
that require that RESPONSE is passed are broken because they don't
sufficiently advertise the fact?

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: Hippo Feed and Acquisition

[Jim Fulton]

Chris Withers wrote:
> 
> Toby Dickenson wrote:
> > Thats fine until:
> >
> 
> 
> I liked that :-))
> 
> very accurately sums up some of the problems...
> 
> Although I'm not sure they're problems with the method binding this
> thread was about.
> 
> The problems you describe seem to be with Acquistion in general. Other
> than that, I don't have the answers :-(
> 
> Any ideas, anyone?

I think that the binding control will address many of the
problems that Toby raised. It boiled down to this:

  - Some bits of code are designed to work on specific objects
or kinds of objects. The binding control will allow authors
to make sure that these are applied to the expected
objects.

  - Some bits are designd to be reusable in any context. 
Authors will still be able to create these.  Note that
authors will be able to do this even when the bits are
used as "sub-templates".

Jim

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] RFClarification: Security on Product Attributes

[Jim Fulton]

Chris Withers wrote:
> 
> Okay, apologies in advance for picking up a thread that's been dorman
> for so long ;-)
> 
> Jim Fulton wrote:
> >
> > Chris Withers wrote:
> > >
> > > self.id = id
> > > self.title = 'Title!'
> > > self.anInt = 0
> > > self.aString = 'testing'
> > >
> > None of the
> > values above can have a __roles__ attribute, so they are covered
> > by assertions made in their containers.
> 
> That's what I thought
> 
> > Note that if you can't adequately control something that
> > can't have __roles__, you can provide an access function
> > (e.g. getAnInt), which you can control
> >
> > > Can I read them? I think the answer is yes for anInt and no for aString.
> >
> > Probably, if you can get at one, you can get at the other.
> 
> That's not my experience.

There's something very odd going on. The issue isn't
stringness or intness but role-less-ness. :)

> If you try and use strings, you get dialog
> boxes popping up. If you use ints, it works fine.

I've never seen this.
 
> So, the problem is how to protect ints when you don't want people to get
> at them... Adding accessors mean you have protected accessors bu there's
> nothing to stop you just going and using the freely available original
> attribute.

Yes there is. Access to attributes is controlled by the roles if their
container.  

You should also be able to create specific unprotected attribute assertions
using the mechanism described in:

  http://www.zope.org/Members/michel/Projects/Interfaces/ZopeSecurityPolicy

I'll admit that I haven't tried this. If you try it soon (like today) and
find it broken, we can fix it for Zope 2.2.3.
 
> Strings; fine, at least they're secure, and when they become proper
> objects in Python 2.0, the problem should go away?

Will Python 2.0 let you assign string attributes?

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Exceptions

[Jim Fulton]

I think that this would make a great dev.zope.org fish-bowl project.

I'd love to see someone come up with an exception model for
Zope:

  - arranging exceptions in a hierarchy.

  - defining standard APIs for exceptions.

To give an example of what I'd like to gain by having
exception APIs, I'd like there to be ab API for
finding out whether an exception is meant for a human
and getting an exception as HTML, so we stop embedding
markup in exception values to indicate errors meant 
for humans.

Any volunteers? :)

Jim

seb bacon wrote:
> 
> I was just building in some error handling into some UI code, and wanted to catch 
>errors relating to duplicate ids.  The problem is that just about every client-side 
>error raises the same kind of Exception, a 'Bad Request'.  A quick grep counted 41 
>different types of 'Bad Request' in my Zope source.
> 
> Wouldn't it be *much* nicer to have a hierarchy of exception types, so that 
>applications can deal with them at an arbitrarily granular level?  e.g.
> 
> ZopeException
> |
> `RequestException
>   |  |
>   |  `XMLException
>   |
>   `IdException
> |  |
> |  `ReservedWordException
> |
> `DuplicateException
> 
> Has this discussion taken place before?  Would it be useful?  Would its 
>effort:benfits ratio be too great to justify doing?
> 
> seb
> 
> ___
> Zope-Dev maillist  -  [EMAIL PROTECTED]
> http://lists.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope )

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] RFC: Python/Zope Interfaces

[Jim Fulton]

Lalo Martins wrote:
> 
> On Fri, Nov 24, 2000 at 08:11:48AM -0800, Michel Pelletier wrote:
> > Python Interface Proposal
> >
> >   I have been working on a proposal for enhancing the existing interface
> >   documentation in Zope.  The Wiki for this project can be found here:
> 
> As far as what's written on the proposal is concerned, I like it.
> 
> Technically, I have one objection to the interface
> documentation system:
> 
> Why must I create a new "dummy" Python file?

I don't think you do need to create a dummy Python
file. You *do* need to define interfaces, in whatever files
you choose. 

> Why can't the
> system extract the data from the Python source itself?

Because the (class) source is about implementation, not
interface.  A class often implements multiple interfacs, 
and multiple classes often implement the same interface.
For that reason, it makes sense to define interfaces and
classes independently.

> Duplicating work is never good, and there are even people who
> like literate programming :-)

There is no duplications. Interfaces and classes are two different
kinds of beasts.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] RFC: Python/Zope Interfaces

[Jim Fulton]

Michel Pelletier wrote:
> 
> Lalo Martins wrote:
> >
> > On Fri, Nov 24, 2000 at 08:11:48AM -0800, Michel Pelletier wrote:
> > > Python Interface Proposal
> > >
> > >   I have been working on a proposal for enhancing the existing interface
> > >   documentation in Zope.  The Wiki for this project can be found here:
> >
> > As far as what's written on the proposal is concerned, I like it.
> >
> > Technically, I have one objection to the interface
> > documentation system:
> >
> > Why must I create a new "dummy" Python file? Why can't the
> > system extract the data from the Python source itself?
> 
> You can, of course, because you can just pass the Interface costructor a
> class.

This is, in my strongly-held opinion, a bad idea.  Most classes
implement multiple interfaces, and trying to suck interfaces out of
a class definition will tend to generate sloppy interfacesl.

I added a comment on this to your Wiki page. :)

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] RFC: Python/Zope Interfaces

[Jim Fulton]

Dieter Maurer wrote:
> 
> Michel Pelletier writes:
>  > Also, defining the interface seperately keep the two things apart,
>  > impementation and interface, and doesn't allow you to sneak in a new
>  > method unless you also sneak it into the interface, thus making a
>  > stronger "contract" with the user.
> I am a bit astonished by this statement:
> 
>   I know the "design by contract" concept from Bertrand Meyer,
>   the Eiffel developper.
> 
>   In Eiffel, essential parts of the contract, among others
>   method prototype, pre- and post-conditions as well as invariants
>   are build directly into the language.
>   A documentation tool extracts these parts
>   from the source to generate the interface, for people
>   that are only interested in how to use the class/method.
> 
>   Programms can be executed in a way, that the various
>   (executable) contract parts can be checked at runtime.
>   *THIS* provides for quite a strong contract.
> 
> I cannot see, why the separation of interface and implementation
> should make the contract stronger.

The interface *is* the contract. If someone builds a house
for me, I don't want the house to *be* the contract. I want the
house to adhere to the contract.  Interface/contract and implementation
are two qualitatively different things.

> I do see, however, that it
> makes it more likely to be broken by the implementation.

I don't think it makes it more or less likely. Of course, 
if the "interface" is always derived from the implemenation
then the two will be consistent, but this is not 
terribly useful.

> It is a very good thing to have the specification very near
> to the implementation -- as a permanent guide to the
> implementor.

Firtunately, modern displays allow multiple side-by-side
windows. ;)

> It is even better, when big parts of the
> specification becomes part of the executable code
> (as is the case for Eiffel's pre- and post-conditions).
 
> If you want to prevent your implementors to change the
> interface specification, generate the interface for the
> implementation and compare against your master copy.
> 
> In my view, it is better to have a somewhat "weaker" contract
> that is met by the partners than a "stronger" contract that
> is violated.

We disagree then. A weak interface that is satisfied by definition
is nearly useless to me.

> Or, to say it differently, it is more essential
> the a system's documentation describes faithfully what is
> rather than what should be (but is not).

I disagree alot. I'd rather have clear documentation of intended 
behavior, rather than have the documentation change based on 
implementation decisions.

Note also that most classes implement multiple interfaces.
Generating interfaces from classes trends to yield bloated
non-cohesive interfaces.

Similarly, most interesting interfaces are implemented by multiple
classes, so nearness to implementation is not really meaningful.

Please note that these points were argued extensively on the Python
types-sig a couple of years ago.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] OracleStorage, and possibly others

[Jim Fulton]

Lalo Martins wrote:
> 
> Well, two betas of OracleStorage in one day, then a month and a
> half of silence. What's the status?

>From our perspective, it's what it was then.  We built it for
a customer who later decided they didn't want it. I'm
glad to hear you found it useful.

> What about the other Storage projects? BerkeleyStorage has been
> dead for an year, and I heard pretty nasty words about
> InterbaseStorage. What about someone who wanted to try to port
> OracleStorage to Postgres or some other RDBMS?

We're working on a new suite of Berkeley storages that we are
writing to the the latest Berkeley database APIs. As Chris
pointed out, these are based on a new Berkeley DB extension that 
Andrew Kuchling started and that Robin Dunn is finishing.

There are a number of interesting new features that will eventually
result from these storages:

  - A new pack/garbage collection approach.  The current storages
perform a mark-and-sweep garbage collection when packing. It turns
out that this really isn't very scalable.  I'm going two 
switch to using the same garbage-collection strategy that Python uses, 
which is reference counting supplimented by an incremental cyclic garbage
detection algorithm. I have a simple storage (no undo or versions) that
needs no packing of your data structures are free of cycles.

I'm hopeful that we can make packing automatic and incremental.

  - Undo-log generation will be much faster, at least for common
cases. Generation of undo logs (for selecting transactions to undo)
is becoming a significant performance problem with FileStorage.

  - ZEO verification, performed when a client with a persistent
cache starts up, will be much faster.

  - Policies to control whether multiple revisions are stored
or whether revisions are removed by packing on a object-by-object
or transaction-by-transaction basis.

For example, you could have a poll/votting product that didn't allow
undo of and didn't require storing multiple records for votes. This
would be cleaner than mounting a separate database with a simple
storage.

You could keep significant historical revisions for important objects, such
as Wiki pages, templates, scripts, etc., with a policy to decide
which revisions are significant.

  - The ability to set policies to implement quotas.

I expect that we'll work out a lot of these ideas for Berkeley storages and
then implement them in OrcaleStorage. Others should then be able to map the
implementation onto other storages.

I'll note, in passing, that it's much nicer working these ideas out
with the BerkelyDB API than dealing with PLSQL or Python SQL.

We are also working on ZEO storage replication. This may have a big
impact on the storage API, or lead to specialized APIs for replicated
storages.
 
> Please help stamp out Data.fs! :-)

I don't think Data.fs will go away.  I do expect it to be relagated to
initial evaluation and development projects. Use of Berkely DB in
transactional mode requires a significant andminstration commitment.
Log files need to be purged. Backup and recovery processes need to be in 
place. A similar cost is associated with using Oracle and many other databases,
I expect. People aren't going to want to deal with these issues when 
initially trying and learning Zope (or ZODB).

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] OracleStorage, and possibly others

[Jim Fulton]

Lalo Martins wrote:
> 
> On Wed, Nov 29, 2000 at 07:02:50AM -0500, Chris McDonough wrote:
> > > Of course it would, for the same reasons as OracleStorage (eg
> > > FileStorage/Data.fs is inefficient)
> >
> > Actually, it's the other way around.  OracleStorage is 30-to-50 times slower
> > than FileStorage on writes.  Reads are slow too but the slowness is somewhat
> > negated by caching.
> 
> Chris, that's only true for small databases. At about 100M of
> Data.fs, OracleStorage starts being faster. It depends on
> hardware too. We made some benchmarks on a major Brazilian
> portal, and well, it's currently running OracleStorage.
> 
> Anyway, I said "inefficient", not "slow".

I think you really mean "scalable".  I suspect that
FileStorage will be faster and use less disk that OracleStorage, 
or any other storage, at least when supporting undo.

To get even comparible speed from Oracle requires substantial
optimization on the Oracle side. Our customer was able to substantially
improve DCOracleStorage performance by doing things like putting
tables and indexes on different disk *controllers*. :)

OTOH, FileStorage has a huge memory hit for lage databases, because
it keeps an in memory index from oid to file position.  It currently
uses a Python dictionary for this. We could probably reduce the memory 
hit by about an order of magnitude by switching to a different data
structure, but DCOracleStorage or a Berkeley DB-based storage doesn't
need this index.

Initial tests with my latest Berkeley DB-based storage show it to be
about three times slower than FileStorage.  I suspect that if the logs
and data files were put on separate disks, as Sleepycat Software recommends, 
that the speed difference could be reduced quite a bit.

Note that, until recently, FileStorage couldn't be used with databases
over 2GB in size, due to a bug that was fixed at around the same time that 
we released the DCOracleStorage.  We have a customer who has a 10GB 
FileStorage. You need large-file support in Python for your OS to 
be able to go over 2GB.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Re: [Geeks] Interface Meta Data proposal

[Jim Fulton]

Chris McDonough wrote:
> 
> Is security really a part of an object's interface? 

IMO yes. I would go even farther and suggest that different
interfaces to an object represent different types of usage
and map very nicely to permissions. That is, there might
be a one-to-one mapping from interface to permission.  In this
model, you don't need to make assertions for components
of an interface (e.g. attributes and methods, which are just special
kinds of attributes), rather you simply say that the whole interface
is associated with a particular permission. 

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: [Geeks] Re: Interface Meta Data proposal

[Jim Fulton]

Guido van Rossum wrote:
> 
> > Is security really a part of an object's interface?  I thought this was more
> > of an implementation thing.
> 
> Good point.  Certainly in Unix I can have two things implementing the
> same interface (e.g. two named pipes) with different security
> settings (i.e. modes).
> 
> However some security stuff can be part of the interface too: e.g. the
> fact that a Foo object has Read, Write and Execute permissions is
> usefully specified as part of its interface.  Whether a particular Foo
> instance has a given permission for a certain user is up to the
> instance.

I think we're talking about a different level of security setting
here.  In unix, you have certain fixed permissions (e.g. read, write,
execute).  The OS maps these permissions onto certain low-level operations
for the few objects it knows about.  This is about mapping permissions
onto objects, rather than deciding what users or groups have 
what permissions on an object.

In Zope, the programmer defines abstract permissions and decides how to map
the abstract permissions to object operations. It is this mapping that
makes sense for interfaces.  

Like Unix, Zope has a separate mechanism for deciding what users/roles
have what permissions on objects. This should not be part of the 
object interface.  These settings are done by users/administrators.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] OracleStorage, and possibly others

[Jim Fulton]

"Phillip J. Eby" wrote:
> 
> At 08:10 AM 11/30/00 -0500, Jim Fulton wrote:
> >
> >I don't think Data.fs will go away.  I do expect it to be relagated to
> >initial evaluation and development projects. Use of Berkely DB in
> 
> >transactional mode requires a significant andminstration commitment.
> >Log files need to be purged. Backup and recovery processes need to be in
> >place.
> 
> FWIW, Ty and I talked about using a daemonic thread in BerkeleyStorage to
> run periodic checkpoints and purge logfiles.  AFAIK, backups of BerkeleyDB
> require only that you make a copy of all the associated files.  Recovery's
> a bit trickier, of course.  (We never got around to implementing it because
> our tests showed that BerkeleyDB didn't solve the performance issue we were
> trying to solve.  So we quit working on it at that point.)

I don't think the storage should get involved with this. There are alot
of knobs and possible policies that can be used. This is all well documented
and supported by Sleepycat Software. IMO, when someone uses a Berkeley DB storage
they are using Berkeley DB and should learn how to administer it. I don't
think it's a good idea to try to hide this from people.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] RFC: Python/Zope Interfaces - More ZCatalog ;-)

[Jim Fulton]

Chris Withers wrote:
> 
> Jim Fulton wrote:
> 
(snip)
> > Please note that these points were argued extensively on the Python
> > types-sig a couple of years ago.
> 
> What was the outcome?

They all agreed



to stop arguing with me. ;)

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] RFC: Python/Zope Interfaces

[Jim Fulton]

Dieter Maurer wrote:
> 
(snip)

> Let me stress my point of view. Maybe, we are not too far away
> from one another:
> 
>   * I like clear specifications (as you do)

Yup.
 
>   * The specification has more value, when its implementations
> adhere to it (you probably will agree).

Well, not quite. If you use documentation of an interface to
figure out how to use an existing implementation, then I agree
that, for that purpose, an interface that documents the 
implementation accurately is valuable.

OTOH, if the implementation is doing the wrong thing, an interface
that documents doing the right thing is more valuable than an interface
that documents the wrong thing. Given an interface that documents
the right behavior, all I have to do is fix the implementation, 
which is easy. If both the interface and the implementation are
wrong, then I not only have to fix the implementation *and* the interface, 
but, much worse, I have to fix all the client implementations that
were written to the wrong interface.
 
>   * A documentation about an existing system is more
> valuable, when it states what is rather than
> what should be (here we may disagree).

It depends on the goal of the person using the documentation.
 
> I usually start by studying the documentation.
> If it descibes what is, then I will recognize
> early what does not fit my requirements and
> can work around it.
> If it describes what should be, I will
> determine the deficiencies much later in my projects,
> modifications are much more difficult and expensive.

Or you could get (or help, as you often do, thanks) the author
to fix the broken implementation.
 
>   * Having the specification near the implementation
> helps the implementors to adhere to it,
> especially with long maintenance periods.
> (Here we may disagree).

Yes, we disagree.
 
> I believe in the value of well documented
> source, in literate programming.

I agree with the value of well documented interfaces 
*and* well documented implementation, source.  I see alot
of value for documenting the implementation in the source.

I see *some* value of seeing the interface when writing and
maintaining the source. I'm really happier seeing a separate
specification. Perhaps this is a matter of taste.  

I wouldn't relish duplicating interface documentation in all of the
implementations that implement it and trying to keep them
consistent.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Turning acquisition off selectively.

[Jim Fulton]

Chris Withers wrote:
> 
> [EMAIL PROTECTED] wrote:
> >
> > That is, with a simple method, and not disabling the Acqusition class,
> > something like self.aq_disabled('attribute') .
> 
> So kindof the inverse of using Aquisition.Explicit and using the
> aq_acquire method?
> 
> What you describe would be really cool...
> 
> The only workaround for now is to use calls to ac_acquire() and provide
> a filter function, but this doesn't help when other code that you have
> no control over accesses an attribute that you'd rather not have
> acquired :-(

I'm inclined to think that in some future version of Zope, we
should switch to making explicit acquisition the norm.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Turning acquisition off selectively.

[Jim Fulton]

Toby Dickenson wrote:
> 
> On Fri, 08 Dec 2000 10:57:48 -0500, Jim Fulton <[EMAIL PROTECTED]>
> wrote:
> 
> >I'm inclined to think that in some future version of Zope, we
> >should switch to making explicit acquisition the norm.
> 
> Jim, do you have any opinions on some of the other acquisition
> strategies that have been mentioned on the list?

I don't have time to read the list normally. :( 
(This is bad. I need to fix this and am trying to )
 
> In particular, there seems to be merit in an acquisition that searches
> only the containment heirarchy.

Of course. Zope already doesn this for security.  It does this
by brute force most of the time, although there's now more API
support for it. There's a 'containment' keyword flag argument
to the aq_acquire method, so, at least from Python, you can
force acquisition to search *only* the containment hierarchy.

I think that there's merit to providing more control over how
acquisition happens. This is consistent with making acquisition
more explicit. 

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] OracleStorage, and possibly others

[Jim Fulton]

Chris Withers wrote:
> 
> Jim Fulton wrote:
> >
> >   - Policies to control whether multiple revisions are stored
> > or whether revisions are removed by packing on a object-by-object
> > or transaction-by-transaction basis.
> 
> > You could keep significant historical revisions for important objects, such
> > as Wiki pages, templates, scripts, etc., with a policy to decide
> > which revisions are significant.
> >
> >   - The ability to set policies to implement quotas.
> 
> >
> > We are also working on ZEO storage replication. This may have a big
> > impact on the storage API, or lead to specialized APIs for replicated
> > storages.
> 
> very cool :-)
> 
> Any idea when these will be around to play with?

Uh, no.  We are actually using a very limited version
in production, but the full-featured version is waiting for
someone with bandwidth to finish it.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: Q: how to retrieve transactions from a Data.fs fragment?

[Jim Fulton]

Shawn Murphy wrote:
> 
> Yes
> 
> We experienced a nasty confluence of suprises:
>  - CanOfRaid is imperfect,
>  - reiserfsck is imperfect,
>  - our backups were imperfect
> [We already knew we were imperfect.]

:]

> The result is that all we have left of about 6 weeks of several peoples
> development effort is two Data.fs scraps recovered from a reiser fs by one
> of the reiser developers.  We also have an intact 2 month old Data.fs
> which we would like to update with whatever we can retrieve from these two
> scraps.
> 
> Fragment#1 is 187MB where the first 147MB is nothing but nulls and the
> remainder is stuff that satisfies a tranalyzer rigged to not gag because
> of the missing leading 'FS21'.  The offsets within Fragment#1 are all
> correct.  That is, each transaction at the location in the file it thinks
> it ought to be.  Fragment#2 is 600KB and has about 500K of nulls at the
> beginning.  I am not yet sure if the offsets are 'proper' in it. I presume
> that these nulls are artifacts of the restoration process from the
> reiserfs.
> 
> So my question is:  What is the best strategy for getting the Folders,
> DTML Methods, DTML Documents and ZSQL Methods contained in these Data.fs
> fragments safely back into production?
> 
> Just to get the ball rolling
> 
> I have a done a evening's worth of experimenting with tranalyzer to
> evaluate the state of these fragments and explore strategies for getting
> their contents back into service.

Have you tried the FileStorage recovery tool, lib/python/ZODB/fsrecover.py?

There's a reasonable chance that if you add the missing "FS21" at the
beginning and use fsrecover:

  cd lib/python
  ZODB/fsrecover.py file_name

you will be able to recover the good data. (Note that it
works in place, so make a copy.)

If this doesn't work, you might be able to figure something
out by reading the function that this script calls. 

> It seems that it will not work to simply append these zope transaction
> records to the end of a Data.fs because these transaction records have
> offset data built into them (the backpointer). 

Right. There can be multiple backpointers.

> Some form of any of the
> following could get the job done, but which one?  Is there some better
> approach?  An existing tool?  Guru wisdom appreciated...
> 
> Plan 1
> ==
> Simply massage the data so it has the right offset values and then
> tack it on the end of the old Data.fs.

You may be able to get away with concatinating your old file
and the damaaged one and then using fsrecover top remove the
nulls and tack the new data onto the old, adjusting the offsets.
 
> Cons:
>   What will happen with transactions which are edits of missing objects?

Edits aren't a problem. It's references to missing objects that will
be problematic.
 
> Plan 2
> ==
> Have the data drive a program which interacts with a running ZODB and
> replays the transactions.  Several ways to do this come to mind...
>   a) write  an http client to perform the operations against a normal,
>  running Zope
>   b) write a tool which uses FileStorage.py to read the frag and write to
>  the clean Data.fs

Take a look at the FileStorage iterator in Zope 2.3 and CVS and at the
BaseStorage copy method.
 
> Plan 3
> ==
> Make a human readable form suitable for cut and pasting into zope via
> the regular management screens.

Eek.
 
Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: Status of aq_chain (was: RE: getPhysicalPath?)

[Jim Fulton]

Brian Lloyd wrote:
> 
> > 
> >
> >So nobody (including the various Products I have installed) uses this
> > incredibly useful attribute. I have numerous places in my source where I
> > loop through aq_parent attributes.
> 
> A good question is whether aq_chain really should be a
> public interface.

Yes.

> It was added (I think) as a debugging
> aid rather than necessarily to be used by application
> code. This is probably one for Jim to answer (I've CC'ed
> him on this).

Done. ;)

> If Jim thinks it should be part of the
> public interface, then someone should add a documentation
> bug report on this so that Amos and Michel can see that
> it is included in the right places.

Well, it is documented, along with other module goodies
in the Interfaces Wiki:

  http://www.zope.org/Members/michel/Projects/Interfaces/AcquisitionModuleInterface

 
> >It'd be nice to have even a mention in the acquisition documentation.
> > Poking around the Acquisition C source tells me that there's another
> > attribute, aq_inner, that isn't documented either.

It's documented in the Wiki too.

> I think aq_inner is a public interface - this should also
> be a doc bug report.
> 
> Before anyone asks, note that 'aq_uncle' is _not_ part of the
> public interface, and is only an exercise in Jim having fun :^)

And Bob's your uncle.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] Re: [ZODB-Dev] External transaction integration bug in TM.py

[Jim Fulton]

Sorry I took so long to reply.

"Randall F. Kern" wrote:
> 
> [Originally sent to zope-dev, but belongs here]

I'm CCing zope-dev, since you posted there too. 

> I may just be missing something obvious here, but it seems like there is
> a hole in ZODB.Transaction.Transaction.commit and Shared.DC.ZRDB.TM.TM,
> that can cause external transactions (those that use the TM mixin class,
> like psycopg) to be abandoned (never get committed or rolled back).
> 
> Let's say somewhere around line 300 in Transaction.py (the call to
> j.commit(o, self)) we get an exception.  Furthermore, let's say the TM
> derived database object has already been committed (the only effect of
> which is to move the DB into the jars mapping, since TM.tpc_begin() and
> TM.commit() both do nothing).
> 
> The exception dumps us down to about line 353, where we call
> _p_jar.abort() on all the uncommitted objects (note: it's important that
> the database connection isn't in this list, because TM.abort() will
> rollback the transaction.  the way we keep the db out of this list is by
> having it appear in objects _before_ the object that failed the commit).
> 
> Next we reach line ~366, where we should "unwind TPC for the jars that
> began it".  What this means is calling tpc_abort() on each jar from the
> objects that were already committed (which includes the database).
> However, TM.tpc_abort() does nothing, leaving the external database
> transaction open.
> 
> Does this make sense?

Yup, fraid so.
 
> If this makes sense (i.e. seems bad :), does catching tpc_abort() in TM
> and calling TM.abort() seem like a valid fix?  This change certainly
> seems
> to fix the problems.

Yes.  I think this is an improvement. Thanks. I'll check this in.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
Technical Director   (888) 344-4332http://www.python.org  
Digital Creationshttp://www.digicool.com   http://www.zope.org

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.interface: verifyObject vs properties

[Jim Fulton]

On Oct 15, 2008, at 3:27 AM, Thomas Lotze wrote:

> There has been a problem with zope.interface's verifyObject function
> that occurs in conjunction with Python properties: when verifyObject
> checks for the presence of an object's attribute, it does so by using
> hasattr() which in turn tries a getattr() call. If the attribute is
> implemented as a property, this may raise an exception which will be
> masked by hasattr() due to a bare except: clause. One scenario where
> this is a problem is a property that performs some component lookup
> which will succeed at application runtime but not in unit tests where
> verifyObject is commonly used.
>
> While it has also been argued that behaviour so complex that it may
> raise an Exception should not be implemented as a property in the
> first place, we believe that verifyObject should handle this case  
> better
> than it currently does. A possible fix would be to add an additional
> check for a data descriptor on the object's class if the hasattr()  
> test
> returns False.
>
> Are there any objections against modifying verifyObject in this way?


I would change it to just use getattr rather than hasattr.

try:
    getattr(ob, name)
except AttributeError:
return False
...

Jim


--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] traversal: different with and without a request

[Jim Fulton]

On Oct 15, 2008, at 12:02 PM, Philipp von Weitershausen wrote:

> Christian Theune wrote:
>> we stumbled over an annoyance that took a while to debug:
>>
>> Writing an ITraversable, we used zope.traversing.api.traverse() in a
>> test to verify our code. We registered the ITraversable as an
>> (non-multi) adapter and ended up with a working test.
>>
>> In the actual system, we found that the traversable would not be  
>> used.
>> After investigation we found a conditional branch in the traverse()
>> function which would look for a multi-adapter if a request was  
>> around,
>> and a regular adapter if not.
>>
>> We didn't anticipate this difference and it cost us some time, so we
>> wonder whether this has to be the way it is, or whether this could be
>> changed to behave more obvious and consistent.
>
> zope.traversing is a mess.

That's a bit strong, but essentially true.  It zope.traversing  
deserves to be reworked if anyone has the time and energy.


> First of all, its name is quite misleading.  It should really be  
> called
> 'zope.resolvepath' because it resolves TALES-like object paths. In  
> fact,
> it's pretty much only used by the PageTemplate machinery to hook it up
> to the TALES engine (with one exception, see below).

Historical note. Until we decided to use the location framework and  
eschew traversal proxies, is was much more widely used.

It would be nice to deprecate zope.traversing and fold it into  
zope.app.pagetemplate.

> The request
> shouldn't really be necessary for this kind of path resolution, I  
> think.

It's needed for looking up views and resources, both of which are  
commonly looked up in ZPT.

> The conditional multi-adaption sounds like a DWIM feature that I would
> consider one of our many mistakes that we made in the beginnings of  
> our
> using the Component Architecture.



I'll note that the fix, in the context of ZPT is to always to a multi- 
adapter lookup using the request.


> There is a process that actually needs the request and this process is
> what I call traversal: breaking down a URL and finding a publishable
> object. zope.traversing has (almost) nothing to do with it, the real
> kind of traversal happens in the publisher and facilitates
> IPublishTraverse adapters (rather than ITraversable).

Yup.

> The only case when
> the two kinds of "traversal" are intermingled is when ++namespaces++  
> are
> involved. Then IPublishTraverse-style traversal uses ITraversable
> adapters. This has long been considered a mistake but was never fixed.

Yup.  I would go so far as to call it a bug. Somebody please fix it. :)

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] SVN: ZODB/trunk/buildout.cfg Changed the generated test script to use a part-local tmp directory

[Jim Fulton]

On Oct 17, 2008, at 12:38 PM, Tres Seaver wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Jim Fulton wrote:
>> Log message for revision 92293:
>>  Changed the generated test script to use a part-local tmp directory
>>  that is cleaned up at the start of a test run.
>>
>>  Unfortunately, the ZODB tests leave lots of temporary files behind
>>  which can cause failures in subsequent test runs and which tend to
>>  litter /tmp. Eventually, I want to clean that up, but, in the mean
>>  time, I can limit the damage to the test part directory.
>>
>>
>> Changed:
>>  U   ZODB/trunk/buildout.cfg
>>
>> -=-
>> Modified: ZODB/trunk/buildout.cfg
>> ===
>> --- ZODB/trunk/buildout.cfg  2008-10-17 12:46:34 UTC (rev 92292)
>> +++ ZODB/trunk/buildout.cfg  2008-10-17 14:02:39 UTC (rev 92293)
>> @@ -6,6 +6,11 @@
>> [test]
>> recipe = zc.recipe.testrunner
>> eggs = ZODB3
>> +initialization =
>> +  import os, tempfile, shutil
>> +  if os.path.exists('tmp'): shutil.rmtree('tmp')
>> +  os.mkdir('tmp')
>> +  tempfile.tempdir = os.path.abspath('tmp')
>>
>> [scripts]
>> recipe = zc.recipe.egg
>
> Maybe zc.recipe.testrunner could add support for a 'finalization' or
> 'cleanup' argument which would do the same think as the  
> 'initialization'
> (or maybe only if there were no failures?)


Yup.  That would be good. In the mean time ... :)

It would also be good to have some automated way to check for leaving  
temp files behind, much as it now checks for leaving threads behind.

I've got a significant project ahead to fix the ZODB tests to be less  
sloppy about the way they manage files, although this initializer hack  
buys me some time. :)

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] traversal: different with and without a request

[Jim Fulton]

On Oct 17, 2008, at 1:13 PM, Stephan Richter wrote:

> On Friday 17 October 2008, Jim Fulton wrote:
>> It would be nice to deprecate zope.traversing and fold it into
>> zope.app.pagetemplate.
>
> Just skimming the thread, wouldn't this make it harder for us to  
> integrate
> z3c.pt, which also needs those traversal APIs?


Yup. Actually, this wants to be in zope.tales. Does z3c.pt use  
zope.tales?

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] [Checkins] SVN: zope.testing/trunk/ - Open doctest files in universal mode, so that packages released in Windoes

[Jim Fulton]

gt; +   8485%   samplelayers   (testrunner-ex/samplelayers.py)
> +1   100%   sampletests.__init__   (testrunner-ex/ 
> sampletests/__init__.py)
> +   48   100%   sampletests.test1   (testrunner-ex/sampletests/ 
> test1.py)
> +   74   100%   sampletests.test11   (testrunner-ex/sampletests/ 
> test11.py)
> +   74   100%   sampletests.test111   (testrunner-ex/sampletests/ 
> test111.py)
> +   76   100%   sampletests.test112   (testrunner-ex/sampletests/ 
> test112.py)
> +   74   100%   sampletests.test12   (testrunner-ex/sampletests/ 
> test12.py)
> +   74   100%   sampletests.test121   (testrunner-ex/sampletests/ 
> test121.py)
> +   74   100%   sampletests.test122   (testrunner-ex/sampletests/ 
> test122.py)
> +   48   100%   sampletests.test_one   (testrunner-ex/ 
> sampletests/test_one.py)
> +  11295%   sampletestsf   (testrunner-ex/sampletestsf.py)
> +Total: 405 tests, 0 failures, 0 errors in 0.630 seconds.
> False
>
> The directory specified with the --coverage option will have been  
> created and
>
> Modified: zope.testing/trunk/src/zope/testing/tests.py
> ===
> --- zope.testing/trunk/src/zope/testing/tests.py  2008-08-27 07:27:11  
> UTC (rev 90400)
> +++ zope.testing/trunk/src/zope/testing/tests.py  2008-08-27 07:41:00  
> UTC (rev 90401)
> @@ -23,10 +23,11 @@
>
> def test_suite():
> return unittest.TestSuite((
> +doctest.DocTestSuite('zope.testing.loggingsupport'),
> doctest.DocTestSuite('zope.testing.renormalizing'),
> +doctest.DocTestSuite('zope.testing.server'),
> +doctest.DocFileSuite('doctest.txt'),
> doctest.DocFileSuite('formparser.txt'),
> -doctest.DocTestSuite('zope.testing.loggingsupport'),
> -doctest.DocTestSuite('zope.testing.server'),
> +doctest.DocFileSuite('module.txt'),
> doctest.DocFileSuite('setupstack.txt'),
> -doctest.DocFileSuite('module.txt'),
> ))
>
> ___
> Checkins mailing list
> [EMAIL PROTECTED]
> http://mail.zope.org/mailman/listinfo/checkins

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] [Checkins] SVN: zope.testing/trunk/ - Open doctest files in universal mode, so that packages released in Windoes

[Jim Fulton]

On Oct 17, 2008, at 3:16 PM, Stephan Richter wrote:

> On Friday 17 October 2008, Jim Fulton wrote:
>> testrunner-coverage.txt is failing since this check in.  What's up
>> with this?
>
> It passes for me on Python 2.5.

Sigh.  It passes for me too with 2.5.

> Can you be more specific?

It fails with Python 2.4.

I guess we should live with this. :( I don't feel like chasing it and  
I'm sure you don't either.

Never mind.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Issues with restricted Python (was Re: Zope 2.12 - supported Python versions)

[Jim Fulton]

On Oct 21, 2008, at 8:29 AM, Chris Withers wrote:

> Sidnei da Silva wrote:
>>> I always was under the impression that Jim feared the code and the
>>> required security audit was perceived as a major painful  
>>> undertaking.
>>
>> That was my perception too. But after looking at the code it is  
>> really
>> not bad at all.
>
> The PyPy guys, who seem to be the authority on this kind of stuff,
> always shake their heads in woe at the mention of RestrictedPython.

As well they should.  Relying on it is a lot of work and brittle.

> I've tried to understand what they perceive the "real problems" with  
> it
> to be but haven't managed to extract anything meaningful :-(

The problem is that it it starts with an environment in which things  
are allowed by default, and takes things away. This means that if  
anything is forgotten, then you end up with holes.

A better approach is to start with something that lets you do nothing  
and add capabilities that are known to be safe.  This is why some  
folks get all excited about capability-based security.

The Zope 3 security proxy approach is much better than relying solely  
on restricted Python, because, as with a capability-based approach, it  
prevents what isn't explicitly allowed.  It does this (for the most  
part) without having
to do code manipulation. It still uses restricted Python do deal in a  
narrow way with basic objects, like strings and numbers, that are  
unproxied. It's use of restricted Python is so narrow that it is far  
less problematic. It would be really great if Zope 2 would switch to  
security proxies, although the transition is likely to be painful.

I'm not sure that the PyPy guys are really authorities on the sorts of  
problems we're trying to address, although there is some overlap.  If  
I remember correctly, they are just focussed on protecting the system  
from untrusted scripts.  Our problem is harder because we also want to  
protect objects available to the scripts.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Issues with restricted Python (was Re: Zope 2.12 - supported Python versions)

[Jim Fulton]

On Oct 24, 2008, at 10:01 AM, Chris Withers wrote:

> Jim Fulton wrote:
>> The problem is that it it starts with an environment in which  
>> things  are allowed by default, and takes things away. This means  
>> that if  anything is forgotten, then you end up with holes.
>
> Isn't there a way we could change the AST manipulation such that we  
> start with nothing and only allow opcodes as and when they're added  
> to the RestrictedPython implementation?

No. we're starting with an existing program written in a Python script  
or expression.  We then have to sanitize it.

>> The Zope 3 security proxy approach is much better
>
> I agree, but it doesn't solve all the problems. My understanding of  
> the original set of requirements which we're trying to solve here  
> was basically that of Python Scripts: to allow python code to be  
> written through the web. This means:
>
> - restricting access to atributes of objects
>  (security proxies give us this, right?)

Yes

> - restricting access to features of the language such as imports such
>  that unsafe things such as stripping security proxies can't be done.
>  (security proxies *don't* give us this, right?)

Yes, they do, as long as all builtins, including __import__, are  
wrapped and as long as you only include safe builtins.


> ...and some nice to haves:
>
> - restricting memory used by executing the code
> - restricting cpu used by executing the code

These require deep changes to the interpreter.

> I know RestrictedPython doesn't support these last two very well,  
> but there are hints that it would have liked to if it could.

It can't support these at all.

>> I'm not sure that the PyPy guys are really authorities on the sorts  
>> of  problems we're trying to address, although there is some overlap.
>
> Who is then? ;-)

In the Python community, I;d say we are.  Beyond that, I don't know.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Blobs and modes

[Jim Fulton]

On Oct 24, 2008, at 3:24 PM, Benji York wrote:

> Is there a good reason blobs don't support "b" or "t" in the mode
> strings passed to "open"?  I'm refactoring some code that expects a
> file-like object to use blobs and it wants to pass "wb" as the mode
> (which is a sane thing to do).

Blobs are implicitly binary. (Note the "B" in Blob.)

> I'll add support for "b" and "t" to the blob code if no one objects.

I object.

BTW, this questions should have been asked o zodb-dev.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] [Checkins] SVN: zc.recipe.cmmi/trunk/README.txt Corrected the link to the SVN repository; I had put in the direct one via

[Jim Fulton]

On Oct 30, 2008, at 8:59 AM, Benji York wrote:

> On Thu, Oct 30, 2008 at 8:15 AM, Jeff Rush <[EMAIL PROTECTED]> wrote:
>> Log message for revision 92695:
>> Corrected the link to the SVN repository; I had put in the direct  
>> one via
>> svn+ssh when it should have been the one accessed via http.
>
>> -  
>> <http://svn.zope.org/repos/main/zc.recipe.cmmi/trunk#egg=zc.recipe.cmmi-dev 
>> >
>> +  <http://svn.zope.org/zc.recipe.cmmi/trunk#egg=zc.recipe.cmmi-dev>
>
> I'd be surprised if setuptools can build an egg from the ViewCVS web
> page that http link points at.

But it will waste time trying. :/
>
>
> I suspect you want svn://svn.zope.org/repos/main/zc.recipe.cmmi/trunk
> which is available anonymously (not the lack of "+ssh" in the  
> protocol).

Yup.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] misterious btree issue

[Jim Fulton]

On Nov 6, 2008, at 8:14 AM, Jacob Holm wrote:

> Hi Adam
>
> Adam GROSZER wrote:
>> Hello,
>>
>> I've run into a misterious issue while evolving generations from an
>> old DB.
>>
>> Quick fact is that it seems like a BTree kept an object reference to
>> an object which was deleted from it.
> [snip]
>
> Yes, the BTree implementations we use may keep a reference to a  
> deleted
> key. I think I have seen this documented somewhere but don't have the
> time to look it up now. It saves some bookkeeping to do it that way,  
> and
> I think it also reduces the risk of conflict errors.

In particular, it might have a reference to the key in an internal  
BTree node.
(I don't know this to be true, but I wouldn't be surprised if it was  
true.)
If this is the case, we can definitely fix it.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] misterious btree issue

[Jim Fulton]

On Nov 6, 2008, at 11:32 AM, Tim Peters wrote:

> [Jim Fulton]
>> In particular, it might have a reference to the key in an internal
>> BTree node.
>> (I don't know this to be true, but I wouldn't be surprised if it was
>> true.)
>
> It was true when I was working on BTrees ... here, from
>
>
> http://svn.zope.org/ZODB/trunk/src/BTrees/Development.txt?rev=85198&view=markup
>
>...
>+ When a key is deleted, it's physically removed from the bucket
> it's in, but
>  this doesn't propagate back up the tree: since keys in interior  
> nodes only
>  serve to guide searches, it's OK-- and saves time --to leave
> "stale" keys in
>  interior nodes.
>...
>
>> If this is the case, we can definitely fix it.
>
> Right, just delete that paragraph ;-)

:)

It would be great if someone (Adam?) created a launchpad issue to fix  
this.

If I get into the BTree code, which I may do to make sizes setable via  
class attributes, I'll look at fixing this.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] misterious btree issue

[Jim Fulton]

On Nov 6, 2008, at 1:06 PM, Adam GROSZER wrote:

> Hello Jim,
>
> Does this fit?
>
> https://bugs.launchpad.net/zope3/+bug/294788


Yup. Thanks!

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] SVN woes.

[Jim Fulton]

On Nov 11, 2008, at 8:52 AM, Jim Fulton wrote:

>
> I'm going to restore svn from a backup and see where that leaves  
> us.  I'm going to disable svn access while I work on this.


The repository is back now.  Much thanks to Christian Theune for  
providing a replication image.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] SVN woes.

[Jim Fulton]

I'm going to restore svn from a backup and see where that leaves us.   
I'm going to disable svn access while I work on this.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Removing ZopeUndo from ZODB, starting with ZODB 3.9

[Jim Fulton]

I swear I remember discussing this a while back but I can't find an  
email trail

I'm going to remove ZopeUndo from ZODB, starting with 3.9.   Zope 2  
already has a copy (well, an external from a ZODB tag) and if someone  
needs it to be in a ZEO server for Zope 2 applications, they can make  
an egg for it and include the egg in the ZEO server.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] Change the CVS repository to read-only

[Jim Fulton]

It appears that no one has committed to the CVS repository in almost 2  
years.  I propose to change it to read-only.

Any objections?

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] deprecate http://download.zope.org/distribution

[Jim Fulton]

http://download.zope.org/distribution was set up as a place to publish  
distributions while we were learning about setuptools, eggs, and  
pypi.  I think now, we're all using PyPI (or local repositories).  I'd  
like to deprecate this site, making it read-only for now, but  
eventually removing it.

Any objections?

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] C-extension in zope.i18nmessageid

[Jim Fulton]

On Dec 12, 2008, at 6:28 AM, Malthe Borch wrote:

> I've branched out this package and removed the C-extension. It's not
> documented in the package why a C-extension is needed or  
> alternatively,
> what it benefits.
>
> If there are no objections, I will merge this into trunk shortly.


I object. Please drop it.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Plans for Zope 2.12

[Jim Fulton]

On Jan 22, 2009, at 9:34 PM, Tres Seaver wrote:
> I'm actually willing to abandon the "big tree" altogether, unless
> somebody comes up with a clever way to automate it from some Z2- 
> specific
> KGS index.  I think the canonical "source install" would be something
> like a tarball of a buildout tree, with the 'download-cache' directory
> already populated (maybe).


Yup (sort a). See zc.sourcerelease.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] can these wrong DEPENDENCIES.cfg files be removed?

[Jim Fulton]

On Jan 27, 2009, at 12:36 PM, Stephan Richter wrote:

> On Tuesday 27 January 2009, Fred Drake wrote:
>> On Tue, Jan 27, 2009 at 6:01 AM, Dan Korostelev   
>> wrote:
>>> AFAIK, these files are for (pre-egg, no-longer-used) zpkg tool. They
>>> are removed in zope.app.renderer's trunk, BTW.
>>
>> I certainly hope no one is relying on that old tools!
>
> I care about those files in Zope3/ and ZODB3/, since they are used  
> to create
> the Zope 3 tar ball. Remember, Zope 3.4 will be the *first* release  
> where we
> officially introduce eggs. And I have heard from a lot of people  
> that would
> like to see a couple more TAR ball releases. I have a script to  
> update the
> Zope3/ tree, so it is fairly easy for me to fulfill that request.

What do people want from a tar release.  Wouldn't their needs be  
adequately met by a source release made with zc.sourcerelease?

Jim


--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.security changes

[Jim Fulton]

On Jan 30, 2009, at 6:59 AM, Chris Withers wrote:

> Fred Drake wrote:
>> On Thu, Jan 29, 2009 at 4:01 AM, Martijn Faassen > > wrote:
>>> I believe it'd be nicer to extract any ZCML related stuff from
>>> zope.component at some point and put it into zope.componentzcml or
>>> something like that. We could then decide to move the  and
>>>  directives in there as well.
>>
>> +1
>>
>> This makes a lot more sense to me than having the ZCML support in
>> either zope.component or zope.security.
>
> Indeed, surely all zcml stuff belongs in zope.configuration anyway?


No!!!

zope.configuration just contains the implmentation of zcml itself.  It  
shouldn't have any application-dependent code.

I have no problem with moving zcml out of other packages, but it has  
to move into new packages, not into zope.configuration.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.security changes

[Jim Fulton]

On Jan 30, 2009, at 12:01 PM, Martijn Faassen wrote:

> Chris Withers wrote:
>> Fred Drake wrote:
>>> On Thu, Jan 29, 2009 at 4:01 AM, Martijn Faassen >> > wrote:
>>>> I believe it'd be nicer to extract any ZCML related stuff from
>>>> zope.component at some point and put it into zope.componentzcml or
>>>> something like that. We could then decide to move the  and
>>>>  directives in there as well.
>>> +1
>>>
>>> This makes a lot more sense to me than having the ZCML support in
>>> either zope.component or zope.security.
>>
>> Indeed, surely all zcml stuff belongs in zope.configuration anyway?
>
> No, not there either, as zope.configuration doesn't define *any*
> directives except the basic ones like 'include' and 'configure'. If  
> you
> would implement zope 3's directives in zope.configuration it'd start
> pulling in dependencies like crazies, creating more dependency cycles.
>
> I think a new package might be in order.


Yes please. (I should have read this before sending the other message  
I just sent. :)

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.security changes

[Jim Fulton]

On Jan 30, 2009, at 1:48 PM, Brian Sutherland wrote:
>
> Please don't, having namespace packages that contain files (as
> zope.configuration already does) breaks setuptools.


zope.configuration isn't a namespace package.  It is simply a package  
with subpackages.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] ** PROBLEM alert - Zope Project Services (SVN/Downloads/Mailman/...)/KGS age is WARNING **

[Jim Fulton]

On Feb 1, 2009, at 5:10 AM, Jens Vagelpohl wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
>
> On Feb 1, 2009, at 06:54 , Stephan Richter wrote:
>
>> On Saturday 31 January 2009, Jens Vagelpohl wrote:
>>> I just need to know what to look for to see if
>>> it's fixed.
>>
>> Works.
>
> Quick question: How is doenload.zope.org maintained? Shell access?

Yes

> And
> who does it?

Mostly Stephan at this point.

> It should be moved to the foundation servers at some point.


+1

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Differing case for Buildout on Windows

[Jim Fulton]

On Feb 5, 2009, at 3:36 PM, Sidnei da Silva wrote:

> I've been constantly bitten by a very annoying bug: depending on how
> you call buildout on Windows, a full rebuild might be triggered, if
> only the case differs. For example, all the following calls cause
> buildout to think settings have changed and that things need to be
> rebuilt. Note only the drive letter changes:
>
>  c:\Python24\python.exe bin\buildout-script.py -c
> c:\src\some-project\buildout.cfg
>  c:\Python24\python.exe bin\buildout-script.py -c
> C:\src\some-project\buildout.cfg
>  C:\Python24\python.exe bin\buildout-script.py -c
> c:\src\some-project\buildout.cfg
>  C:\Python24\python.exe bin\buildout-script.py -c
> C:\src\some-project\buildout.cfg
>
> Now, I'm wondering what is the best way to fix it. I originally
> thought of fixing the data as it comes into buildout, but that would
> cause existing buildouts to all possibly trigger a rebuild. I'm now
> thinking of fixing the place where buildout detects that the path has
> been changed as to normalize the path.
>
> Thoughts?


I think it would be easiest to have buildout normalize the config path  
using os.path.normcase where it is input.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] multiple packages in the same namespace

[Jim Fulton]

On Feb 6, 2009, at 7:35 AM, Chris Withers wrote:

> Jim Fulton wrote:
>> zope.configuration isn't a namespace package.  It is simply a  
>> package  with subpackages.
>
> Does setuptools support something like:
>
> "packagea":
> packagea/__init__.py
> packagea/amodule.py
>
> "packagea.something":
> packagea/__init__.py
> packagea/something/__init__.py
> packagea/something/bmodule.py

No.

> If so, which packagea/__init__.py gets used?


That's why it doesn't. :) That's also why zope.configuration isn't a  
namespace package.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] ZCML implementations: where should they go

[Jim Fulton]

On Feb 7, 2009, at 11:07 AM, Martijn Faassen wrote:

> Hi there,
>
> We've recently had some discussions on where to place the  
> implementation
> of various ZCML directives. This post is to try to summarize the issue
> and analyze the options we have.
>
> Right now ZCML directives are implemented in packages that contain  
> other
> implementation. For example, zope.component implements various ZCML
> directives, and zope.security implements some more.
>
> In the case of zope.component, a special [zcml] extras dependency
> section is  declared. If the ZCML dependencies are asked for, using
> zope.component will suddenly pull in a much larger list of  
> dependencies
> than the original zope.component dependency list. The ZCML directives
> are component-related, but do offer extra options that need bits from
> the wider Zope 3 framework, such as the security infrastructure.
>
> In the case of zope.security, this isn't the case. As far as I can  
> see,
> it doesn't declare any dependency beyond zope.configuration to allow  
> it
> to implement its ZCML directives.
>
> The dependency situation for the ZCML implementations in  
> zope.component
> doesn't appear ideal. It was therefore proposed to move the ZCML
> implementations to another package. This could be a new package, or it
> could be created.
>
> Following up on that, it was considered we should move *all*  
> directives
> from the packages that implement them now into special packages. This
> would allow some packages to lose the dependency on  
> zope.configuration,
> which is a relatively minor gain.
>
> We have several ways to go:
>
> a) continue with the current extra dependencies situation like in
> zope.component, and in fact clean up other packages that define ZCML  
> to
> declare ZCML extra dependencies.

I did this in zope.component out of desperation.  (I was preparing to  
teach a PyCon tutorial on using zope.component outside of Zope.) I'm  
not at all happy with it, however, I'd be in favor of continuing it  
for existing packages with zcml implementations so as not to introduce  
backward incompatibilities.  I'd rather not do it for new packages.   
IMO, introducing an extra is like introducing a new package and in a  
rather complicated way.

> b) pull out all ZCML implementations from where they are now and put
> them in special ZCML implementation packages. We could for instance  
> have
> zcml.component, or zope.component_zcml, or  
> zope.configuration.component.
> We had a bit of a side-tracked discussion about naming and namespace
> packages here.

I think this is the right way to go for new software.


> c) pull out only those ZCML implementations that cause extra
> dependencies beyond zope.configuration. So, we extract the bits of
> zope.component into a new package, but we don't extract bits from
> zope.security.

Too complicated imo. :)

> I personally don't like extras. I think the ideal situation would be  
> if
> packages had *no* extras at all (even test extras), as it complicates
> reasoning about the dependency structure.

+1

...

> For that reason, a) is not really an option for me.

As I said above, I'm for a) because I think it is less disruptive,  
even though I share your distaste for extras.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] opensp...@pycon 2009 about Zope/Repoze/Grok/Deliverence etc.

[Jim Fulton]

On Feb 9, 2009, at 12:03 PM, Lennart Regebro wrote:
> So, I propose to have an Open Space session at PyCon, Chicago, March
> 27-29 .

Count me in. (I'll also be sprinting the first 2 sprint days.)

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] ZCML implementations: where should they go

[Jim Fulton]

On Feb 10, 2009, at 1:49 PM, Dieter Maurer wrote:

> Jim Fulton wrote at 2009-2-8 13:00 -0500:
>> ...
>> IMO, introducing an extra is like introducing a new package and in a
>> rather complicated way.
>
> I agree with the first part of your sentence -- but cannot follow you
> with the second part:
>
>  How can "'extra' : "
>  be more complicated than creating, maintaining and
>  distributing a complete package?


Because you have to remember to test each valid permutation of the  
package.  I bet no one does.  Also, users have to be aware of the  
extras. PyPI doesn't advertise extras nor are there standard ways to  
document them. In general, it makes an already complicated packaging  
system more complicated.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Coding style clarifications

[Jim Fulton]

On Feb 19, 2009, at 4:47 AM, Christian Theune wrote:

> Hi,
>
> while gathering, cleaning and consolidating the various statements  
> that
> float around about the coding style for Zope 3, I found a couple of
> issues that I'd like to get clarification for.
>
> What I found is currently gathered at
> http://svn.zope.org/zope3docs/source/codingstyle/python-style.rst?rev=96729&view=auto
>
> Which attribute naming is current?
> ==
>
> Do we use under_scores or mixedCaseNames?
>
> I think I remember that we decided to follow PEP 8 for new code and
> invoke the "local consistentency" rule on old code. Is that correct?

This is a mess, mainly because PEP 8 changed after an alternative  
style had been established in the Zope community.  This is complicated  
by the fact that many people who work on Zope projects also work on  
many non-zope projects.

I only mention this so that readers know the history.

I don't really care much about this issue.  Whatever decision is made  
is arbitrary.


> Global variable names
> =
>
> I don't understand that rule. I kept it around because someone might  
> be
> able to explain it more in-depth and whether it still applies.

I don't understand it either.  I think it mostly follows established  
Python practices which have been changing recently.

I don't care much about this myself.

> Import ordering
> ===
>
> Jim proposed an alternative rule for ordering imports. Is this  
> official?
>
> (My vote is +1 on it)


PEP 8's rule is silly subjective and generally a waste of time. I'm  
not going to waste my time following it.

I sort my imports.  Period. This makes from imports come before  
regular imports (because f comes before i).  I discourage from  
imports, so this isn't much of an issue for me except for old code.  
Having imports sorted takes very little effort and makes imports  
easier to find and duplicated easier to spot. Grouping imports makes  
imports harder to maintain and read, especially since groupings are  
arbitrary unless they follow package boundaries, and just sorting  
imports groups imports by package boundaries.

BTW, I strongly discourage from imports. (I didn't always have this  
opinion, but have seen the error of my ways. Thanks to Fred Drake for  
nudging me in this direction.) IMO, this is wildly more important than  
any of the issues raised in this thread.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Coding style clarifications

[Jim Fulton]

On Feb 19, 2009, at 12:43 PM, Tres Seaver wrote:
> - -1.  I prefer the PEP8 grouping, where "stdlib" imports are  
> separated
> from "dependecy" imports, which are separated from "local" imports.
> Note that this is *not* subjective (an import is unambiguously in
> exaclty one of those three classifications.)


I find it very subjective, or, at best, orthogonal to anything I care  
about.

If I'm working in a ZEO package, is an import from ZODB local? How  
about transaction?

If I import asyncore or cPickle, is that coming from the standard  
library? It is today, but it wasn't in the past, when the code I'm  
reading may have been written.

The grouping in PEP 8 is based on packaging decisions that are  
irrelevant to the meaning of the software.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Coding style clarifications

[Jim Fulton]

On Feb 19, 2009, at 2:07 PM, Marius Gedminas wrote:
...
>  -1 for repeating
> english.adjective.fully english.adverb.qualified english.noun.package
> english.noun.names all over the place in the code

If you have package hierarchies remotely that deep, you have bigger  
problems.

If dotted names are long enough to be a significant bother, packages  
are too deeply nested.  Too deeply nested packages was a mistake we  
made in Zope 3 early on.

I prefer to have no-more than two levels of packages, a namespace  
package and a project package within the namespace packages.  (I can  
live with 3 levels in some cases.)

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Coding style clarifications

[Jim Fulton]

On Feb 19, 2009, at 4:07 PM, Shane Hathaway wrote:

> Fred Drake wrote:
>> On Thu, Feb 19, 2009 at 11:03 AM, Jim Fulton  wrote:
>>> BTW, I strongly discourage from imports. (I didn't always have this
>>> opinion, but have seen the error of my ways. Thanks to Fred Drake  
>>> for
>>> nudging me in this direction.) IMO, this is wildly more important  
>>> than
>>> any of the issues raised in this thread.
>>
>> You're welcome.  :-)
>
> You're saying that:
>
> import zope.interface.Interface
>
> class IFoo(zope.interface.Interface):
> ...
>
> is better than:
>
> from zope.interface import Interface
>
> class IFoo(Interface):
> ...


This is an interesting case.  This could make a good exception to the  
rule.  Interface is so commonly used, at least within the zope  
community it really wants to be a builtin, like list or tuple.  Most  
names are not nearly as common.

Even with this example, which I consider atypical, I think there are a  
couple of interesting things to note:

1. While Interface is well known to us, it isn't well known to other  
possible readers of the code.  When working on a project, certain  
names are very will known to me and don't seem to benefit from  
qualification, however, when I leave a project and come back to it  
much later, I appreciate the qualification that makes interpreting the  
no-longer familiar names much easier.

2. Depending on what code you might unearth, Interface could be either  
zope.interface.Interface, or the older version Interface.Interface.   
Package qualification makes this unambiguous.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Coding style clarifications

[Jim Fulton]

On Feb 20, 2009, at 7:48 AM, Joachim König wrote:
...
>> 2. Depending on what code you might unearth, Interface could be  
>> either  zope.interface.Interface, or the older version  
>> Interface.Interface.   Package qualification makes this unambiguous.
>>
> if module qualification is enough and "from" discouraged one could  
> as well write:
>
>  import zope.interface as interface
>
>  class IFoo(interface.Interface):
>
>   ...


"as" is evil and should never be used except in the most extreme  
circumstances, if there even are any that extreme.

We have recommended:

  from zope import interface, component

and then not having to type "zope.", as you suggest above.  I've come  
to dislike this because:

- It doesn't buy that much, and

- It's an odd case that has to be recognized.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Coding style clarifications

[Jim Fulton]

On Feb 20, 2009, at 10:15 AM, Martijn Faassen wrote:

> Jim Fulton wrote:
>
>> BTW, I strongly discourage from imports. (I didn't always have this
>> opinion, but have seen the error of my ways. Thanks to Fred Drake for
>> nudging me in this direction.) IMO, this is wildly more important  
>> than
>> any of the issues raised in this thread.
>
> I like from imports as they allow me to provide a public API in the
> __init__.py of a package (instead of scattering it all over the  
> place in
> sub-modules of the package nobody should need to know about). Absolute
> imports tend to break this ability in subtle ways.
>
> I suspect there are two possibilities:
>
> * no API defined in __init__.py and absolute imports
>
> * API defined in __init__.py and dotted.package.name.references  
> don't work.

In what way don't they work?

I don't see how this has anything to do with from imports.  Putting an  
API in the package __init__ just makes object paths shorter.  I don't  
see how the location of an API has much bearing on whether to use from  
imports or not.

> This looks like a religious debate though.


Maybe. The most important reason for a style guide is to make code  
more readable.  In my experience, from imports make code significantly  
harder to read, so this is fairly important to me.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Coding style clarifications

[Jim Fulton]

On Feb 20, 2009, at 10:22 AM, Martijn Faassen wrote:

> Jim Fulton wrote:
>> On Feb 19, 2009, at 2:07 PM, Marius Gedminas wrote:
>> ...
>>> -1 for repeating
>>> english.adjective.fully english.adverb.qualified  
>>> english.noun.package
>>> english.noun.names all over the place in the code
>>
>> If you have package hierarchies remotely that deep, you have bigger
>> problems.
>
> Dotted name imports within a package's modules like that break the
> ability for people to create package specific namespaces in  
> __init__.py
> due to subtle circular dependency issues. from imports tend to avoid
> this as the namespace package isn't actually needed yet for dotted
> access at a critical point.


Names exported to a containing package cause circular import problems  
whether or not from imports are used.  I've seen from imports make  
this worse. I believe you've seen cases where they make it better.  I  
think the only way to avoid this is to use a deferred import mechanism  
such as zope.deferredimport. (This is one of the few batteries I  
actually wish Python included.)

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Coding style clarifications

[Jim Fulton]

On Feb 20, 2009, at 11:00 AM, Martijn Faassen wrote:
...
> The main take-home message was that the import mechanics of Python are
> rather surprising in operation here and it's very hard to reason about
> it. It has something to do with 'foo'" having to be more initialized
> during importing than in the other case. I do know that the pattern of
> 'from' imports in a package's module will allow me to write a
> __init__.py assembling a namespace from sub-modules, while the pattern
> of "import foo.bar" in a package's module will give circular import
> related errors. "foo" in the latter case will have to be a more
> complete object.


You will still likely have other problems unless you use deferred  
imports which will generally solve this problem in a robust way.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.publisher dependencies

[Jim Fulton]

On Feb 24, 2009, at 3:08 AM, Shane Hathaway wrote:

> I've been working on the dependencies to and from zope.publisher.
> Refining the dependencies should make it easier to integrate
> zope.pipeline when it's ready.

Can you elaborate on this a bit?

> I've noticed that nearly all packages that depend on zope.publisher
> depend only on a few pieces of it:
>
>   - zope.publisher.interfaces
>
>   - zope.publisher.browser.Browser{View|Page}
>
>   - zope.publisher.browser.TestRequest


I'd like to turn this around a little bit.  Only browser-based code  
should depend on zope.publisher.  This seems like a very reasonable  
dependency.  It's like wxwindows UI code depending on wxwindows.   
Maybe the browser code should be factored out of the packages that  
depend on zoep.publisher so that only *that* code has the dependency  
and non-browser code doesn't.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.publisher dependencies

[Jim Fulton]

On Feb 24, 2009, at 9:26 AM, Tres Seaver wrote:
...
>> As for TestRequest, I could update the setup.py of various packages  
>> that
>> currently depend on zope.publisher just for TestRequest.  I would  
>> make
>> zope.publisher a test-only requirement.
>
> Frankly, any code using a testing stub which is that heavyweight needs
> to be refactored.


There's nothing all that heavyweight about TestRequest.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.publisher dependencies

[Jim Fulton]

On Feb 24, 2009, at 11:12 AM, Tres Seaver wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Jim Fulton wrote:
>> On Feb 24, 2009, at 9:26 AM, Tres Seaver wrote:
>> ...
>>>> As for TestRequest, I could update the setup.py of various packages
>>>> that
>>>> currently depend on zope.publisher just for TestRequest.  I would
>>>> make
>>>> zope.publisher a test-only requirement.
>>> Frankly, any code using a testing stub which is that heavyweight  
>>> needs
>>> to be refactored.
>>
>>
>> There's nothing all that heavyweight about TestRequest.
>
> - - It derives from BrowserRequest, which means carrying along a *lot*
>  of extra implementation baggage.  Tests which use this class, rather
>  than stubbing out a dummy request which provides only the API  
> required
>  by the application-under-test, will tend to grow unexpected /
>  undesirable tentacles to the request implementation.
>
> - - Using TestRequest involves pulling in all of zope.publisher, a  
> *big*
>  dependency;  Shane wants to reduce such dependencies.


OK, I don't agree that zope.publisher is a big dependency, especially  
for code that is meant to run in the context of it.  Any view (or  
resource) code, which is the only code who's tests would need  
zope.publisher, will be used in together with zope.publisher in  
practice.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )