Re: [Zope-dev] Re: hasattr geddon

2004-08-18 Thread Michael Dunstan
On 12/07/2004, at 12:29 PM, Chris McDonough wrote:
FWIW, as far as I understand the prevent commit on conflict behavior
*is* currently the behavior for caught ReadConflictErrors.  The only
time ZODB doesn't exhibit this behavior is *during the commit of a
transaction*.  If a commit is attempted at this point and a conflict
error is caught unintentionally, the following happens (as explained by
Tim at http://mail.zope.org/pipermail/zodb-dev/2004-May/007404.html :
- the attempt to commit fails, raising a write conflict error
- the write conflict error is unintentionally caught (whether by 
hasattr
or a bare except or by whatever)

- a new transaction begins implicitly (as if 
get_transaction().abort()
or get_transaction().begin() were called)

- ... whatever happens after this happens ..
Since it's extremely unlkely for a write conflict error to be caught
unintentionally in Zope (it would require an application to perform a
get_transaction().commit() going outside the boundaries of the 
default
transaction manager), it's an obscure failure mode.

If the transaction was not put into a committable state after a caught
exception error, there would be no problem.  But to be fair, I don't
think there is much of a problem right now, as very few Zope apps 
manage
their own transactions (except for the Catalog... gulp ;-).  But it
would be nice if ZODB didn't do the implicit 
get_transaction().abort()
after a write conflict error and left the transaction in an
uncommittable state instead... or at least had an option to do this.
So... it sounds like ZODB will take care of itself in the face of 
ReadConflictErrors swallowed by bare excepts (and hasattr's). (Unless 
your managing your own transactions.)

One penalty of bare excepts swallowing a ConflictError is running more 
code than is needed. The transaction is going to be thrown away 
eventually.

Is there problem with seeing bogus exceptions in code after a swallowed 
ConflictError?

I'm thinking that when zope catches any exception in a transaction it 
should check the state of the transaction to decide if should treat 
this as a ConflictError or report the exception it caught.

Consider a contrived example::
def setFoo(self, value):
try:
self.foo = value
except:
pass
self.fooUpdated()
Assume that the except swallows some ConflictError then an exception is 
raised from fooUpdated()
but not a ConflictError, lets say a KeyError. Sure, ZODB is tough 
enough to look after itself in this case. What about the developer 
trying to interpret the KeyError that is written to event.log?

(Hmm... I should really check that is what you could potentially see 
with a current Zope :-)

Michael
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: hasattr geddon

2004-07-12 Thread Dieter Maurer
Shane Hathaway wrote at 2004-7-10 02:41 -0600:
 ...
Hmm.  I just heard about this hasattr geddon.  hasattr is *good*.  Why are 
we fixing hasattr and bare excepts when the real problem is ZODB?

That's even better!

However, we may still be interested to get an exception unswallowed
by hasattr. Usually, there is some sense in not continuing
an expensive computation when we know it will finally fail...

When we fix ZODB, we should note that information about the precise
cause is helpful in fixing the underlaying problem.
Thus, something like the original exception, maybe even with a
traceback representation, should be available.

 ...
That said, I was surprised to discover that Python 2.3 implements hasattr this 
way (from bltinmodule.c):

Me, too...

 ...
I suggest 
the consistency between getattr and hasattr should be fixed in Python, not 
Zope.

Tim explained a bit why hasattr is like it is...

And when we cannot change it in Python, we may need our own hasattr...


-- 
Dieter
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


ZODB sticky conflicts (was: Re: [Zope-dev] Re: hasattr geddon)

2004-07-12 Thread Dieter Maurer
Tim Peters wrote at 2004-7-10 17:55 -0400:
 ...
Which is why the current ZODB releases intend to prevent committing a
transaction if a conflict error occurred during the transaction.  It
shouldn't matter to this ZODB machinery if the application suppressed
the original conflict error(s), ZODB remembers that it raised
ReadConflictError regardless (via the Connection._conflicts
set-implemented-as-a-dict).

Hmm.  The lack of design documentation is frustrating.  Staring at the
code, it could be that  ZODB only prevents a transaction commit if the
transaction tries to *commit* at least one object that suffered a
conflict error,  I don't know ... if you have a specific case in mind,
please open a Collector report about it.

This is indeed the case:

  The code you mentioned prevents the modification (i.e. commit)
  of an object that caused a ReadConflictError when it
  has been accessed -- whether or not the exception has been
  caught or not.

Shane seems to want a stricter behaviour: something like:

  def commit(...):
  ...
  if self._conflicts:
  raise SpecialConflictError(...)

   i.e. prevent the commit, when the transaction saw any
   conflict whether or not it tries to modify such an object.

However, I think his wish is not necessary.

  Even when a ReadConflictError was swallowed, the object has not
  been read and its state is still undefined.
  Whenever the application tries to read an attribute,
  the object is refetched and the ReadConflict reissued.
  When the conflict successfully trickles down to the
  commit, then in fact, the transaction was independent
  from the state of the respective object...

  The only thing, the application can do is do write
  the object. However, writing implies fetching the
  object -- with the consequence of another ReadConflict
  (if I am right, then the code in commit is redundant,
  as the application is unable to write an object that
  could not be fetched due to a ReadConflict).

The problem that remains:

  The application swallowed an exception and, as a consequence,
  did not recognize that it was unable to *write* an object.
  It performs its other writes and commits. In this
  case, the originally written object remains unchanged
  and inconsistent with respect to other objects.

  Shanes wish would recognize this situation.

Obviously, I have to retract my previous thought :-)

-- 
Dieter
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: hasattr geddon

2004-07-11 Thread Shane Hathaway
On Saturday 10 July 2004 15:55 pm, Tim Peters wrote:
 [Shane Hathaway]

  Here is what often happens in Zope:
 
  def setFoo(self, value):
   try:
 self.foo = value
   except:
 LOG(Oops, ERROR, Some error happened, error=sys.exc_info())
   self.change_count += 1
   self.get_indexes().update(self)
 
  Some piece of code has a legitimate reason to catch but log all
  exceptions. Some other piece of code updates indexes.  The database has
  now committed a partial transaction.

 I need more words -- I don't see a commit() in that code, and I don't
 know what partial transaction might mean (in ZODB you can commit a
 transaction, or abort it -- there's no facility I know of for a
 partial commit (or a partial abort, for that matter)).

Commit occurs later, in the publisher.  It is a partial transaction because 
the first part was implicitly aborted.  The change to self.foo was aborted, 
while the change to self.change_count and the indexes was not.

  Even worse, this can happen within the indexes, making the indexes
  inconsistent with themselves.

 Sorry, since I didn't understand the first part, I'm way lost here.

  Once a conflict error has occurred on any object, the rest of the
  transaction is on shaky grounds.

 Which is why the current ZODB releases intend to prevent committing a
 transaction if a conflict error occurred during the transaction.  It
 shouldn't matter to this ZODB machinery if the application suppressed
 the original conflict error(s), ZODB remembers that it raised
 ReadConflictError regardless (via the Connection._conflicts
 set-implemented-as-a-dict).

Well, you described what I'm asking for: prevent committing a transaction if 
a conflict error occurred during the transaction.  I don't believe it's 
doing that, though.  If a conflict occurs on any object (read conflict or 
write conflict), ZODB should prevent any attempt to commit even if no one 
tries to change that object a second time.

 Dieter already wrote it wink.  Zope/ZODB's use of Python's machinery
 is sometimes so far from the goals Guido had in mind that Zope is
 alone in wanting a Python change.  For example, I don't know of other
 apps where calling hasattr() can have disastrous consequences.

I think it's wrong that hasattr can have disastrous consequences.  I think 
Zope is more wrong than hasattr.  However, I don't like hasattr now because I 
had no idea that it hides errors.

 That's certainly not needed for ordinary uses, like

  if hasattr(errno, WSAEWOULDBLOCK):# Windows

 Uses of hasattr() on base Python objects and modules is safe (as I
 believe it is in *almost* all applications outside our part of the
 Python world).

In your example, imagine that errno has a __getattribute__ hook that tries to 
interface with C code.  Imagine some kind of linker error occurs and a 
meaningful error results, but hasattr swallows it.  Wouldn't you like to know 
about the linker error rather than receive a possibly wrong answer?

Shane
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: hasattr geddon

2004-07-11 Thread Shane Hathaway
On Saturday 10 July 2004 20:54 pm, Casey Duncan wrote:
 Shane Hathaway [EMAIL PROTECTED] wrote:
  Why are
  we fixing hasattr and bare excepts when the real problem is ZODB?

 Well, the real problem is *probably* people, but I digress...

http://c2.com/cgi/wiki?TheProcessIsTheProblem ;-)

 Still, swallowing read conflicts can lead to bugs, even in read-only
 tranactions. Consider the following function:

 def getStuffFrom(ob, default):
 if hasattr(ob, 'stuff'):
 return ob.stuff
 return default

 I have seen this idiom about in some zope application/framework code. If
 the hasattr results in a read conflict, then it returns false and
 default is returned by the function even if ob does have the attribute.
 In the example I saw (in DCWorkflow) this would result in the wrong UI
 elements being displayed for ob mysteriously. Reloading the page would
 likely fix the problem, baffling both the user and app developer, who
 now both think Zope is playing with their minds.

The change to ZODB that I'm suggesting would fix this.  The suppressed 
conflict error would still mark the transaction as uncommitable.  When the 
publisher calls commit() (which it does on every request even if nothing 
changed), it would result in a ConflictError, then the publisher would call 
abort() and the transaction would be retried.  Users would never see results 
from a transaction that had any conflict.

Shane
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: hasattr geddon

2004-07-11 Thread Chris McDonough
FWIW, as far as I understand the prevent commit on conflict behavior
*is* currently the behavior for caught ReadConflictErrors.  The only
time ZODB doesn't exhibit this behavior is *during the commit of a
transaction*.  If a commit is attempted at this point and a conflict
error is caught unintentionally, the following happens (as explained by
Tim at http://mail.zope.org/pipermail/zodb-dev/2004-May/007404.html :

- the attempt to commit fails, raising a write conflict error

- the write conflict error is unintentionally caught (whether by hasattr
or a bare except or by whatever)

- a new transaction begins implicitly (as if get_transaction().abort()
or get_transaction().begin() were called)

- ... whatever happens after this happens ..

Since it's extremely unlkely for a write conflict error to be caught
unintentionally in Zope (it would require an application to perform a
get_transaction().commit() going outside the boundaries of the default
transaction manager), it's an obscure failure mode.

If the transaction was not put into a committable state after a caught
exception error, there would be no problem.  But to be fair, I don't
think there is much of a problem right now, as very few Zope apps manage
their own transactions (except for the Catalog... gulp ;-).  But it
would be nice if ZODB didn't do the implicit get_transaction().abort()
after a write conflict error and left the transaction in an
uncommittable state instead... or at least had an option to do this.

- C


On Sun, 2004-07-11 at 19:47, Shane Hathaway wrote:
 On Saturday 10 July 2004 20:54 pm, Casey Duncan wrote:
  Shane Hathaway [EMAIL PROTECTED] wrote:
   Why are
   we fixing hasattr and bare excepts when the real problem is ZODB?
 
  Well, the real problem is *probably* people, but I digress...
 
 http://c2.com/cgi/wiki?TheProcessIsTheProblem ;-)
 
  Still, swallowing read conflicts can lead to bugs, even in read-only
  tranactions. Consider the following function:
 
  def getStuffFrom(ob, default):
  if hasattr(ob, 'stuff'):
  return ob.stuff
  return default
 
  I have seen this idiom about in some zope application/framework code. If
  the hasattr results in a read conflict, then it returns false and
  default is returned by the function even if ob does have the attribute.
  In the example I saw (in DCWorkflow) this would result in the wrong UI
  elements being displayed for ob mysteriously. Reloading the page would
  likely fix the problem, baffling both the user and app developer, who
  now both think Zope is playing with their minds.
 
 The change to ZODB that I'm suggesting would fix this.  The suppressed 
 conflict error would still mark the transaction as uncommitable.  When the 
 publisher calls commit() (which it does on every request even if nothing 
 changed), it would result in a ConflictError, then the publisher would call 
 abort() and the transaction would be retried.  Users would never see results 
 from a transaction that had any conflict.
 
 Shane
 ___
 Zope-Dev maillist  -  [EMAIL PROTECTED]
 http://mail.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists - 
  http://mail.zope.org/mailman/listinfo/zope-announce
  http://mail.zope.org/mailman/listinfo/zope )
 

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: hasattr geddon

2004-07-10 Thread Shane Hathaway
On Friday 09 July 2004 11:50 am, Casey Duncan wrote:
 On Fri, 9 Jul 2004 16:22:17 +0200
 Dieter Maurer [EMAIL PROTECTED] wrote:
 [..]

  The hasattr replacement in Python's __builtin__ could look like:
 
  _marker = []
  def hasattr(obj, attr):
  return getattr(obj, attr, _marker) is not _marker

Hmm.  I just heard about this hasattr geddon.  hasattr is *good*.  Why are 
we fixing hasattr and bare excepts when the real problem is ZODB?  ZODB 
should *not* be sensitive to the way the application handles ConflictErrors.  
When a ConflictError (or ReadConflictError) occurs, the transaction should 
fall into an uncommitable state.  From this state, you can only abort the 
transaction; any attempts to write an object or commit cause another 
ConflictError.  Then, only code that can guarantee that the attempted 
transaction is complete should actually abort the transaction, and 
fortunately ZPublisher fits that role.  Today, the abort is implicit, and 
that's the mistake that has caused us to litter the code with knowledge of 
ConflictErrors.  With the uncommitable state, it would not matter if the 
application swallows ConflictErrors.

That said, I was surprised to discover that Python 2.3 implements hasattr this 
way (from bltinmodule.c):

v = PyObject_GetAttr(v, name);
if (v == NULL) {
PyErr_Clear();
Py_INCREF(Py_False);
return Py_False;
}
Py_DECREF(v);
Py_INCREF(Py_True);
return Py_True;

It should not swallow all errors, especially now that descriptors make 
computed attributes quite common.  getattr() only recently started catching 
only AttributeErrors, but apparently hasattr is lagging behind.  I suggest 
the consistency between getattr and hasattr should be fixed in Python, not 
Zope.

Shane
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: hasattr geddon

2004-07-10 Thread Lennart Regebro
Shane Hathaway wrote:
On Friday 09 July 2004 11:50 am, Casey Duncan wrote:
On Fri, 9 Jul 2004 16:22:17 +0200
Dieter Maurer [EMAIL PROTECTED] wrote:
[..]

The hasattr replacement in Python's __builtin__ could look like:
   _marker = []
   def hasattr(obj, attr):
return getattr(obj, attr, _marker) is not _marker

Hmm.  I just heard about this hasattr geddon.  hasattr is *good*.  Why are 
we fixing hasattr and bare excepts when the real problem is ZODB?  ZODB 
should *not* be sensitive to the way the application handles ConflictErrors.  
When a ConflictError (or ReadConflictError) occurs, the transaction should 
fall into an uncommitable state.  From this state, you can only abort the 
transaction; any attempts to write an object or commit cause another 
ConflictError.  Then, only code that can guarantee that the attempted 
transaction is complete should actually abort the transaction, and 
fortunately ZPublisher fits that role.  Today, the abort is implicit, and 
that's the mistake that has caused us to litter the code with knowledge of 
ConflictErrors.  With the uncommitable state, it would not matter if the 
application swallows ConflictErrors.

That said, I was surprised to discover that Python 2.3 implements hasattr this 
way (from bltinmodule.c):

v = PyObject_GetAttr(v, name);
if (v == NULL) {
PyErr_Clear();
Py_INCREF(Py_False);
return Py_False;
}
Py_DECREF(v);
Py_INCREF(Py_True);
return Py_True;
It should not swallow all errors, especially now that descriptors make 
computed attributes quite common.  getattr() only recently started catching 
only AttributeErrors, but apparently hasattr is lagging behind.  I suggest 
the consistency between getattr and hasattr should be fixed in Python, not 
Zope.
Guido rejected that idea 18 months ago. I think he is wrong, and after 
som discussion on #python-dev today (which practically enough was a 
python bug day) there seemed to be some support for this there too. It 
might be worth raising this on the python-dev mailing list. The main 
reason for not doing this change seems to be that it would break too 
much code. Here we have a case where lot's of code IS broken and will be 
fixed. ;)

If this doesn't happen in Python 2.4, I think Zope patching this is the 
best solution. It is natural to use hasattr, and it will be very hard to 
get the whole zope-community to stop using it.

https://sourceforge.net/tracker/index.php?func=detailaid=504714group_id=5470atid=305470
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: hasattr geddon

2004-07-10 Thread Tim Peters
[Shane Hathaway]
 Hmm.  I just heard about this hasattr geddon.  hasattr is *good*.  Why are
 we fixing hasattr and bare excepts

Well, bare excepts are generally bad news in any application -- they
almost always turn out to catch more than the author intended
(including things like SystemExit and KeyboardInterrupt, which are
rarely intended to be swallowed).

 when the real problem is ZODB?  ZODB
 should *not* be sensitive to the way the application handles ConflictErrors.
 When a ConflictError (or ReadConflictError) occurs, the transaction should
 fall into an uncommitable state.  From this state, you can only abort the
 transaction; any attempts to write an object or commit cause another
 ConflictError.

AFAIK, that is the way the current releases of ZODB work.  I'm not
sure what write an object means there, though.  I think I know what
commit means.  The logic in Connection.commit() raises
ReadConflictError upon an attempt to commit an object that previously
suffered a conflict during the transaction.  How does that differ from
what you want?

If you know of a case where this doesn't work, please open a Collector
report.  I can *believe* there's a problem -- I just don't know of one
(intended preconditions and postconditions aren't documented, and the
implementation has grown so many clever(?) special cases I'm rarely
100% sure of what it's trying to accomplish).

 Then, only code that can guarantee that the attempted
 transaction is complete should actually abort the transaction, and
 fortunately ZPublisher fits that role.  Today, the abort is implicit, and
 that's the mistake that has caused us to litter the code with knowledge of
 ConflictErrors.  With the uncommitable state, it would not matter if the
 application swallows ConflictErrors.
 
 That said, I was surprised to discover that Python 2.3 implements hasattr this
 way (from bltinmodule.c):
 
v = PyObject_GetAttr(v, name);
if (v == NULL) {
PyErr_Clear();
Py_INCREF(Py_False);
return Py_False;
}
Py_DECREF(v);
Py_INCREF(Py_True);
return Py_True;

hasattr has always been like that (that exact code, or a workalike
spelling), and that's always how hasattr has been documented to work
(the oldest docs I have handy are for Python 1.4).

 It should not swallow all errors,

Guido doesn't agree; hoping that Python will change here is probably
futile (Guido has said several times that hasattr() should never raise
an exception, and that matches how it's always been documented and
implemented).

 especially now that descriptors make computed attributes quite common.
 getattr() only recently started catching only AttributeErrors, but apparently
 hasattr is lagging behind.

The implementation of 2-argument getattr() didn't match the docs until
Python 2.2, when someone finally noticed that the implementation
didn't match the docs.  Then the getattr() implementation was
repaired, to match the docs.  The docs have never been clear (and
still aren't, IMO) about what 3-argument getattr() does with
exceptions.  I think the current implementation is the most reasonable
one, with 3-argument getattr()  propagating any exceptions the
2-argument form would have propagated, except for AttributeError --
but the docs don't really say that's what 3-arg getattr() does.

 I suggest the consistency between getattr and hasattr should be fixed in Python,
 not Zope.

They weren't intended to treat exceptions the same way, so an argument
based on inconsistency in that area won't make progress.  As I said
recently in zodb-dev,

Zope and ZODB do incredibly complex stuff as side effects in what Guido
surely thought of as tiny hooks.  He had in mind that hasattr()
might look for
a key in a dict or list, and suppress a harmless KeyError or IndexError,
not start World War III and then send an endless sequence of Terminators
back in time to change the outcome wink.

BTW, there's a ton of existing code that relies on hasattr()
suppressing things like KeyError and IndexError, so I doubt Python
would change here even if Guido agreed hasattr() was wrong to suppress
all exceptions.  But that debate belongs on python-dev.
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: hasattr geddon

2004-07-10 Thread Shane Hathaway
On Saturday 10 July 2004 11:41 am, Tim Peters wrote:
 [Shane Hathaway]
  when the real problem is ZODB?  ZODB
  should *not* be sensitive to the way the application handles
  ConflictErrors. When a ConflictError (or ReadConflictError) occurs, the
  transaction should fall into an uncommitable state.  From this state,
  you can only abort the transaction; any attempts to write an object or
  commit cause another ConflictError.

 AFAIK, that is the way the current releases of ZODB work.  I'm not
 sure what write an object means there, though.  I think I know what
 commit means.  The logic in Connection.commit() raises
 ReadConflictError upon an attempt to commit an object that previously
 suffered a conflict during the transaction.  How does that differ from
 what you want?

Here is what often happens in Zope:

def setFoo(self, value):
  try:
self.foo = value
  except:
LOG(Oops, ERROR, Some error happened, error=sys.exc_info())
  self.change_count += 1
  self.get_indexes().update(self)

Some piece of code has a legitimate reason to catch but log all exceptions.  
Some other piece of code updates indexes.  The database has now committed a 
partial transaction.  Even worse, this can happen within the indexes, making 
the indexes inconsistent with themselves.  Once a conflict error has occurred 
on any object, the rest of the transaction is on shaky grounds.

 Guido doesn't agree; hoping that Python will change here is probably
 futile (Guido has said several times that hasattr() should never raise
 an exception, and that matches how it's always been documented and
 implemented).

That's unfortunate.  Now I'm inclined to never use hasattr again.  Can we 
petition for hasattr2 then? :-)

Shane
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Re: hasattr geddon

2004-07-10 Thread Tim Peters
[Shane Hathaway]
 Here is what often happens in Zope:

 def setFoo(self, value):
  try:
self.foo = value
  except:
LOG(Oops, ERROR, Some error happened, error=sys.exc_info())
  self.change_count += 1
  self.get_indexes().update(self)

 Some piece of code has a legitimate reason to catch but log all exceptions.
 Some other piece of code updates indexes.  The database has now committed a
 partial transaction.

I need more words -- I don't see a commit() in that code, and I don't
know what partial transaction might mean (in ZODB you can commit a
transaction, or abort it -- there's no facility I know of for a
partial commit (or a partial abort, for that matter)).

Is a commit() hiding inside the update() call?  If so, and we're
assuming the bare except swallowed a ReadConflictError, then I have
to repeat:

 The logic in Connection.commit() raises ReadConflictError upon an attempt to
 commit an object that previously suffered a conflict during the transaction.
 How does that differ from what you want?

 Even worse, this can happen within the indexes, making the indexes
 inconsistent with themselves.

Sorry, since I didn't understand the first part, I'm way lost here.

 Once a conflict error has occurred on any object, the rest of the transaction is
 on shaky grounds.

Which is why the current ZODB releases intend to prevent committing a
transaction if a conflict error occurred during the transaction.  It
shouldn't matter to this ZODB machinery if the application suppressed
the original conflict error(s), ZODB remembers that it raised
ReadConflictError regardless (via the Connection._conflicts
set-implemented-as-a-dict).

Hmm.  The lack of design documentation is frustrating.  Staring at the
code, it could be that  ZODB only prevents a transaction commit if the
transaction tries to *commit* at least one object that suffered a
conflict error,  I don't know ... if you have a specific case in mind,
please open a Collector report about it.

 That's unfortunate.  Now I'm inclined to never use hasattr again.  Can we
 petition for hasattr2 then? :-)

Dieter already wrote it wink.  Zope/ZODB's use of Python's machinery
is sometimes so far from the goals Guido had in mind that Zope is
alone in wanting a Python change.  For example, I don't know of other
apps where calling hasattr() can have disastrous consequences.  If so,
there's slim constituency for adding another variant of hasattr to the
core language.  Does a thing have attribute so-and-so? is such a
mind-numbingly complex question in Zope that I would indeed never use
hasattr() in Zope/ZODB code to query general objects.  Parts of ZODB
3.3 have been rewritten to use this instead when querying general
objects:

_marker = object()

# The point of this is to avoid hiding exceptions (which the builtin
# hasattr() does).
def myhasattr(obj, attr):
return getattr(obj, attr, _marker) is not _marker

That's certainly not needed for ordinary uses, like

 if hasattr(errno, WSAEWOULDBLOCK):# Windows

Uses of hasattr() on base Python objects and modules is safe (as I
believe it is in *almost* all applications outside our part of the
Python world).
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Re: hasattr geddon

2004-07-09 Thread Casey Duncan
On Fri, 9 Jul 2004 16:22:17 +0200
Dieter Maurer [EMAIL PROTECTED] wrote:
[..]
 The hasattr replacement in Python's __builtin__ could look like:
 
 _marker = []
 def hasattr(obj, attr):
   return getattr(obj, attr, _marker) is not _marker
 
 
 Opinions?

+1

Another advantage to a global patch is that 3rd party products (such as
CMF) would benefit automatically. I noticed yesterday some hasattr calls
in DCWorkflow that could cause inaccurate results if the hasattr
returned false due to a ConflictError.

It might be worth profiling this and an alternate implementation in C to
see if the latter is worthwhile. I can't imagine the C implementation
would be all that complex. That would dispense with any performance
disadvantage of the patch.

-Casey

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )