Re: [Zope] ZODB.POSException.ReadConflictError
On Tue, 2005-05-03 at 09:18 -0700, [EMAIL PROTECTED] wrote:
I cc'ed Tim Peters on this, perhaps to his chagrin, but he seemed to be
interested in this. ;-)
> Thank you for that reference (I have an older dead trees version). Now
> I understand that these errors happen, but what I don't understand is
> why my users see them ("The requests which experience conflict errors
> will be retried automatically by Zope, and the end user should *never*
> see one." emphasis in original).
Yes. Although that's an intent and not a promise.
> Is it simply a matter of too much traffic for the server? This is a
> dual 2.3 GHz box, 2G of RAM, 10,000 RPM SCSI drives (two, mirrored as 1
> logical RAID 1 drive), serving 8,000 members (now about 1,000 new
> members a month, hence the urgency), 80,000 visits per month, 400,000
> page views per month. Is it time to dip my toe into ZEO?
ZEO may help. But may not help here, especially if you put your session
data in it. The main reason for read conflict errors are that a
particular transaction is "slow" and that other simultaneous
transactions come along and modify the data which the slow transaction
is trying to read. When the slow transaction figures this out, it bails
out. ZEO actually just makes it slower because the data involved in a
transaction typically needs to be encoded and decoded for travel across
a network. That said, if you have multiple appservers, and you keep
your session configuration as the default, each appserver will have its
own (non-ZEO)_ session database, and thus will presumably be less prone
to conflict due to the disbursement of sessions across multiple
machines. But you'll need to keep "session affinity" (every subsequent
request from a user with a particular browser id needs to go to the same
appserver). I use pound for this.
You should try to determine a pattern to the errors your users are
seeing (many simultaneous requests to a specific resource). This may be
better solved by changing application code to do fewer session accesses
or might be solveable via changing sessioning configuration.
See http://www.plope.com/Members/dunny/conflicts/view for an idea about
what sort of conflict rates to expect under torture situations.
Apparently your configuration has managed to torture sessions more than
our torture tests, however, because I've personally not managed to have
Zope try more than three times to complete a request that encounters a
sessioning conflict during this type of testing in the "default"
configuration.
- C
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZODB.POSException.ReadConflictError
I have recently upgraded the server (see previous message about my current box) and that definitely has had a positive effect. I've also been tweaking the caches, to get the most out of what I have. The instances of these errors rearing their ugly heads are definitely down, but 10% error rate on 1,000 registrations per month still means a lot of me registering members manually. I'll explore the member registration process on the Plone list. Thanks for all your help here. Marshall On May 3, 2005, at 9:17 AM, Tim Peters wrote: ... [Tim Peters] Conflict errors are certainly normal, but IIUC the OP is seeing site errors, not just log msgs ... [Chris McDonough] This is true. I misunderstood the OP inasmuch as I thought he was seeing the error message in his event log, which would be normal It is certainly the intent that end users should never see this error message in a browser, but it's not a promise as it depends how the developer's code is written. Zope will retry a request three times due to conflict errors before it gives up and presents the conflict-caused error to the user. So maybe it's really a Plone question (as in, is Plone doing something here for member registration that makes read conflicts unusually likely?); or maybe the OP's box is overburdened, so that transactions take much longer than "normal" (e.g., too little RAM, whatever); or ... It would at least be interesting to know how often conflict errors are just _logged_ (as opposed to surviving long enough to show up to the end user). ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZODB.POSException.ReadConflictError
Chris,
Thank you for that reference (I have an older dead trees version). Now
I understand that these errors happen, but what I don't understand is
why my users see them ("The requests which experience conflict errors
will be retried automatically by Zope, and the end user should *never*
see one." emphasis in original).
Is it simply a matter of too much traffic for the server? This is a
dual 2.3 GHz box, 2G of RAM, 10,000 RPM SCSI drives (two, mirrored as 1
logical RAID 1 drive), serving 8,000 members (now about 1,000 new
members a month, hence the urgency), 80,000 visits per month, 400,000
page views per month. Is it time to dip my toe into ZEO?
Marshall
P.S. To keep up-to-date, be sure to subscribe to the LiveModern News at
http://www.livemodern.com/prefs_mynewsletters_form .
LiveModern - Making Modernism Affordable
n: Marshall Mayer
e: [EMAIL PROTECTED]
w: http://www.livemodern.com
p: 415.462.0561
LiveModern.com - http://www.livemodern.com
On May 3, 2005, at 8:14 AM, Chris McDonough wrote:
This is actually normal. See the end of
http://www.plope.com/Books/2_7Edition/Sessions.stx ("Conflict Errors")
for a short explanation.
On Tue, 2005-05-03 at 08:00 -0700, [EMAIL PROTECTED] wrote:
I'm getting a ZODB.POSException.ReadConflictError on my Plone 2.0.5
site (Zope 2.7.3, Python 2.3.4 on Mac OS X Server 10.3.9). It happens
most frequently when members try to register on my site (about 10% of
the time), but occasionally happens when other pages are accessed (no
pattern). Alternatively, a blank page is returned. No messages appear
in the Zope or Plone error log.
I've reproduced the error messages below, all of which resulted from a
member registration. I received 7 error reports were from the past two
days, with two types of errors. Of the two types of errors, the "oid
0x07e8, class BTrees.IOBTree.IOBucket" error was seen 5 times and the
"oid 0x08, class Products.Transience.Transience.Length2" error was
seen
twice.
Is there something wrong with my installation or is there something
wrong with my data.fs file? How do I correct either?
Marshall
--
TYPE 1 ERROR
Site Error
An error was encountered while publishing this resource.
ZODB.POSException.ReadConflictError
Sorry, a site error occurred.
Traceback (innermost last):
* Module ZPublisher.Publish, line 175, in publish_module_standard
* Module Products.PlacelessTranslationService.PatchStringIO, line
51, in new_publish
* Module ZPublisher.Publish, line 149, in publish
* Module Products.PlacelessTranslationService.PatchStringIO, line
51, in new_publish
* Module ZPublisher.Publish, line 149, in publish
* Module Products.PlacelessTranslationService.PatchStringIO, line
51, in new_publish
* Module ZPublisher.Publish, line 149, in publish
* Module Products.PlacelessTranslationService.PatchStringIO, line
51, in new_publish
* Module ZPublisher.Publish, line 139, in publish
* Module Zope.App.startup, line 163, in zpublisher_exception_hook
* Module ZPublisher.Publish, line 101, in publish
* Module ZPublisher.mapply, line 88, in mapply
* Module ZPublisher.Publish, line 39, in call_object
* Module Products.CMFFormController.ControllerPageTemplate, line
80, in __call__
* Module Products.CMFFormController.BaseControllerPageTemplate,
line 39, in _call
* Module Products.CMFFormController.ControllerBase, line 243, in
getNext
* Module Products.CMFFormController.Actions.TraverseTo, line 36,
in
__call__
* Module ZPublisher.mapply, line 88, in mapply
* Module ZPublisher.Publish, line 39, in call_object
* Module Products.CMFFormController.FSControllerPythonScript,
line
102, in __call__
* Module Products.CMFFormController.FormController, line 354, in
validate
* Module ZPublisher.BaseRequest, line 156, in items
* Module ZPublisher.HTTPRequest, line 1178, in get
* Module Products.Sessions.SessionDataManager, line 94, in
getSessionData
* Module Products.Sessions.SessionDataManager, line 181, in
_getSessionDataObject
* Module Products.Transience.Transience, line 840, in
new_or_existing
* Module Products.Transience.Transience, line 487, in get
* Module Products.Transience.Transience, line 293, in _move_item
* Module ZODB.Connection, line 599, in setstate
* Module ZODB.Connection, line 628, in _is_invalidated
ReadConflictError: database read conflict error (oid 0x0f81, class
BTrees.IOBTree.IOBucket)
--
TYPE 2 ERROR
Site Error
An error was encountered while publishing this resource.
ZODB.POSException.ReadConflictError
Sorry, a site error occurred.
Traceback (innermost last):
* Module ZPublisher.Publish, line 175, in publish_module_standard
* Module Pr
Re: [Zope] ZODB.POSException.ReadConflictError
... [Tim Peters] >> Conflict errors are certainly normal, but IIUC the OP is seeing site >> errors, not just log msgs ... [Chris McDonough] > This is true. I misunderstood the OP inasmuch as I thought he was > seeing the error message in his event log, which would be normal It is > certainly the intent that end users should never see this error message > in a browser, but it's not a promise as it depends how the developer's > code is written. Zope will retry a request three times due to conflict > errors before it gives up and presents the conflict-caused error to the > user. So maybe it's really a Plone question (as in, is Plone doing something here for member registration that makes read conflicts unusually likely?); or maybe the OP's box is overburdened, so that transactions take much longer than "normal" (e.g., too little RAM, whatever); or ... It would at least be interesting to know how often conflict errors are just _logged_ (as opposed to surviving long enough to show up to the end user). ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZODB.POSException.ReadConflictError
On Tue, 2005-05-03 at 11:58 -0400, Tim Peters wrote:
> [Chris McDonough]
> > This is actually normal. See the end of
> > http://www.plope.com/Books/2_7Edition/Sessions.stx ("Conflict Errors")
> > for a short explanation.
>
> Conflict errors are certainly normal, but IIUC the OP is seeing site
> errors, not just log msgs. The text referenced also says:
>
> The requests which experience conflict errors will be retried
> automatically by Zope, and the end user should never see one.
>
> I'm not clear on why it says that, since Zope gives up after a few
> tries (i.e., "never" seems an awfully strong promise, right?).
This is true. I misunderstood the OP inasmuch as I thought he was
seeing the error message in his event log, which would be normal It is
certainly the intent that end users should never see this error message
in a browser, but it's not a promise as it depends how the developer's
code is written. Zope will retry a request three times due to conflict
errors before it gives up and presents the conflict-caused error to the
user.
- C
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZODB.POSException.ReadConflictError
[Chris McDonough]
> This is actually normal. See the end of
> http://www.plope.com/Books/2_7Edition/Sessions.stx ("Conflict Errors")
> for a short explanation.
Conflict errors are certainly normal, but IIUC the OP is seeing site
errors, not just log msgs. The text referenced also says:
The requests which experience conflict errors will be retried
automatically by Zope, and the end user should never see one.
I'm not clear on why it says that, since Zope gives up after a few
tries (i.e., "never" seems an awfully strong promise, right?).
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZODB.POSException.ReadConflictError
This is actually normal. See the end of
http://www.plope.com/Books/2_7Edition/Sessions.stx ("Conflict Errors")
for a short explanation.
On Tue, 2005-05-03 at 08:00 -0700, [EMAIL PROTECTED] wrote:
> I'm getting a ZODB.POSException.ReadConflictError on my Plone 2.0.5
> site (Zope 2.7.3, Python 2.3.4 on Mac OS X Server 10.3.9). It happens
> most frequently when members try to register on my site (about 10% of
> the time), but occasionally happens when other pages are accessed (no
> pattern). Alternatively, a blank page is returned. No messages appear
> in the Zope or Plone error log.
>
> I've reproduced the error messages below, all of which resulted from a
> member registration. I received 7 error reports were from the past two
> days, with two types of errors. Of the two types of errors, the "oid
> 0x07e8, class BTrees.IOBTree.IOBucket" error was seen 5 times and the
> "oid 0x08, class Products.Transience.Transience.Length2" error was seen
> twice.
>
> Is there something wrong with my installation or is there something
> wrong with my data.fs file? How do I correct either?
>
> Marshall
>
> --
>
> TYPE 1 ERROR
>
> Site Error
>
> An error was encountered while publishing this resource.
>
> ZODB.POSException.ReadConflictError
> Sorry, a site error occurred.
>
> Traceback (innermost last):
>
> * Module ZPublisher.Publish, line 175, in publish_module_standard
> * Module Products.PlacelessTranslationService.PatchStringIO, line
> 51, in new_publish
> * Module ZPublisher.Publish, line 149, in publish
> * Module Products.PlacelessTranslationService.PatchStringIO, line
> 51, in new_publish
> * Module ZPublisher.Publish, line 149, in publish
> * Module Products.PlacelessTranslationService.PatchStringIO, line
> 51, in new_publish
> * Module ZPublisher.Publish, line 149, in publish
> * Module Products.PlacelessTranslationService.PatchStringIO, line
> 51, in new_publish
> * Module ZPublisher.Publish, line 139, in publish
> * Module Zope.App.startup, line 163, in zpublisher_exception_hook
> * Module ZPublisher.Publish, line 101, in publish
> * Module ZPublisher.mapply, line 88, in mapply
> * Module ZPublisher.Publish, line 39, in call_object
> * Module Products.CMFFormController.ControllerPageTemplate, line
> 80, in __call__
> * Module Products.CMFFormController.BaseControllerPageTemplate,
> line 39, in _call
> * Module Products.CMFFormController.ControllerBase, line 243, in
> getNext
> * Module Products.CMFFormController.Actions.TraverseTo, line 36, in
> __call__
> * Module ZPublisher.mapply, line 88, in mapply
> * Module ZPublisher.Publish, line 39, in call_object
> * Module Products.CMFFormController.FSControllerPythonScript, line
> 102, in __call__
> * Module Products.CMFFormController.FormController, line 354, in
> validate
> * Module ZPublisher.BaseRequest, line 156, in items
> * Module ZPublisher.HTTPRequest, line 1178, in get
> * Module Products.Sessions.SessionDataManager, line 94, in
> getSessionData
> * Module Products.Sessions.SessionDataManager, line 181, in
> _getSessionDataObject
> * Module Products.Transience.Transience, line 840, in
> new_or_existing
> * Module Products.Transience.Transience, line 487, in get
> * Module Products.Transience.Transience, line 293, in _move_item
> * Module ZODB.Connection, line 599, in setstate
> * Module ZODB.Connection, line 628, in _is_invalidated
>
> ReadConflictError: database read conflict error (oid 0x0f81, class
> BTrees.IOBTree.IOBucket)
>
> --
>
> TYPE 2 ERROR
>
> Site Error
>
> An error was encountered while publishing this resource.
>
> ZODB.POSException.ReadConflictError
> Sorry, a site error occurred.
>
> Traceback (innermost last):
>
> * Module ZPublisher.Publish, line 175, in publish_module_standard
> * Module Products.PlacelessTranslationService.PatchStringIO, line
> 51, in new_publish
> * Module ZPublisher.Publish, line 149, in publish
> * Module Products.PlacelessTranslationService.PatchStringIO, line
> 51, in new_publish
> * Module ZPublisher.Publish, line 149, in publish
> * Module Products.PlacelessTranslationService.PatchStringIO, line
> 51, in new_publish
> * Module ZPublisher.Publish, line 149, in publish
> * Module Products.PlacelessTranslationService.PatchStringIO, line
> 51, in new_publish
> * Module ZPublisher.Publish, line 139, in publish
> * Module Zope.App.startup, line 163, in zpublisher_exception_hook
> * Module ZPublisher.Publish, line 101, in publish
> * Module ZPublisher.mapply, line 88, in mapply
> * Module ZPublisher.Publish, line 39, in call_object
> * Module Products.CMFFormController.ControllerPageTemplate, line
> 80, in __call__
> * Module Products.CMFFormController.BaseControllerPageTemplate,
> line 39, in _call
> * Module Products.CMFFormController.Control
[Zope] ZODB.POSException.ReadConflictError
I'm getting a ZODB.POSException.ReadConflictError on my Plone 2.0.5 site (Zope 2.7.3, Python 2.3.4 on Mac OS X Server 10.3.9). It happens most frequently when members try to register on my site (about 10% of the time), but occasionally happens when other pages are accessed (no pattern). Alternatively, a blank page is returned. No messages appear in the Zope or Plone error log. I've reproduced the error messages below, all of which resulted from a member registration. I received 7 error reports were from the past two days, with two types of errors. Of the two types of errors, the "oid 0x07e8, class BTrees.IOBTree.IOBucket" error was seen 5 times and the "oid 0x08, class Products.Transience.Transience.Length2" error was seen twice. Is there something wrong with my installation or is there something wrong with my data.fs file? How do I correct either? Marshall -- TYPE 1 ERROR Site Error An error was encountered while publishing this resource. ZODB.POSException.ReadConflictError Sorry, a site error occurred. Traceback (innermost last): * Module ZPublisher.Publish, line 175, in publish_module_standard * Module Products.PlacelessTranslationService.PatchStringIO, line 51, in new_publish * Module ZPublisher.Publish, line 149, in publish * Module Products.PlacelessTranslationService.PatchStringIO, line 51, in new_publish * Module ZPublisher.Publish, line 149, in publish * Module Products.PlacelessTranslationService.PatchStringIO, line 51, in new_publish * Module ZPublisher.Publish, line 149, in publish * Module Products.PlacelessTranslationService.PatchStringIO, line 51, in new_publish * Module ZPublisher.Publish, line 139, in publish * Module Zope.App.startup, line 163, in zpublisher_exception_hook * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Products.CMFFormController.ControllerPageTemplate, line 80, in __call__ * Module Products.CMFFormController.BaseControllerPageTemplate, line 39, in _call * Module Products.CMFFormController.ControllerBase, line 243, in getNext * Module Products.CMFFormController.Actions.TraverseTo, line 36, in __call__ * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Products.CMFFormController.FSControllerPythonScript, line 102, in __call__ * Module Products.CMFFormController.FormController, line 354, in validate * Module ZPublisher.BaseRequest, line 156, in items * Module ZPublisher.HTTPRequest, line 1178, in get * Module Products.Sessions.SessionDataManager, line 94, in getSessionData * Module Products.Sessions.SessionDataManager, line 181, in _getSessionDataObject * Module Products.Transience.Transience, line 840, in new_or_existing * Module Products.Transience.Transience, line 487, in get * Module Products.Transience.Transience, line 293, in _move_item * Module ZODB.Connection, line 599, in setstate * Module ZODB.Connection, line 628, in _is_invalidated ReadConflictError: database read conflict error (oid 0x0f81, class BTrees.IOBTree.IOBucket) -- TYPE 2 ERROR Site Error An error was encountered while publishing this resource. ZODB.POSException.ReadConflictError Sorry, a site error occurred. Traceback (innermost last): * Module ZPublisher.Publish, line 175, in publish_module_standard * Module Products.PlacelessTranslationService.PatchStringIO, line 51, in new_publish * Module ZPublisher.Publish, line 149, in publish * Module Products.PlacelessTranslationService.PatchStringIO, line 51, in new_publish * Module ZPublisher.Publish, line 149, in publish * Module Products.PlacelessTranslationService.PatchStringIO, line 51, in new_publish * Module ZPublisher.Publish, line 149, in publish * Module Products.PlacelessTranslationService.PatchStringIO, line 51, in new_publish * Module ZPublisher.Publish, line 139, in publish * Module Zope.App.startup, line 163, in zpublisher_exception_hook * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Products.CMFFormController.ControllerPageTemplate, line 80, in __call__ * Module Products.CMFFormController.BaseControllerPageTemplate, line 39, in _call * Module Products.CMFFormController.ControllerBase, line 243, in getNext * Module Products.CMFFormController.Actions.TraverseTo, line 36, in __call__ * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Products.CMFFormController.FSControllerPythonScript, line 102, in __call__ * Module Products.CMFFormController.FormController, line 354, in validate * Module ZPublisher.BaseRequest, line 156, in items * Module ZPublisher.HTTPRequest, lin
