Re: [Zope] Does anyone care whether we deprecate ZClasses?
+1 for deprecation. On Fri, 1 Apr 2005, Jim Fulton wrote: ZClasses are a feature that support through-the-web development. Many people have found them useful in the past, but they have some significant deficiencies, including: - They can't be managed with file-system tools, especially revision control systems like CVS and subversion. - They don't work well with Python development tools, like profilers and debugger. - They aren't being actively maintained. Most serious Zope developers stopped using them a long time ago and are frustrated that we still expend resources keeping them around. For example, the release of Zope 2.8 has been delayed by the requirement of getting ZClasses working with Zope 2.8. We could choose to deprecate ZClasses. If we deprecated them in Zope 2.8, they would still work in Zope 2.8 and Zope 2.9, but their support would be removed in Zope 2.10. Would anyone be upset if this happened? Jim ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Does anyone care whether we deprecate ZClasses?
On Sun, 3 Apr 2005, Garito wrote: My +1 for ZClasses deprecation And +1 for DTML deprecation Try to keep Zope simple -1 for DTML deprecation. It serves a different purpose that ZPT-- DTML is Logic+HTML ZPT is HTML+Logic. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] segregated users
Certainly possible, but you'll need to do a bit of application specific coding. On Tue, 5 Apr 2005, Tim Evans wrote: I apologize if this issue is something that has been discussed before, but I searched the archives to no avail. I'm evaluating zope for a project, and I have some questions regarding the extensibility of the user security model. The company I work for would like to provide documents to clients via the web, and only allow one particular client (or group of users from the same client) access to those documents. I don't want any user to be able to detect the presence of any other user. I essentially want several sites, one for each client, with a group of administrative users responsible for maintaining these sites and publishing content to all of them. What this would require is a group of administrators that can see all sites, as well as restricted users with privileges to exactly one site. I'd also like to avoid having a role for each site, as that could get ugly for almost 1000 clients. It would also be great if we could designate a user to administer only one site, so that they could only publish data to one client. I guess I want zope-level users and application-level users. Is this something that sane people do? I don't really need a step-by-step, just a yes, that is possible or a no, you're an idiot before I start digging in to try and do it. Tim ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Python2.4.1 and Zope 2.7.X
Has anyone had problems with Python 2.4.1 with the latest Zope 2.7.X versions? -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] problems rendering objects stored in the local file system
I am having trouble figuring out how to get Zope to access and render html and images files properly in the following context: In the local file system (that is, the Linux file system to be explicit) I have a collection of directories each containing an index.html file consisting of HTML and a collection of image files (*.jpg, *.gif, *.swf) referenced by the HTML. From a DTML object in the ZODB, I want to render the index.html and have it properly access the local image files. Calling the file index.html may be a bad name choice because it may be interpreted especially. In fact, what I really would like to be able to do is to reference all files relatively and allow internal directories and the like. We've been using the LocalFS product, mostly with great success, but this has got me stumped. Any suggestions? hints? pointers? -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Session Variable Error
On Thu, 28 Apr 2005, Asad Habib wrote:
I have a session variable called 'undergraduateProjectsDirected' that I
declare using the following syntax:
dtml-call REQUEST.SESSION.set('undergradProjectsDirected', [ ])
When I try to append items to this list variable, I get the following
error:
AttributeError: 'NoneType' object has no attribute 'append'
I know for a fact that this variable exists since flushing the SESSION
attribute of the REQUEST variable using dtml-var REQUEST.SESSION
yields its name and shows that it is initially empty.
I am using the following statement to append items:
dtml-call
(REQUEST.SESSION.get('undergraduateProjectsDirected')).append([student_name,
description])
The strange thing is that when I change the name of this variable to
'undergradProjectsDirected', the error does not occur anymore.
Once you get beyond this problem, you may want to think about persistence
issues. See The Zope Book for details. The list probably should be an
instance of a PersistentList and you should use the copy-out/copy-in
technique described in the API and in the Developers Guide.
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Python 2.4.X
Are there any problems using Python 2.4.X with Zope 2.7.X? -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] question about login
You might take a look at the CookieCrumbler product. It does pretty much what you seem to want manages much of the hard stuff. On Tue, 3 May 2005, Phillip Hutchings wrote: On 03/05/05, u1207440 [EMAIL PROTECTED] wrote: I have created a login form an a few users in acl_users, both exists in a ordered folder called conference. My login Form post to a login dtml method and login dtml method redirect me to another page. Even though the user exists and username and password is correct, somehow zope won't redirect me. The login query window pops out three time and in the end it denied me You are not allowed to access '' in this context. Can someone help me, is there any thing I have to set?? should I move all the files into the root?? It sounds like you're calling a method that's either restricted or set to a non-anonymous access level. What product are you using to allow you to use forms to log in? Can you post the DTML code? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] question about login
Another option is to look at the other User Folder products which do things differently. Alternatively, you could modify the CookieCrumbler product to make it fit your model. On Tue, 3 May 2005, Phillip Hutchings wrote: On 03/05/05, Dennis Allison [EMAIL PROTECTED] wrote: You might take a look at the CookieCrumbler product. It does pretty much what you seem to want manages much of the hard stuff. I was getting there ;) Though I don't really appreciate the way CookieCrumbler stores the passwords... -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO and session variables.
I am just setting up a multi-headed distributed system with several Zope heads all communications with a ZEO backend. The zopes can be on the same machine or on different machines connected through a high speed backbone. In our system, all zopes are frontended with pound. zope1 | zope2- ZEO-zope3 | zope4 We make heavy use of session variables. Normally, with a single Zope session variables are stored in a temporary folder, but, as such, they are are not shared between the zope heads. In the multiheaded system, the session variables need to be shared and therefore must be managed by ZEO. What is the recommended best practice for this? -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: ZEO and session variables.
Hi Paul, Thank you. And, yes, I saw Chris' posting, but I don't think that's the right solution for us. A fast shared session variable store seems to me to be a better way to go and a whole lot more fail safe. On Tue, 3 May 2005, Paul Winkler wrote: On Tue, May 03, 2005 at 05:14:56PM -0400, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dennis Allison wrote: I am just setting up a multi-headed distributed system with several Zope heads all communications with a ZEO backend. The zopes can be on the same machine or on different machines connected through a high speed backbone. In our system, all zopes are frontended with pound. zope1 | zope2- ZEO-zope3 | zope4 We make heavy use of session variables. Normally, with a single Zope session variables are stored in a temporary folder, but, as such, they are are not shared between the zope heads. In the multiheaded system, the session variables need to be shared and therefore must be managed by ZEO. What is the recommended best practice for this? Richard Jones recently released a SQL-based sessioning implementation: http://www.zope.org/Members/richard/ZSQLSessionDataManager Or you could look through today's messages to this list, where I recall Chris McDonough mentioned using pound to force sessions to stick to a ZEO client. That's appealingly simple; the obvious drawback is that the session would be lost if the ZEO client restarts. Presumably details can be found in the pound docs. -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: Zope 2.75 tgz distribution
I've built a working 2.7.6 which, while not fully tested, seems to be fine. What symptoms have you encountered. On Tue, 3 May 2005, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 [EMAIL PROTECTED] wrote: Er... is something missing from the Zope 2.7.6 tgz on zope.org? http://www.zope.org/Products/Zope/2.7.6/Zope-2.7.6-final.tgz Can someone else confirm that there might be a wee problem? Looks fine from here (Firefox 1.0.2 on Ubuntu Hoary handing off to File-Roller). Also, I just built a sandbox today from the download, using wget. Tres. - -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCeCNhGqWXf00rNCgRAksUAJ0cbLArHKgogBkcrAVVioelSmaAtQCfSgPB ooPRN/qXIJ+zW+13kzavjjs= =leq1 -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO and session variables.
I am hoping someone with more experience that I with multi-headed
Zope/ZEO installations can point out my problem.
I am using with Zope 2.7.6-final and Python 2.4.1. Yes, I know I am
jumping the gun with 2.4.1 but I expect that it is not the source of the
problem. Hardware is an Athlon dual processor CPU running Linux (RH 7.3).
I am trying to configure multiple Zope heads with a ZEO with shared
session variables. This means that the session variables need to be in a
temporary folder associated with the ZEO instance rather than the Zope
instance. The logs seem to indicate both ZEO and the ZEO temporary
storage is properly hooked up.
My setup dies with an unhandled exception during startup with a
ConnectionStateError. Since it appeared when I moved temporary
storage from Zope to ZEO, I suspect the problem is somewhere in
that space.
The traceback appended below is truncated for some reason. I was
running the zope instance with the ``runzope`` program with output
appended to the controlling terminal.
The relevant part of the zope.conf the configuration file is:
# ZEO client storage:
#
zodb_db main
mount-point /
# ZODB cache, in number of objects
cache-size 5000
zeoclient
server 192.168.0.92:8301
storage 1
var $INSTANCE/var
# ZEO client cache, in bytes
cache-size 20MB
# Uncomment to have a persistent disk cache
client zeo1
/zeoclient
/zodb_db
#
# ZEO temporary storage
#
zodb_db temporary
zeoclient
server 192.168.0.92:8301
storage temp
name zeostorage
var $INSTANCE/var
/zeoclient
mount-point /temp_folder
container-class Products.TemporaryFolder.TemporaryContainer
/zodb_db
And the relevant portion of the zeo.conf configuration file is:
zeo
address 192.168.0.92:8301
read-only false
invalidation-queue-size 100
pid-filename $INSTANCE/var/ZEO.pid
# monitor-address PORT
# transaction-timeout SECONDS
/zeo
filestorage 1
path $INSTANCE/var/Data.fs
/filestorage
# temporary storage has to be ZEO side
%import tempstorage
temporarystorage temp
name temporary storage for sessioning
/temporarystorage
Running with this configuration, dies with an exception:
2005-05-09T23:01:43 INFO(0) ZCS:10522 ClientStorage (pid=10522) created
RW/normal for storage: '1'
--
2005-05-09T23:01:43 INFO(0)
ZEC:/opt2/zope/zinstances/leibnitz/var/c1-zeo1-0.zec ClientCache:
storage='1', size=20971520;
file[0]='/opt2/zope/zinstances/leibnitz/var/c1-zeo1-0.zec'
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Testing connection
ManagedClientConnection ('192.168.0.92', 8301)
--
2005-05-09T23:01:43 INFO(0) zrpc-conn(C):192.168.0.92:8301 received
handshake 'Z201'
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Server authentication protocol None
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Connected to storage: ('x-epaul',
8301)
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Verifying cache
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Waiting for cache verification to
finish
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Waiting for cache verification to
finish
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 endVerify finishing
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 endVerify finished
--
2005-05-09T23:01:43 INFO(0) ZODB Opening database for mounting:
'1087156928_1109278350.209647'
--
2005-05-09T23:01:43 INFO(0) ZODB Mounted database
'1087156928_1109278350.209647' at /temp_folder
--
2005-05-09T23:01:43 INFO(0) Zope Ready to handle requests
Unhandled exception in thread started by class
ZServer.PubCore.ZServerPublisher.ZServerPublisher at 0x40467d4c
Traceback (most recent call last):
File
/usr/local/src/zope/Zope2.7/lib/python/ZServer/PubCore/ZServerPublisher.py,
line 23, in __init__
response=response)
File /opt2/zope/zproducts/standard/ZopeProfiler/MonkeyPatcher.py, line
35, in __call__
return self._function(*args,**kw)
File /opt2/zope/zproducts/standard/ZopeProfiler/ZopeProfiler.py, line
335, in _profilePublishModule
_hookApplicationManager()
File /opt2/zope/zproducts/standard/ZopeProfiler/ZopeProfiler.py, line
383, in _hookApplicationManager
R._p_jar.close()
File
/usr/local/src/zope/Zope2.7/lib/python/Products/ZODBMountPoint/Mount.py,
line 210, in close
self._real_close()
File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Connection.py, line
306, in close
raise ConnectionStateError(Cannot close a connection joined to
ZODB.POSException.ConnectionStateError: Cannot close a connection joined
to a transaction
While the ZEO log shows
2005-05-09T22:46:35 INFO(0) ZD:10284 daemonizing the process
--
2005-05-09T22:46:35 INFO(0) ZD:10284 set current directory:
'/opt2/zope/zeoinstances/leibnitz'
--
2005-05-09T22:46:35 INFO(0) ZD:10284 daemon manager started
--
2005-05-09T22:46:35 INFO(0) ZD:10284 spawned process pid=10285
--
2005-05-09T22:46:36 INFO(0) RUNSVR created PID file
'/opt2/zope/zeoinstances/leibnitz/var/ZEO.pid'
--
[Zope] (no subject)
Dieter, I get the following error at startup (Zope 2.7.6-final, Python 2.3.4, ZopeProfiler 1.1 in a multi-headed Zope/ZEO configuration with shared session variables. My hypothesis is that the culprit is the ZopeProfiler and that I need to move to Version 1.4. Are there any problems using it in a Zope/ZEO environment? --- Unhandled exception in thread started by class ZServer.PubCore.ZServerPublisher.ZServerPublisher at 0x40467d1c Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZServer/PubCore/ZServerPublisher.py, line 23, in __init__ response=response) File /opt2/zope/zproducts/standard/ZopeProfiler/MonkeyPatcher.py, line 35, in __call__ return self._function(*args,**kw) File /opt2/zope/zproducts/standard/ZopeProfiler/ZopeProfiler.py, line 335, in _profilePublishModule _hookApplicationManager() File /opt2/zope/zproducts/standard/ZopeProfiler/ZopeProfiler.py, line 383, in _hookApplicationManager R._p_jar.close() File /usr/local/src/zope/Zope2.7/lib/python/Products/ZODBMountPoint/Mount.py, line 210, in close self._real_close() File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Connection.py, line 306, in close raise ConnectionStateError(Cannot close a connection joined to ZODB.POSException.ConnectionStateError: Cannot close a connection joined to a transaction ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO and session variables.
That seems to have been the problem. The ZopeProfiler version I was using
does not work with Zope2.7 although Dieter has a version (1.4) that works
with Zope2.7.3, I was not using it. I also suspect there may be a problem
with multiheaded Zope/ZEO configurations.
On Tue, 10 May 2005, Chris McDonough wrote:
It appears that something in the ZopeProfiler product is unhappy with
your configuration. Does it work if you remove this product?
On Mon, 2005-05-09 at 23:37 -0700, Dennis Allison wrote:
I am hoping someone with more experience that I with multi-headed
Zope/ZEO installations can point out my problem.
I am using with Zope 2.7.6-final and Python 2.4.1. Yes, I know I am
jumping the gun with 2.4.1 but I expect that it is not the source of the
problem. Hardware is an Athlon dual processor CPU running Linux (RH 7.3).
I am trying to configure multiple Zope heads with a ZEO with shared
session variables. This means that the session variables need to be in a
temporary folder associated with the ZEO instance rather than the Zope
instance. The logs seem to indicate both ZEO and the ZEO temporary
storage is properly hooked up.
My setup dies with an unhandled exception during startup with a
ConnectionStateError. Since it appeared when I moved temporary
storage from Zope to ZEO, I suspect the problem is somewhere in
that space.
The traceback appended below is truncated for some reason. I was
running the zope instance with the ``runzope`` program with output
appended to the controlling terminal.
The relevant part of the zope.conf the configuration file is:
# ZEO client storage:
#
zodb_db main
mount-point /
# ZODB cache, in number of objects
cache-size 5000
zeoclient
server 192.168.0.92:8301
storage 1
var $INSTANCE/var
# ZEO client cache, in bytes
cache-size 20MB
# Uncomment to have a persistent disk cache
client zeo1
/zeoclient
/zodb_db
#
# ZEO temporary storage
#
zodb_db temporary
zeoclient
server 192.168.0.92:8301
storage temp
name zeostorage
var $INSTANCE/var
/zeoclient
mount-point /temp_folder
container-class Products.TemporaryFolder.TemporaryContainer
/zodb_db
And the relevant portion of the zeo.conf configuration file is:
zeo
address 192.168.0.92:8301
read-only false
invalidation-queue-size 100
pid-filename $INSTANCE/var/ZEO.pid
# monitor-address PORT
# transaction-timeout SECONDS
/zeo
filestorage 1
path $INSTANCE/var/Data.fs
/filestorage
# temporary storage has to be ZEO side
%import tempstorage
temporarystorage temp
name temporary storage for sessioning
/temporarystorage
Running with this configuration, dies with an exception:
2005-05-09T23:01:43 INFO(0) ZCS:10522 ClientStorage (pid=10522) created
RW/normal for storage: '1'
--
2005-05-09T23:01:43 INFO(0)
ZEC:/opt2/zope/zinstances/leibnitz/var/c1-zeo1-0.zec ClientCache:
storage='1', size=20971520;
file[0]='/opt2/zope/zinstances/leibnitz/var/c1-zeo1-0.zec'
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Testing connection
ManagedClientConnection ('192.168.0.92', 8301)
--
2005-05-09T23:01:43 INFO(0) zrpc-conn(C):192.168.0.92:8301 received
handshake 'Z201'
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Server authentication protocol None
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Connected to storage: ('x-epaul',
8301)
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Verifying cache
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Waiting for cache verification to
finish
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 Waiting for cache verification to
finish
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 endVerify finishing
--
2005-05-09T23:01:43 INFO(0) ZCS:10522 endVerify finished
--
2005-05-09T23:01:43 INFO(0) ZODB Opening database for mounting:
'1087156928_1109278350.209647'
--
2005-05-09T23:01:43 INFO(0) ZODB Mounted database
'1087156928_1109278350.209647' at /temp_folder
--
2005-05-09T23:01:43 INFO(0) Zope Ready to handle requests
Unhandled exception in thread started by class
ZServer.PubCore.ZServerPublisher.ZServerPublisher at 0x40467d4c
Traceback (most recent call last):
File
/usr/local/src/zope/Zope2.7/lib/python/ZServer/PubCore/ZServerPublisher.py,
line 23, in __init__
response=response)
File /opt2/zope/zproducts/standard/ZopeProfiler/MonkeyPatcher.py, line
35, in __call__
return self._function(*args,**kw)
File /opt2/zope/zproducts/standard/ZopeProfiler/ZopeProfiler.py, line
335, in _profilePublishModule
_hookApplicationManager()
File /opt2/zope/zproducts/standard/ZopeProfiler/ZopeProfiler.py, line
383, in _hookApplicationManager
R._p_jar.close()
File
/usr/local/src/zope/Zope2.7/lib
Re: [Zope] How do I logout using DTML
Looks like your userfolder does not support a logout method as an inspection of the code would have established. In fact, the only logout method that appears to be supported is in the App manager for the ZMI. You may want to look at the code and adapt it for your purposes. On Wed, 11 May 2005, Allen Huang wrote: How do I logout current user using DTML I used dtml-call expr=acl_users.logout(REQUEST) but it didn't work and zope gave me this. Site Error An error was encountered while publishing this resource. Error Type: AttributeError Error Value: logout What did I do wrong?? - Yahoo! Mail Mobile Take Yahoo! Mail with you! Check email on your mobile phone. -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] surprising interaction between Javascript and Zope
We have a need to set a session variable concurrently with a
user submitting a form on a mouse click. I accomplish this with
a bit of JavaScript associated with the OnClick attribute of the
form:
location.replace('/setSesVar?var=namsval=xxx;frm.submit();
which does the job except that the setSesVar script is executed twice, a
fact we discovered by monitoring calls to the setSesVar script. Once with
an incorrect parameter, and secondly with a correct parameter.
There is probably some obvious explanation, but I am stumped. Setting
invalid values into session variables, even transient invalid values, can
cause problems.
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZSyncer and ZEO
Does ZSyncer have any problem with Zope/ZEO systems? I have some anecdotal evidence of problems as we migrate to a multiheaded Zope/ZEO systems architecture. Is there any difficult using ZSyncer between a Zope and a Zope/ZEO combination. Are there problems syncing between Zope 2.6.X and Zope 2.7.X systems? All the systems run ZSyncer 0.5.1. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope 2.7.2 with Ape 1.0 installed in root, fails to restart, without any usefull logging, after adding a local role
Tim, You may want to run Zope in debug mode-- bin/runzope should do it. Also, be sure you have the event.log configured and enable. -d On Wed, 25 May 2005, Tim Connor wrote: I sent roughly this email directly to Shane Hathaway, since I hadn't asked for any help on Zope, yet, so didn't know where to look. Maybe this was a better avenue? If you read this list, Shane (or Mr. Hathaway, if you prefer), respond on whichever seems more appropriate, I guess. I don't know, let me know if I'm missing some etiquette point. Anyways, I'm using Ape 1.0 set-up in the root of Zope 2.7.2 install, based on the example fs config, and that is working fine. I was having a problem where some change was making that Zope instance not even get far enough in booting to write to the event.log (set to level all). I narrowed it down to adding a local role on the security tab of the root object. I revert the .properties file and restart - it works. I make the change again and restart - it doesn't. I diff the file, and the only difference is the define-role line. The problem is I have no idea where to start even, since no errors are thrown where I know to find them. Zope just doesn't respond at all when I try to restart it, and writes nothing to the log. If anyone could tell me what I can do to at least track down the problem, if not fix it, it would be much appreciated. Thanks, Tim ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] No ZMI - panic!!!!
The ZMI is accessed by appending /manage to the site url, so if you have ports redirected properly and reach the site with http://mysite.com you access the ZMI with http://mysite.com/manage If you have forgotten your password, you need to setup an emergency user --- google for it and read The Zope Book for detailed instructions. On Wed, 25 May 2005, John Poltorak wrote: I'm trying to update a Zope site which I set up some time ago but can't get the ZMI to start up. Any ideas on what to do? When I try to acccess http://mysite:8080/ I get - Zope Site Error. How do I force a login prompt or include a login in the URL? If this is a FAQ, please tell me which Q to look under. -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: ZSyncer and ZEO
Furher investigation has shown that the failing ZSyncer target had a problem with its authentication code and that was likely the cause of the failure. We are continuing to test, but it appears everything is working. We'll continue to test but I am pretty sure this was not a ZSyncer problem. Thanks to all. -d On Wed, 25 May 2005, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dennis Allison wrote: Does ZSyncer have any problem with Zope/ZEO systems? I have some anecdotal evidence of problems as we migrate to a multiheaded Zope/ZEO systems architecture. Is there any difficult using ZSyncer between a Zope and a Zope/ZEO combination. There shouldn't be any issues there: ZSyncer runs at the application level, not the database level. Are there problems syncing between Zope 2.6.X and Zope 2.7.X systems? *That* might be problematic, although I can't think of a case offhand. All the systems run ZSyncer 0.5.1. Tres. - -- === Tres Seaver [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFClGJy+gerLs4ltQ4RAqZzAKCC1aj/6x9z5DEBcwtoY66dL33DqACgq8CY AmaeKQ6cb8R0hDyqIaLFfUw= =Hyq6 -END PGP SIGNATURE- -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: ZSyncer and ZEO
I noted that and will upgrade in this round. It's the old problem of having live systems, distributed development, and all. The usual groundrules are: only transparent changes, no risk of errors, no disruption of 24x7 use, etc. On Wed, 25 May 2005, Paul Winkler wrote: On Wed, May 25, 2005 at 09:15:32AM -0700, Dennis Allison wrote: Furher investigation has shown that the failing ZSyncer target had a problem with its authentication code and that was likely the cause of the failure. We are continuing to test, but it appears everything is working. We'll continue to test but I am pretty sure this was not a ZSyncer problem. You might be interested to know that I made substantial improvements to the error reporting and logging for ZSyncer 0.6. Problems like this are generally easier to diagnose now. -PW -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] using DOM from within Zope
A common approach is to isolate the functionality that uses the module(s) and put them in an Exernal Method or write a Product that incorporates them. These approaches give much more freedom than Python Scripts which are given limited access because of security issues. On Wed, 25 May 2005 [EMAIL PROTECTED] wrote: Maybe this is the wrong way or something, but how can I use xml.dom.minidom (or other module in its entirety) from within Zope? I've tried various stuff in the MyScriptModules Product. This same approach seems to work with the re module. Example: Products/MyScriptModules/__init__.py: from Products.PythonScripts.Utility import allow_module ... allow_module(xml) allow_module(xml.dom) allow_module(xml.dom.minidom) Python Script inside Zope: import xml.dom.minidom xmlstring=abtexttextext/bchejhejhej/c/a domtree=xml.dom.minidom.parseString(xmlstring) return domtree.toxml() Result: Username/Password question. Error Type: Unauthorized Error Value: You are not allowed to access 'toxml' in this context ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Opening .doc files in zope
On Fri, 27 May 2005, Allen Huang wrote: Is there any way to open a .doc files using dtml or script and print them?? The short answer is it depends. To make such files generally available you have to convert them to HTML so arbirary browsers can access them. You can use MS Word to do the conversion. If your Zope is running on a Windows machine that also has Word, you can manage the conversion dynamically using an External Method or a Product (which you'd have to write). Alternatively, you could store the MS Word document in it's XML form and use that to drive the rendering. If you are not interested in general availability and you users are all running Windows with Word installed, you can configure IE, I believe, to render Word documents directly. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] XML Content handling questions.
While you need to write appropriate External Methods and/or Products to use it, we have found that the ReportLab pyRXP system works well. On Sat, 28 May 2005, Andreas Jung wrote: Zope itself has no native XML support. However there are some products like ParsedXML that provide additional functionality. You might also look at Silva CMS which is provides native XML support. -aj --On 27. Mai 2005 14:53:40 -0600 Nikko Wolf [EMAIL PROTECTED] wrote: Three questions, in order of importance (to me): 1) Is it possible to validate an (uploaded) XML file against a schema (currently on the Zope server)? What, if any, extra packages are needed?I want to use this in my upload forms to alert the user if their files are invalid. 2) I also need to serve up web-based forms and allow the user to create XML content based on a user-selected schema. Something like what Chiba (http://chiba.sf.net/) is targeting. Of course, though, that isn't a Python/Zope solution. Has anyone seen anything that works? At this point, installing Tomcat and Chiba is what I presume I'll be doing in a couple of weeks 3) Is there a function built-in to Zope that allows merging XML files? (ie: all the abc within a list of user-selected files). I've got an Extension that works correctly (using the Amara/4Suite packages), so an answer here is mostly for me to learn Zope better. Thanks, Nikko. PS: I'm pretty new to Python Zope, but not web servers or other programming. Server: Fedora Core 3 Python 2.3.4 Zope 2.7.6-final Plone 2.0.5 4Suite 1.0b1 Amara 1.0b2 ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZODB Lock Errors and Storage Errors
Zope 2.7.6b3, ZEO, Python 2.4.1 (yes, I know it is not yet official but ... ZEO runs on the same machine but is accessed through an internal network ZMYSQLDA 2.0.9 MySQL-1.1.7 I have seen several Lock problems and a couple of storage error problems as I upgrade our system. Anyone seen anything like this? Andy-- I'm guessing this is MySQL related. I did not upgrade from my MySQLdb from 1.1.7. Should I have? from cvs? or is there a later release? -- 2005-06-06T13:02:40 ERROR(200) ZODB A storage error occured during object abort. This shouldn't happen. Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Transaction.py, line 412, in _commit_error j.tpc_abort(self) # This should never fail File /usr/local/src/zope/Zope2.7/lib/python/Shared/DC/ZRDB/TM.py, line 63, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-06-06T13:02:40 INFO(0) ZODB conflict error at /XXX/YYY/topic_html (3 conflicts since startup at 2005-06-04T22:49:38) -- -- 2005-06-06T13:23:18 ERROR(200) ZODB A storage error occurred during object abort. This shouldn't happen. Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Transaction.py, line 412, in _commit_error j.tpc_abort(self) # This should never fail File /usr/local/src/zope/Zope2.7/lib/python/Shared/DC/ZRDB/TM.py, line 63, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-06-06T13:23:18 INFO(0) ZODB conflict error at /AAA/BBB/topic_html (4 conflicts since startup at 2005-06-04T22:49:38) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] XMLRPC problems?
Zope 2.7.6, ZEO, Python 2.4.1 (yes, I know it is not yet official but ... ZEO runs on the same machine but is accessed through an internal network ZMYSQLDA 2.0.9 MySQL-1.1.7 I've seen a couple of problems attributable to XLRPC failures in our system after an upgrade to Zope 2.7.6. Has anyone else experienced problems? I know there were issues at one point, but I believed that they have been resolved. I checked and found no active bugs. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] XMLRPC problems?
There was a security fix for XMLRPC which was installed in 2.3.5 and 2.4.1 and elsewhere which is not fully backwards compatible. I have yet to establish whether the fix is causing problems, but that was the information I needed. On Tue, 7 Jun 2005, Andreas Jung wrote: --On 6. Juni 2005 15:14:21 -0700 Dennis Allison [EMAIL PROTECTED] wrote: Zope 2.7.6, ZEO, Python 2.4.1 (yes, I know it is not yet official but ... ZEO runs on the same machine but is accessed through an internal network ZMYSQLDA 2.0.9 MySQL-1.1.7 I've seen a couple of problems attributable to XLRPC failures in our system after an upgrade to Zope 2.7.6. a couple of problems* is a totally insufficient problem description. If you have some problems then you would like to give a description of the problem you encounter on your side...it's hard to investigate or even fix phantom problems. If you want that some of the Zope developers gets attracted to your problem you must provide reasonable evidence. Speaking for myself I won't ask bunch of times back...if there is a problem then tell us or file a bugreport but please avoid such unspecific reports. -aj -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Z SQL test in the ZMI aborts on key error
Zope 2.7.6 Python 2.3.5 MySQLDA-1.2.1c3 ZMYSQLDA-2.09b3 Site Error An error was encountered while publishing this resource. Error Type: KeyError Error Value: 'query' Troubleshooting Suggestions * This resource may be trying to reference a nonexistent object or variable 'query'. * The URL may be incorrect. * The parameters passed to this resource may be incorrect. * A resource that this resource relies on may be encountering an error. -- Error log: 2005/06/07 03:33:03.344 GMT-7 User Name (User Id) allison (allison) Request URL http://group1.agilemind.com/Scratch/simpleq/manage_test Exception Type KeyError Exception Value 'query' Traceback (innermost last): * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Shared.DC.ZRDB.DA, line 337, in manage_test * Module DocumentTemplate.DT_String, line 474, in __call__ * Module DocumentTemplate.DT_In, line 602, in renderwb * Module DocumentTemplate.DT_Var, line 219, in render KeyError: 'query' There does not seem to be a bug like this in the collectors. Any help/pointers appreciated. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Z SQL test in the ZMI aborts on key error
I've never had any problem with parameterless queries. (-: No, this is a simple case of a parameter somehow being lost in the ZMI interface. On Tue, 7 Jun 2005, Joe Bezier wrote: Hi there, By any chance are you trying to test a ZSQL method that doesn't (shouldn't) need any query parameters? For some reason (and I'm not the guy who can tell you why...) these types of tests always break - try adding '?query=' to the end of the URL. and see if that helps. Cheers, Joe. On 7 Jun 2005, at 11:41, Dennis Allison wrote: Zope 2.7.6 Python 2.3.5 MySQLDA-1.2.1c3 ZMYSQLDA-2.09b3 Site Error An error was encountered while publishing this resource. Error Type: KeyError Error Value: 'query' Troubleshooting Suggestions * This resource may be trying to reference a nonexistent object or variable 'query'. * The URL may be incorrect. * The parameters passed to this resource may be incorrect. * A resource that this resource relies on may be encountering an error. -- Error log: 2005/06/07 03:33:03.344 GMT-7 User Name (User Id) allison (allison) Request URL http://group1.agilemind.com/Scratch/simpleq/manage_test Exception Type KeyError Exception Value 'query' Traceback (innermost last): * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Shared.DC.ZRDB.DA, line 337, in manage_test * Module DocumentTemplate.DT_String, line 474, in __call__ * Module DocumentTemplate.DT_In, line 602, in renderwb * Module DocumentTemplate.DT_Var, line 219, in render KeyError: 'query' There does not seem to be a bug like this in the collectors. Any help/pointers appreciated. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] SQL query manage_test throws KeyError vith value 'query'
Failure occurs with Zope 2.7.6 final Python 2.3.5 or Phython 2.4.1 MySQLdb 1.2.1 gamma or MySQLdb 1.1.1 final ZMySQLDA 2.08 or ZMYSQLDA 2.09 Failure does not occur with Zope 2.6.4 python 2.3.5 MySQLdb 1.1.1 final ZMySQLDA 2.08 The traceback is: Time 2005/06/07 16:47:44.967 GMT-7 User Name (User Id) allison (allison) Request URL http://dev.agilemind.com/draPlay/webpagetest/manage_test Exception Type KeyError Exception Value 'query' Traceback (innermost last): * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Shared.DC.ZRDB.DA, line 337, in manage_test * Module DocumentTemplate.DT_String, line 474, in __call__ * Module DocumentTemplate.DT_In, line 602, in renderwb * Module DocumentTemplate.DT_Var, line 219, in render KeyError: 'query' Failure is highly repeatable and happens for queries of the form select * from table The REQUEST shows: form SUBMIT 'Submit Query' cookies tree-s 'eJzTyCkw5NLIKTDiClZ3hANXW3WuAmOuxEQ9AIOOB9Q' __ac_name 'allison' lazy items SESSION bound method SessionDataManager.getSessionData of SessionDataManager instance at 40ddef50 other n_ 5 REQUEST_METHOD'GET' HTTP_KEEP_ALIVE'300' SERVER_PROTOCOL'HTTP/1.1' QUERY_STRING'SUBMIT=Submit+Query' Somewhere between manage_testForm and manage_test something fails, possibly as the query goes off to be rendered through the DocumentTemplate mechanism. Right at the moment, I'm stuck. Advice, clues appreciated. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] aggreagating and merging logs under Zope 2.7.X
Is there any way to configure multi-headed Zopes running on different machines but sharing the same ZEO to log to the same file (in time order would also be nice...)? -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: aggreagating and merging logs under Zope 2.7.X
Well, I am not really ready to address the multiple concurrent writers problem across NFS and all the issues it entails. I'd somehow dismissed the SysLog solution, but I can certainly configure to use it. Thanks for the pointer. On Fri, 10 Jun 2005, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dennis Allison wrote: Is there any way to configure multi-headed Zopes running on different machines but sharing the same ZEO to log to the same file (in time order would also be nice...)? zope.conf supports logging to syslog, which would be the sanest way to do that, assuming that you can run the syslog daemon; otherwise, you have to work out the multiple concurrent writers in the same file across NFS problem, which would probably involve more research than I imagine you are contemplating. Tres. - -- === Tres Seaver +1 202-558-7113 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCqgJL+gerLs4ltQ4RArqRAJ9TkXM0EG2lcI3uKOIVFGps7ULUGgCeMVfA N4hy/9kViG9e5BDzBbj6hbg= =5dA5 -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Datetime bug in Zope-2.7.6?
Zope 2.7.6
Python 2.4.1 (and yes, I know I am jumping the gun a bit)
The problem appears to be in the parse component --
Consider a test script:
for y in range(1995,2005):
for d in (1, 3, 13, 25,):
for m in (1,4,5,6,):
dd = '%d-%02d-%02d' % (y,m,d,)
ee = '%d/%d/%d' % (m, d, y)
D = DateTime(dd)
E = DateTime(ee)
print dd, D.strftime('%Y-%m-%d')
print ee, E.strftime('%Y-%m-%d')
return printed
In all cases, each of the dates shoudl be the same. As can be seen,
for dates sourced from a string of the form
1995-04-25
the resulatant date is off by one day
1995-04-24
The problem is not seen with Zope 2.6.4 running with
Python 2.3.5.
A quick diff shows tht there are many changes between 2.6 and 2.7.
Anyone else seen this problem?
Partial Results:
1995-04-25 1995-04-24
4/25/1995 1995-04-25
1995-05-25 1995-05-24
5/25/1995 1995-05-25
1995-06-25 1995-06-24
6/25/1995 1995-06-25
--
Dennis Allison * Computer Systems Laboratory * Gates 227
* Stanford University * Stanford CA 94305
* (650) 723-9213 * (650) 723-0033 fax
* [EMAIL PROTECTED]
* [EMAIL PROTECTED]
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: datetime bug
Thanks Lupa.
I have tracked it down and created a patch which fixes the problem in
general. Seems the ISO 8601 implementation had some warts. We are
testing the patch now. When I am sure it's OK I'll submit it to the
Collector with an URGENT tag. I'll also send you a copy.
I suspect that the daylight savings time behavior is unrelated to this
problem and is likely to be a bad entry in a table.
Thanks for you helpful response. BTW if you want an alpha copy of the
patch, let me know and I'll send it to you. The more testing the better.
The patched version has the following semantics
dates and times with no timezone tag resolve to the
timezone of the host machine (Zope standard behavior)
dates and times with a trailing Z are guaranteed to be
GMT
dates and times with a trailing offset have the offset
as their time zone.
It is illegal to combine a 'Z' and an offset. (Hacked in
the code for 2.7 beta)
I think this matches the spec at
http://www.w3.org/TR/NOTE-datetime
On Sun, 12 Jun 2005, +lupa+ wrote:
Hi Dennis,
I'm responding directly to you as I'm not on the zope list. I see the
same error on Zope 2.7.6 and with 2.3.5 python, not 2.4.x. I've narrowed
the behavior down a bit more and it seems that
dd = '%d-%02d-%02d' % (y,m,d,)
D = DateTime(dd)
converts to local time, while
ee = '%d/%d/%d' % (m, d, y)
E = DateTime(ee)
converts to Mean time. I'm not absolutely sure of that, but in a quick
look, I can see that the difference between D and E changes from -5 hours
to -3 when Daylight Savings Time change comes around... which seems a bit
weird in itself. I wrote and maintain CalendarX for Plone, so this
behavior concerns me greatly and I'll be looking into it more very soon.
+lupa+
Lupa Zurven, CalendarX.org
At 12:00 PM 6/12/2005 -0400, you wrote:
From: Dennis Allison [EMAIL PROTECTED]
Precedence: list
Subject: [Zope] Datetime bug in Zope-2.7.6?
Date: Sat, 11 Jun 2005 16:54:03 -0700 (PDT)
To: zope@zope.org
Message-ID: [EMAIL PROTECTED]
Content-Type: TEXT/PLAIN; charset=US-ASCII
MIME-Version: 1.0
Message: 5
Zope 2.7.6
Python 2.4.1 (and yes, I know I am jumping the gun a bit)
The problem appears to be in the parse component --
Consider a test script:
for y in range(1995,2005):
for d in (1, 3, 13, 25,):
for m in (1,4,5,6,):
dd = '%d-%02d-%02d' % (y,m,d,)
ee = '%d/%d/%d' % (m, d, y)
D = DateTime(dd)
E = DateTime(ee)
print dd, D.strftime('%Y-%m-%d')
print ee, E.strftime('%Y-%m-%d')
return printed
In all cases, each of the dates shoudl be the same. As can be seen,
for dates sourced from a string of the form
1995-04-25
the resulatant date is off by one day
1995-04-24
The problem is not seen with Zope 2.6.4 running with
Python 2.3.5.
A quick diff shows tht there are many changes between 2.6 and 2.7.
Anyone else seen this problem?
Partial Results:
1995-04-25 1995-04-24
4/25/1995 1995-04-25
1995-05-25 1995-05-24
5/25/1995 1995-05-25
1995-06-25 1995-06-24
6/25/1995 1995-06-25
--
Dennis Allison * Computer Systems Laboratory * Gates 227
* Stanford University * Stanford CA 94305
* (650) 723-9213 * (650) 723-0033 fax
* [EMAIL PROTECTED]
* [EMAIL PROTECTED]
--
Dennis Allison * Computer Systems Laboratory * Gates 227
* Stanford University * Stanford CA 94305
* (650) 723-9213 * (650) 723-0033 fax
* [EMAIL PROTECTED]
* [EMAIL PROTECTED]
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Python Scripts
We have been seeing a number of instances where python scripts fail due to an apparent syntax error but the syntax is correct and simply storing the method restores it to functionality. Anyone else seeing this? -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope programms for databases
Take a look at http://http://products.bluedynamics.org/Products/Znolk, a wizard method which generates a wide variety of different Zope methods for maintaining databases. I don't understand your concurrent update issue. Such problems should be handled by the backend database, careful table design, and by using transactions when doing the update. On Sat, 18 Jun 2005, Ralph wrote: I'm new to Zope and made a small database application with zope (mostly DTML) to learn the idea behind Zope, but I'm not satisfied with the structure of my application. I have for each table a Form (DTML-method) with 3 buttons (new,update,delete). Each button triggers a DTML-script actionTable actionTable calls either a DTML-method to display a form (update, insert) or calling the zsql-method to delete a data record. The disadvantage with this structure is, that it is difficult to maintain programs like this. I wish to have optimistic locking but to send variables to the next DTML-method is a hell. Example: I you have a database table with the rows id,name and desc I have to send all record values through all methods after the user is triggering update to secure that there is no concurrent update. But with my design approach it's not possible. Is there someone who has a better design approach or an idea? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZMYSQLDA 2.0.9 lock problem
IE, Zope 2.7.6, Python 2.4.1, MYSQLDA 2.0.9 One of our developers is reporting a curious problem, which I have not been able to track down. It appears to be threading related, somewhere in the ZBMYSQLDB adaptor. The database connection string is bare of transaction indicators. The database is local. All tables are MyISAM which is non-transactioal. Unfortunately the user does not remember the exact sequence of events which generate the problem so I am unable to duplicate. What we do know is that the problem occurs at the beginning of a development session begun with a cold boot of the client machine and a fresh browser and it occurs once and only onece during the development session. The developer normally has two active Zope threads going--one working with the application and one working with the ZMI. He moves between them using the Open/View tab and other buttons. He reports that the first time he moves focus from the ZMI to the application, he gets the following error: --- Time 2005/06/20 07:12:03.478 GMT-7 User Name (User Id) (X) Request URL http://xxx..com/logged_in Exception Type error Exception Value release unlocked lock Traceback (innermost last): * Module ZPublisher.Publish, line 107, in publish * Module Zope.App.startup, line 222, in commit * Module ZODB.Transaction, line 260, in commit * Module ZODB.Transaction, line 377, in _finish_many * Module Shared.DC.ZRDB.TM, line 59, in tpc_finish * Module Products.ZMySQLDA.db, line 376, in _finish error: release unlocked lock The lock being released is a transactional lock. None of our locks should be transactional locks, given our setup. I traced through the __init__() method for DB and did not see how transactional locking could be specified so the lock relaease code could run--I must be missing something. No thread in our system uses transactional locking as the tables do no support transactions. The logged_in method references the MySQL database in several places; a couple of them are compound queries where multiple queries are joined by dtml-var mysql_delimiter values. Many of our queries use this in combination with the use database name SQL query. The system is using ZODB filestorage (Data.fs) with ZEO and is pretty much out of the box. Has anyone else encountrered a similar problem. Any suggestions as to how to find this bug? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] External editor for ZMI
emacs via ftp -- see the Zope Book for funky details. On Thu, 23 Jun 2005, Paul Winkler wrote: On Thu, Jun 23, 2005 at 04:27:50PM +0100, John Poltorak wrote: What alternatives are there to editing objects through ZMI? ExternalEditor. -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] External editor for ZMI
External editor is an interface product that allows you to edit Zope objects with your local editor (vi, emacs, nano, wordpad, etc) but it does require installation of a helper product and Python on the client machine. On Thu, 23 Jun 2005, John Poltorak wrote: On Thu, Jun 23, 2005 at 11:49:19AM -0400, Paul Winkler wrote: On Thu, Jun 23, 2005 at 04:27:50PM +0100, John Poltorak wrote: What alternatives are there to editing objects through ZMI? ExternalEditor. Do you mean this? http://plope.com/software/ExternalEditor Are there any screen shots of it in action? I'm not exactly sure how it works, but have feeling it won't work for me. -- Paul Winkler http://www.slinkp.com -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZMySQLDB problems
I am seeing occasional errors like those below. Any have any idea what my problem here is? Zope 2.7.6 MySQL-server-4.0.20-0 MySQLDB aka MySQL-python-1.2.tc3 ZMySQL 2.0.9 Python 2.4.1 Typically the errors occur at initial login when the database is consulted to populate various session variables, etc. The my.cnf file is, in part: # change timeouts to close rather than wait # interactive == 1 hour, programs == 5 mins set-variable=interactive_timeout=3600 set-variable=wait_timeout=300 # uncomment to not log connection problems #set-variable=log-warnings=off 2005-06-26T23:11:06 ERROR(200) ZMySQLDA exception during _begin Traceback (most recent call last): File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 349, in _begin self.db.ping() OperationalError: (2013, 'Lost connection to MySQL server during query') -- 2005-06-26T23:11:06 PANIC(300) ZODB A storage error occurred in the last phase of a two-phase commit. This shouldn't happen. The application will not be allowed to commit until the site/storage is reset by a restart. Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Transaction.py, line 377, in _finish_many jar.tpc_finish(self) File /usr/local/src/zope/Zope2.7/lib/python/Shared/DC/ZRDB/TM.py, line 59, in tpc_finish try: self._finish() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 376, in _finish self._tlock.release() error: release unlocked lock -- 2005-06-26T23:11:06 ERROR(200) ZODB A storage error occured during object abort. This shouldn't happen. Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Transaction.py, line 412, in _commit_error j.tpc_abort(self) # This should never fail File /usr/local/src/zope/Zope2.7/lib/python/Shared/DC/ZRDB/TM.py, line 63, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-06-26T23:11:06 ERROR(200) SiteError http://dev.agilemind.com/logged_in Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZPublisher/Publish.py, line 107, in publish transactions_manager.commit() File /usr/local/src/zope/Zope2.7/lib/python/Zope/App/startup.py, line 222, in commit get_transaction().commit() File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Transaction.py, line 260, in commit self._finish_many(jars) File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Transaction.py, line 377, in _finish_many jar.tpc_finish(self) File /usr/local/src/zope/Zope2.7/lib/python/Shared/DC/ZRDB/TM.py, line 59, in tpc_finish try: self._finish() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 376, in _finish self._tlock.release() error: release unlocked lock -- -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Checking for an Integer Argument
in Zope Python Scriptssametype( x, 1 ) On Thu, 30 Jun 2005, Asad Habib wrote: Is there any way to check if an argument is an integer in Python? Offcourse, there is the int function but it chokes when passed an argument that is not a string and then the user is inconvenienced (it throws a ValueError to be specific). I guess I could check each digit of the integer to determine if its ASCII value falls within the appropriate range and if each digit passed this test, then the entire number would be an integer. Does anyone know of a simpler way that involves use of a predefined Python function? Likewise, does anyone know if a similar function exists to detect floating point numbers? Any help would be appreciated. Thanks. - Asad ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Dennis Allison * Computer Systems Laboratory * Gates 227 * Stanford University * Stanford CA 94305 * (650) 723-9213 * (650) 723-0033 fax * [EMAIL PROTECTED] * [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] where is Zope-2.7.6
The download list at zope.org does not have any releases between 2.7.3 and 2.8. We've been using 2.7.6 and would like to be able to download clean copies... ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] where is Zope-2.7.6
Thanks Chris, I found my master copy... but there are (I think) other users. On Sat, 9 Jul 2005, Chris McDonough wrote: I think you're going to need to guess the URL at the moment (shouldn't be too hard)... the releases are there, they just don't show up in the old releases list. On Sat, 2005-07-09 at 15:31 -0700, Dennis Allison wrote: The download list at zope.org does not have any releases between 2.7.3 and 2.8. We've been using 2.7.6 and would like to be able to download clean copies... ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] where is Zope-2.7.6
Thanks Tim. I'm not really sure where I was looking--I followed the standard download, then to the 'other releases tab. And the 2.7.X X3 releases somehow were not visible On Sat, 9 Jul 2005, Tim Peters wrote: [Dennis Allison] The download list at zope.org does not have any releases between 2.7.3 and 2.8. We've been using 2.7.6 and would like to be able to download clean copies... The download list at http://www.zope.org/Products/Zope/ has an entry for 2.7.6. If you ended up at http://www.zope.org/Products/Zope/folder_contents/ instead, there's still an entry for 2.7.6, but it's on the second page (click the next 46 items link near the upper right to get to the second page). Or go straight to http://www.zope.org/Products/Zope/2.7.6/ -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] dtml to pdf
For simple conversion, you may find htmldoc to be a good choice. Reportlab is a great tool but takes considerable effort; htmldoc gives a good enough solution in many cases. On Fri, 15 Jul 2005, Chris Withers wrote: Allen Huang wrote: How do I change my webpage into pdf using dtml or python script? Reportlab is your friend. Google might help you find it ;-) Chris -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] configurating Zopes for merged logs
Yup, that is one of my fallback approachs. It seems to me that I should be able to write a configuration for the Zopes that either invokes a remote syslogd server (or other server) which will merge multiple logging entries into a single file on one of the machines in the cluster, preferably the ZEO host. Another approaches would be to maintain local logs and then have a process running on the remote machine that performs the merge. We post-process the logs in nearly real-time and we expect related transactions to spread across multiple Zopes. On Mon, 1 Aug 2005, Andreas Jung wrote: --On 1. August 2005 09:51:37 -0700 Dennis Allison [EMAIL PROTECTED] wrote: Zope 2.7.6 with ZEO Python 2.4.1 I would like to configure for a single merged log. If BE is the backend machine running ZEO and FEA FEB and FEC are various front-ends for the ZEO, what's the configuration supposed to look like to get a merged log collected on the machine on which ZEO runs. The only log I need to merge is the Z2.log--the othes can be left per-Zope frontend. While it would be nice if the merged entries were serialized, it is not strictly necessary. Cross machine transmission can be either UDP or TCP. I don't know what the sense of a merged Z2.log is. You can install a front-end proxy + load balancer so you could use the logfile of the proxy or write a small script to merge the the Z2.log from multiple clients into a single file. You should be straight forward with some line of Python. -aj -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] configurating Zopes for merged logs
Exactly what I was planning to do. The docs themselves indicate they are incomplete. Is there any documentation beyond that supplied with the distribution. (I have read the logging docs and PEP 282.) Python 2.4.1 seems to be clean and stable with Zope 2.7.6. We've been running it in production for a while with no problems attributable to it. My understanding is that the only reason it is unsupported is that a security audit has not been done on the changes from Python 2.3.5. On Mon, 1 Aug 2005, Jens Vagelpohl wrote: Zope 2.7.6 with ZEO Python 2.4.1 I would like to configure for a single merged log. If BE is the backend machine running ZEO and FEA FEB and FEC are various front-ends for the ZEO, what's the configuration supposed to look like to get a merged log collected on the machine on which ZEO runs. The only log I need to merge is the Z2.log--the othes can be left per-Zope frontend. While it would be nice if the merged entries were serialized, it is not strictly necessary. Cross machine transmission can be either UDP or TCP. Apart from the fact that Python 2.4.1 is unsupported on Zope 2.7.6, have you looked at the logger section in zope.conf? You can configure a syslog loghandler for any log and send the logging output to a syslog server. See zope.conf and zope/lib/python/zLOG/component.xml for the configuration knobs. jens ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] (no subject)
We've been seeing this problem for some time. Earlier postings to the list were helpful in that we have eliminated possible causes, but the problem has never been resolved. Any assistance in tracking down the problem and fixing it would be greatly appreciated. Our configuration: Athlon dual processor machine RH7.3 Zope 2.7.6 with ZEO Python 2.4.1 (same problem occurs with Python 2.3.5) Mysql-Python 1.21.c3 ZMySQLDA 2.09b3 MySql-4.0.20 This is a fragment from the event log: 2005-06-07T16:06:42 ERROR(200) ZMySQLDA exception during _begin Traceback (most recent call last): File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 349, in _begin self.db.ping() OperationalError: (2013, 'Lost connection to MySQL server during query') -- 2005-06-07T16:06:42 PANIC(300) ZODB A storage error occurred in the last phase of a two-phase commit. This shouldn't happen. The application will not be allowed to commit until the site/storage is reset by a restart. Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Transaction.py, line 377, in _finish_many jar.tpc_finish(self) File /usr/local/src/zope/Zope2.7/lib/python/Shared/DC/ZRDB/TM.py, line 59, in tpc_finish try: self._finish() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 376, in _finish self._tlock.release() error: release unlocked lock -- 2005-06-07T16:06:42 ERROR(200) ZODB A storage error occured during object abort. This shouldn't happen. Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Transaction.py, line 412, in _commit_error j.tpc_abort(self) # This should never fail File /usr/local/src/zope/Zope2.7/lib/python/Shared/DC/ZRDB/TM.py, line 63, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-06-07T16:06:42 ERROR(200) SiteError http://dev.agilemind.com/logged_in Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZPublisher/Publish.py, line 107, in publish transactions_manager.commit() File /usr/local/src/zope/Zope2.7/l[safe_mysqld] err-log=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.-- 2005-08-06T20:47:21 ERROR(200) ZMySQLDA exception during _begin Traceback (most recent call last): File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 349, in _begin self.db.ping() OperationalError: (2013, 'Lost connection to MySQL server during query') -- We get the release unlocked lock message occasionally, frequently but not always preceded by the connection lost message. Our my.cnf file for MySQL is [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock skip-innodb set-variable=max_connections=2048 set-variable=long_query_time=1 #log-bin=/var/log/mysql/epaul-bin #log-bin-index=/var/log/mysql/epaul-bin.index #log-slow-queries=/var/log/mysql/epaul-slow.log # change timeouts to close rather than wait # interactive == 1 hour, programs == 5 mins set-variable=interactive_timeout=3600 set-variable=wait_timeout=300 # uncomment to not log connection problems #set-variable=log-warnings=off [mysql.server] user=mysql basedir=/var/lib [safe_mysqld] err-log=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid We use MyISAM tables and the MySQL autocommit mode rather than handling locks and transactions explicitly. From a user perspective, refreshing the Zope page recovers from the error and allows one to continue working. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] error: release unlocked lock with Zope/MySQL
[Reposted with a non-blank Subject field] We've been seeing this problem for some time. Earlier postings to the list were helpful in that we have eliminated possible causes, but the problem has never been resolved. Any assistance in tracking down the problem and fixing it would be greatly appreciated. Our configuration: Athlon dual processor machine RH7.3 Zope 2.7.6 with ZEO Python 2.4.1 (same problem occurs with Python 2.3.5) Mysql-Python 1.21.c3 ZMySQLDA 2.09b3 MySql-4.0.20 This is a fragment from the event log: 2005-06-07T16:06:42 ERROR(200) ZMySQLDA exception during _begin Traceback (most recent call last): File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 349, in _begin self.db.ping() OperationalError: (2013, 'Lost connection to MySQL server during query') -- 2005-06-07T16:06:42 PANIC(300) ZODB A storage error occurred in the last phase of a two-phase commit. This shouldn't happen. The application will not be allowed to commit until the site/storage is reset by a restart. Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Transaction.py, line 377, in _finish_many jar.tpc_finish(self) File /usr/local/src/zope/Zope2.7/lib/python/Shared/DC/ZRDB/TM.py, line 59, in tpc_finish try: self._finish() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 376, in _finish self._tlock.release() error: release unlocked lock -- 2005-06-07T16:06:42 ERROR(200) ZODB A storage error occured during object abort. This shouldn't happen. Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZODB/Transaction.py, line 412, in _commit_error j.tpc_abort(self) # This should never fail File /usr/local/src/zope/Zope2.7/lib/python/Shared/DC/ZRDB/TM.py, line 63, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-06-07T16:06:42 ERROR(200) SiteError http://dev.agilemind.com/logged_in Traceback (most recent call last): File /usr/local/src/zope/Zope2.7/lib/python/ZPublisher/Publish.py, line 107, in publish transactions_manager.commit() File /usr/local/src/zope/Zope2.7/l[safe_mysqld] err-log=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.-- 2005-08-06T20:47:21 ERROR(200) ZMySQLDA exception during _begin Traceback (most recent call last): File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 349, in _begin self.db.ping() OperationalError: (2013, 'Lost connection to MySQL server during query') -- We get the release unlocked lock message occasionally, frequently but not always preceded by the connection lost message. Our my.cnf file for MySQL is [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock skip-innodb set-variable=max_connections=2048 set-variable=long_query_time=1 #log-bin=/var/log/mysql/epaul-bin #log-bin-index=/var/log/mysql/epaul-bin.index #log-slow-queries=/var/log/mysql/epaul-slow.log # change timeouts to close rather than wait # interactive == 1 hour, programs == 5 mins set-variable=interactive_timeout=3600 set-variable=wait_timeout=300 # uncomment to not log connection problems #set-variable=log-warnings=off [mysql.server] user=mysql basedir=/var/lib [safe_mysqld] err-log=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid We use MyISAM tables and the MySQL autocommit mode rather than handling locks and transactions explicitly. From a user perspective, refreshing the Zope page recovers from the error and allows one to continue working. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] database connections in a multi-Zope zeo environment
I am sure this is a synaptic disconnect, but I can't seem to get my head around this one. I have remote Zopes (call them A, B, C ) and a database server (D). I also have a bunch of mysql databases. All of the machines run mysql. All the Zope instances share the same Data.fs connected via Zeo. The two cases of interet are how to access a local database, that is, one the same machine as the running Zope, and, how to access a remote mysql database. ZMYSQLDA provides a static binding to connections. MYSQL grants access privileges based on source IP address. So, here are my questions: 1. When I open a database connection with the connection string -u dbroot -h localhost -passwd mypassword what machine is accessed? I'm presuming localhost is always the local machine. Is that correct? 2. How do I open a remote database connection so it works transparently across multiple instances of Zope? The Zope code, shared across instances, uses a single connection with a connection string like -u dbroot -h 192.168.0.3 -passwd somepassword Presumably each of the remote machines (and the database server if it runs Zope) needs an entry in the grant table. And all the database connections need to share the same password. Right? 3. Since queries are bound to their connection, there appears to be no easy way to manage connections in a simple way programmatically. Any help/comments much appreciated. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] redirecting logs
I want to direct the Z2.log file to both a local file and to a remote machine. I think the configuration might be something like this: logger access level WARN logfile path $INSTANCE/log/Z2.log format %(message)s /logfile syslog address 192.168.0.92:514 format %(message)s /syslog /logger but it does not seem to function correctly. Has anyone tried this? Any assistance would be appreciated. This is Zope 2.7.6. -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] RFC: Requiring Python 2.4 in Zope 2.9
I have been running Zope 2.7.6 with Python 2.4.1 without problems for several months now. On Wed, 7 Sep 2005, Jim Fulton wrote: At: http://www.zope.org/Wikis/DevSite/Proposals/RequirePython24 Is a proposal to require Python 2.4 for Zope 2.9 (to be released this December). Comments are welcome. :) Jim -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Log aggregator?
When you get a solution let me know. I am in the middle of log hell at the moment (including Zopes logs) trying to get things to sync up properly across multiple machines. When my stuff works, I'll share. -d On Fri, 23 Sep 2005, Paul Winkler wrote: Anybody know of any software that will aggregate logs in a variety of formats and allow me to correlate events in one log with another? I'm getting really REALLY tired of eyeballing and grepping a multitude of logs on different servers, mentally translating incorrect timestamps, and trying to work out what the heck from one log corresponds to something in another. Anytime something goes wrong, i waste a huge amount of time on this. Well, duh, this should be automated, right? I googled a bit and came up with nothing useful; then I started hacking up a python script, which I'm sure I could do in an hour or three, but - blah, it would be a huge timesaver if somebody had written something like this already. I need to handle: * Different log formats. I'm mainly interested in the zope access log format (aka apache format) and the zope event log format. Must tolerate malformed lines like the #!@)%* ftp lines in the zope access log. * System clocks being incorrect. On some of the servers I work with, I am not the primary admin and can't do much about the [EMAIL PROTECTED] clock being wrong. (*cough* firewall blocking ntp for no good reason *cough*) So I'd like the software to let me specify how far wrong the various logs are, and compensate. * Servers in different time zones. Rare for me, but it does come up. * Filtering based on arbitrary strings, or (better) regular expressions, on a per-log basis. * Multi-megabyte logs. Sometimes i go to check a log and sysadmin has forgotten to rotate ever... -PW ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Write log file from script
Depends upon what you want to do and how much access you have. For
debugging purposes I often use an external procedure
def debugWindow( data ):
fd = open('/tmp/debugWindow,'a')
fd.write( str(data))
fd.close()
and look at the output with
tail -f /tmp/debugWindow
Or, you can use the same approach to write to syslog using the Python
logging module
Or you can piggyback into the Zope logging mechanism--see the sources for
that exercise.
I find the external procedure approach to be useful in its simplicity
On Fri, 7 Oct 2005, Brian Sullivan wrote:
I am looking for a simple strategy to write a debug log file from a python
script.
I used to have such a beast but somewhere in a shuffle it got lost -- and
my brain seems to be dead this morning trying to figure it out.
Anybody out there have something like this or a url explaining how to do
it?
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Write log file from script
We are running on a linux host. The file information ends upon
in the local file system in the /tmp directory.
On Fri, 7 Oct 2005, Brian Sullivan wrote:
On 10/7/05, Tino Wildenhain [EMAIL PROTECTED] wrote:
Am Freitag, den 07.10.2005, 09:03 -0700 schrieb Dennis Allison:
Depends upon what you want to do and how much access you have. For
debugging purposes I often use an external procedure
def debugWindow( data ):
fd = open('/tmp/debugWindow,'a')
fd.write( str(data))
fd.close()
Actually this is a bit dangerous if you dont lock
the file.
import zLOG
def log(self,message):
zLOG.LOG(PythonScript, zLOG.INFO http://zLOG.INFO, message)
as external method should be enough.
And where does this log information end up?
--
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ooops... ZMYSQL tests fail in the ZMI on batching dynamic queries
Sorry for the long message, but there's a lot of context. I've tried to factor the problem out and then provide supporting stuff for the interested. Zope 2.7.6, Python 2.4.1 (Python 2.3.5 behaves in the same fashion). In summary, the problem is that a query run as a test from the ZMI of the form select * from dtml-var udb.tablename fails on the second batch whereas select * from foo.tablename works fine for all batches. The relevant Zope code distributed with Zope 2.7.6-final had a bug in it which was discussed earlier on the list. The distributed code did not terminate batching properly. The problematic code was in /home/allison/Zope-2.7.6-final/lib/python/Shared/DC/ZRDB/dtml and the fix was to patch the two link expressions to have a missing element, that is, a href=dtml-URL;?query_start=dtml-next-sequence-start-number; \ query=dtml-var query url_quote missing= which seemed to resolve the problem. BUT, there is now a second problem. When the query is dynamic, the second batch fails on a key error -- for the example above, it would fail with a Z SQL Method at /testfolder/batchcheck External Editor Help! Error, exceptions.KeyError: 'udb' SQL used: Could not render the query template! message in the ZMI. However, REQUEST has the proper value for the key according to the Error Log: form query '' query_start '21' other n_ 5 a_ 0 It's pretty clear that the query is being passed as an empty string due to the key error. Presumably the query should be rendered and saved in a persistent manner throughout the batching process. It is not clear to me where this should be done. Any help in localizing and fixing the problem would be appreciated. -d For the record, the original posting is appended From [EMAIL PROTECTED] Sat Oct 8 10:52:32 2005 Date: Tue, 7 Jun 2005 03:41:21 -0700 (PDT) From: Dennis Allison [EMAIL PROTECTED] To: zope@zope.org Subject: Z SQL test in the ZMI aborts on key error Zope 2.7.6 Python 2.3.5 MySQLDA-1.2.1c3 ZMYSQLDA-2.09b3 Site Error An error was encountered while publishing this resource. Error Type: KeyError Error Value: 'query' Troubleshooting Suggestions * This resource may be trying to reference a nonexistent object or variable 'query'. * The URL may be incorrect. * The parameters passed to this resource may be incorrect. * A resource that this resource relies on may be encountering an error. -- Error log: 2005/06/07 03:33:03.344 GMT-7 User Name (User Id) allison (allison) Request URL http://group1.agilemind.com/Scratch/simpleq/manage_test Exception Type KeyError Exception Value 'query' Traceback (innermost last): * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Shared.DC.ZRDB.DA, line 337, in manage_test * Module DocumentTemplate.DT_String, line 474, in __call__ * Module DocumentTemplate.DT_In, line 602, in renderwb * Module DocumentTemplate.DT_Var, line 219, in render KeyError: 'query' There does not seem to be a bug like this in the collector. Any help/pointers appreciated ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: ooops... ZMYSQL tests fail in the ZMI on batching dynamic queries
Right. And that is the problem. I patched lib/python/Shared/DC/ZRDB/dtml/customDefaultReport.dtml per your suggestion and it fixed the bug I'd reported earlier which had to do with proper termination of the ranges. Those now work. Now, there is a second bug. When the query is dynamic, that is, the result of a dtml evaluation, the query value (that is, the result of the evaluation) does not seem to be preserved and re-evaluated. On Sat, 8 Oct 2005, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dennis Allison wrote: Sorry for the long message, but there's a lot of context. I've tried to factor the problem out and then provide supporting stuff for the interested. Zope 2.7.6, Python 2.4.1 (Python 2.3.5 behaves in the same fashion). In summary, the problem is that a query run as a test from the ZMI of the form select * from dtml-var udb.tablename fails on the second batch whereas select * from foo.tablename works fine for all batches. The relevant Zope code distributed with Zope 2.7.6-final had a bug in it which was discussed earlier on the list. The distributed code did not terminate batching properly. The problematic code was in /home/allison/Zope-2.7.6-final/lib/python/Shared/DC/ZRDB/dtml and the fix was to patch the two link expressions to have a missing element, that is, a href=dtml-URL;?query_start=dtml-next-sequence-start-number; \ query=dtml-var query url_quote missing= which seemed to resolve the problem. BUT, there is now a second problem. When the query is dynamic, the second batch fails on a key error -- for the example above, it would fail with a Z SQL Method at /testfolder/batchcheck External Editor Help! Error, exceptions.KeyError: 'udb' SQL used: Could not render the query template! message in the ZMI. However, REQUEST has the proper value for the key according to the Error Log: form query '' query_start '21' other n_ 5 a_ 0 It's pretty clear that the query is being passed as an empty string due to the key error. Presumably the query should be rendered and saved in a persistent manner throughout the batching process. It is not clear to me where this should be done. Any help in localizing and fixing the problem would be appreciated. Note that the missing key in your case is 'udb', while in the older case it was 'query'. I don't know where the first page was getting its value for 'udb'; there is no key for it in the error log entry you cite. Error Type: KeyError Error Value: 'query' Tres. - -- === Tres Seaver +1 202-558-7113 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDSG7B+gerLs4ltQ4RAjNlAJ0YaJL4WYWWplyAm3j+godYePWDeACgtcmH bTmMvGeXUpg9sFZTVd2Vrb4= =QSIr -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope 2.7.8 or Zope 2.8.X
We have been running on Zope 2.7.6 with a few patches. I am now planning to migrate to Zope 2.7.8 but am pondering the move to Zope 2.8. We are in a production situation and cannot accept any downtime, so I am very concerned about stability and compatibility. Can the list provide words of advice? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] resolving conflict errors
Zope 2.7.6 I am a bit confused. I have a Zope DTML method that is generating ZODB conflict errors. The DTML method identified as producing the conflicts is a list of calls to other methods, conditionally executed. Most conflicts don't cause problems because the backoff and restart of the initial transaction will not have changed global state. In our particular case, the conflicting transaction has changed global state in our RDBMS so when it gets rerun, some RDBMS transactions are duplicated. And that's a problem. The solution, of course, is to resolve the conflicts properly. The first question: what data is generating the conflict? The DTML code and all the method references are static and unchanged. What data does Zope store in the ZODB when an object is evaluated? Presumably conflicts can be reolved programatically by setting a method on the object _p_resolveConflict( self, old, saved, new ) and returning one or another of the states (old, saved, new). It's not real clear how to do it. -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] resolving conflict errors
Thanks Chris. On point as usual! I was unaware that the session mechanism used the ZODB although a bit of thought says it has to. We don't store any data into the ZODB in these methods, but we do use the session mechanism heavily. I suppose that moving to a fully transactional database system would be the simplest solution, but I'm a bit wary of doing so on a live system. On Fri, 14 Oct 2005, Chris McDonough wrote: On Fri, 2005-10-14 at 09:27 -0700, Dennis Allison wrote: Zope 2.7.6 I am a bit confused. I have a Zope DTML method that is generating ZODB conflict errors. The DTML method identified as producing the conflicts is a list of calls to other methods, conditionally executed. Most conflicts don't cause problems because the backoff and restart of the initial transaction will not have changed global state. In our particular case, the conflicting transaction has changed global state in our RDBMS so when it gets rerun, some RDBMS transactions are duplicated. And that's a problem. The solution, of course, is to resolve the conflicts properly. Another solution is to use a RDBMS that fully supports transactions. This is almost always the easiest solution. ConflictErrors are a fact of life if you use ZODB to store data; they are impossible to eliminate entirely so no matter what, if you need to communicate with external data stores that aren't transactional (MyISAM tables, LDAP, sendmail, etc.) you need to anticipate duplicated requests in the code that communicates with these systems. For example, Jens Vagelpohl has a replacement for Zope's MailHost that prevents retried requests due to conflict errors from causing a mail to be resent. I've written payment systems that anticipate the fact that the request may be retried, so instead of submitting the payment request twice, the code keeps around a little cache about what it did last so it doesn't do it again. And so on. The first question: what data is generating the conflict? I believe that if you run Zope event logging at BLATHER level, the traceback of every ConflictError exception is logged, which can give you an idea of what is causing the errors. The DTML code and all the method references are static and unchanged. What data does Zope store in the ZODB when an object is evaluated? None that you don't tell it to. Typically conflict errors are a result of two threads calling code which changes the same object at the same time, but nothing that Zope does under the hood causes it; it is always caused by application code. One exception to this rule is conflict errors raised when using Zope sessions. It's not actually an exception to the rule, but programmers are shielded from the fact that sessions store data in ZODB when you use the session API (e.g. REQUEST.SESSION). The sessioning machinery needs to manage housekeeping info whenever the API is used to expire old sessions and create new ones, so although it may not look like you are writing to the ZODB when you use sessions (even to read data out of them), you potentially are. Zope 2.8 has a ZODB that support multiversion concurrency control, which eliminates a certain class of conflict errors (read conflict errors), so if you are getting a lot of these, and you can get away with using 2.8, I'd suggest doing so. Presumably conflicts can be reolved programatically by setting a method on the object _p_resolveConflict( self, old, saved, new ) and returning one or another of the states (old, saved, new). It's not real clear how to do it. There are examples of real-world conflict resolution using this mechanism in the Transience product included in Zope's lib/python/Products. - C -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Error on starting and stopping Zope 2.8.2
whrandom is a deprecated python library package which should be replaced by random. Python 2.3.5 does not produce the deprecation warning but lacks some of the features of the Python 2.4.X series. On Fri, 14 Oct 2005 [EMAIL PROTECTED] wrote: Hello. On starting and stopping Zope 2.8.2 (upgraded from 2.8.1, compiled with Python 2.4.2, running on Fedora Core 3) I get the following error. Performance does not seem to be affected. Resolution? # service zopectl start /opt/python2.4.2/lib/python2.4/whrandom.py:38: DeprecationWarning: the whrandom module is deprecated; please use the random module DeprecationWarning) . daemon process started, pid=22829 # service zopectl stop /opt/python2.4.2/lib/python2.4/whrandom.py:38: DeprecationWarning: the whrandom module is deprecated; please use the random module DeprecationWarning) . daemon process stopped -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Error on starting and stopping Zope 2.8.2
Pascal, Why does Zope lag the release of Python by so very long? The Python 2.4.X series includes language features and poerformance enhancements which ought to be available in Zope. For the most part, Python 2.3.5 is a subset of Python 2.4.X. Compatibility is a language and implementation design priority. Python 2.4 was released November 30, 2004, almost a year ago. Python 2.3.5 was released Feb 8, 2005 (after Python 2.4.0, released November 30, 2004, and shortly before Python 2.4.1 which was released March 30, 2005). Python 2.3 was released July 29, 2003. The current Python 2.4.2 was released just recently. My understanding is that the reason the Python 2.4.X is not the recommended Python is because a security audit has not been performed and not because of functionality issues.We have been using Python 2.4.1 in production without apparent problems; I am about to move to 2.4.2 which fixes some bugs we have not encountered. IMHO it would be wise to track the releases of Python a bit more closely. On Fri, 14 Oct 2005, Pascal Peregrina wrote: From the Zope 2.8.2 announcement on this mailing list : Please also keep in mind that Zope 2.8.2 requires Python 2.3.5. Zope 2.8.2 is not certified for any Python 2.4.x versions. So using Python 2.4 is neither recommended nor supported and any related questions or problems are likely to be ignored until 2.4 is an officially supported Python version for Zope. Pascal -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Error on starting and stopping Zope 2.8.2
Andreas, Aside from a security audit what problems have been reported using Python 2.4.X? Those of use who run Python 2.4.X do it becasue we want access to the language features and performance. If there are stability issues attributed to that choice, we'd like to know what they are. If the reason for trying to stick with 2.3.5 is to maintain a stable test and development environment for Zope 2.8, so be it. It is much much easier to fix one of the multitude of possible variables in a complex system. I, for one, have occasionally created a special instance of Zope running a different Python to ensure a problem I had discovered was not python version related. -d On Fri, 14 Oct 2005, Andreas Jung wrote: --On 14. Oktober 2005 10:09:39 -0600 [EMAIL PROTECTED] wrote: Hello. On starting and stopping Zope 2.8.2 (upgraded from 2.8.1, compiled with Python 2.4.2, running on Fedora Core 3) I get the following error. Performance does not seem to be affected. Resolution? Quote from my announcement yesterday: Please also keep in mind that Zope 2.8.2 requires Python 2.3.5. Zope 2.8.2 is not certified for any Python 2.4.x versions. So using Python 2.4 is neither recommended nor supported and any related questions or problems are likely to be ignored until 2.4 is an officially supported Python version for Zope. :-) -aj -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Error on starting and stopping Zope 2.8.2
Andreas, I understand both the importance of stability and the need for a security audit at some point. (I do wish we had the funds or I had the time to help move it forward, but I don't.) I also understand the need for a consistent framework for reporting and resolving bugs. It is reasonable to expect that all bugs be reported against the same framework to eliminate one significant possible variable. What does concern me is the way in which the recommendation to use (at the moment) Python 2.3.5 is explained. I may be willing to accept the risks of using a system which has not yet been audited in terms of security, but I want to know if there are any reported instabilities or incompatibilities which have been identified when, say, Python 2.4.X is used. I'd rather people say that the standard reference platform against which all bugs should be reported uses 2.3.5, and that use of other, later versions of Python is at your own risk. When using another Python that is known to cause problems, it would make sense to identify the problem so that users can make an informed decision. There are times when there are Python version related problems and these need to be identified and publicized. We certainly collect the incompatilities (if there are any) so they can be fixed as eventually the code base will move to later python systems. -d On Sun, 16 Oct 2005, Andreas Jung wrote: --On 16. Oktober 2005 11:54:18 -0700 Dennis Allison [EMAIL PROTECTED] wrote: IMHO it would be wise to track the releases of Python a bit more closely. Software components choosen for a framework have to be solid and approved. There is no reason to run after every new python version or whatever. Stability and performance is somewhat more important than hunting for new language features.. Some features of Python 2.4 are nice2have but we can perfectly live with Python 2.3.5. If you want Python 2.4, use it (at your own risk). As we already explained a bunch of times (sorry, this issue is bothering the more people ask about the same issue), a security audit has not happened yet. Why not? Because it takes time to do such an audit and the persons that can do such an audit likely had not time so far. So things are as they are and will change as they change. If you have the skills, resources and perhaps some money to fund the audit then raise your hand. Otherwise we have to wait until it will happen. -aj -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope 2.8.4 and Plone
We've run into problems tryin to run Plone under Zope 2.8.4. What is the current best practice? -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Sessions and Persistence
Current configuration: Linux RH7.3 Zope 2.8.4 Python 2.4.2 (but the same is observed with Python 2.3.5) I continue to observe occasional failures of the Zope session variable mechanism. The problem manifests itself by a KeyError when accessing data in the session data container. We have recently moved to Zope 2.8.4 from Zope 2.7.6. The frequency of these errors aooears to have increased significantly; of late, we have occasionally seen error messages saying that an entire SESSION data container is missing. The session variable interface is very simple: a pair of Python Scripts, one that gets a session variable and another that sets it. Both use the copy-out/copy-in mechanism required for persistence. ## Script (Python) getSessionVariable ##bind container=container ##bind context=context ##bind namespace= ##bind script=script ##bind subpath=traverse_subpath ##parameters=varname ##title= ## request=container.REQUEST session=request['SESSION'] return session[varname] # Script (Python) setSessionVariable ##bind container=container ##bind context=context ##bind namespace= ##bind script=script ##bind subpath=traverse_subpath ##parameters=var, val ##title= ## request = container.REQUEST RESPONSE = request.RESPONSE session=request['SESSION'] session[var]=val request.set( 'SESSION', session ) Most of the information we store in session variables are simple numbers or strings. In a few instances, we store python lists, python tuples, or python dictionaries. I am beginning to suspect the source of my bug is a persistence problem perhaps triggered by a conflict error or garbage collection in the temporary store used to hold the session variables. I further suspect that lists and dictionaries stored in session variables need to be instances of PersistentDict and PersistentList and not ordinary python data structures. I'd appreciate confirmation on these two issues from someone who knows the persistence mechanism better than I. Neither PersistentDict nor PersistentList are exposed in the restricted namespace. Advice as to how to expose them would be helpful as well. If I have a complex data structure (a list of dictionaries) does all the substructure need to have persistence wrappers as well--that is, does it have to be turtles all the way down? Thanks in advance. -d ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] session variables in the presence of conflicts
Zope 2.8.4 ZEO 3.4.2 ZODB 3.4.2 Python 2.4.2 or 2.3.5 MySQL 4.0.20 MySQL-Python 1.2.0 MYSQLDA 2.0.9 We have just moved from Zope 2.7.6 to Zope 2.8.4 motivated, in part, but the ability to avoid read conflicts under ZODBÂ 3.4.2. We have been having a lot of problems: more conflict errors, release of unreleased locks in MySQL transactions, loss of session variables, and others. We use session variables and MySQL very heavily. Our MySQL tables are MyISAM and are not transactional. ZODB3 3.4.1 introduce some changes to the conflict resolution which appears to be a logical improvement but also appears to have acerbated our problems. In the ZODB# 3.4.2 system, if a commit fails, an exception is raised and the transaction is aborted. Zope then attempts to reschedule the transaction. The abort needs to roll back MySQL transactions that are part of the conflicting ZODB and the Session Variable Store so the restart is clean. In our case, we have been using MyISAM tables, which are not transactional. The rollbacks fail and the locking in the database adapter gets confused. Changing the table type from MyISAM to Innodb, which is transactional, should resolve the MySQL problems, although it has yet to be tested. Conflict errors also arise through the session variables. Since we have a multi-Zope system configuration sharing a single ZEO, the session variables are stored in a Temporary Storage object on the ZEO host. When a conflict occurs, a number of things, none good, apparently can happen. Session variable transactions cannot be rolled back (as the temporary storage mechanism is not transactional). And, at a different level, session variable persistence requires that transactions require a copy-out/copy-in mechanism to identify those changes which have been changed and need to persisted; an aborted transaction may not be marked as persistent and so will disappear at some apparently random time in the future when the temporary store is packed and cruft removed. Some refactoring of the programming can help minimize the potential for session variable conflicts, but they cannot be eliminated. If systems stability in the face of session variable conflicts is important, 1. Does the store for session variable need to be transactional and support rollbacks? and 2. What Temporary Storage object provides the right mechanism to manage session variables with rollback efficiently? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] still having problems
ZEO 3.4.2 Zope 2.8.4 ZODB 3.4.2 as relesed with Zope 2.8.4B Python 2.4.2 or 2.3.5 MySQL 4.0.20 MySQL-Python 1.2.0 MYSQLDA 2.0.9 We have just moved from Zope 2.7.6 to Zope 2.8.4 motivated, in part, but the ability to avoid read conflicts under ZODBÂ 3.4.2. We have been having a lot of problems: more conflict errors, release of unreleased locks in MySQL transactions, loss of session variables, and others. We use session variables and MySQL very heavily. All database tables are now transactional (innodb) migrated from the MyISAM tables in use earlier. The connection string, by the way, does not use either the '-' or '+' prefix but relies on a run-time test for the table to be determined to be transactional. All our Zope instances run with ZEO and use a Temporary Store instance for session variables. The ZEO configuration is: filestorage 1 path $INSTANCE/var/Data.fs /filestorage # temporary storage has to be ZEO side %import tempstorage temporarystorage temp name temporary storage for sessioning /temporarystorage and is matched to the Zope configuration: zodb_db main mount-point / # ZODB cache, in number of objects cache-size 5000 zeoclient server 192.168.0.92:8301 storage 1 var $INSTANCE/var # ZEO client cache, in bytes cache-size 20MB # Uncomment to have a persistent disk cache client group1-zeo /zeoclient /zodb_db # # ZEO temporary storage # zodb_db temporary zeoclient server 192.168.0.92:8301 storage temp name zeostorage var $INSTANCE/var /zeoclient mount-point /temp_folder container-class Products.TemporaryFolder.TemporaryContainer /zodb_db Are these configurations correct? The snippet of log shown below show the problem that plagues us: -- 2005-11-14T06:20:40 ERROR txn.4100 Error in tpc_abort() on manager MultiObjectResourceAdapter for Products.ZMySQLDA.db.DB instance at 0x42c6a52c at 1186514860 Traceback (most recent call last): File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 462, in _cleanup rm.tpc_abort(self) File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 548, in tpc_abort self.manager.tpc_abort(txn) File /usr/local/src/zope/Zope2.8/lib/python/Shared/DC/ZRDB/TM.py, line 64, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-11-14T06:20:40 INFO ZODB conflict error at /classes/class_applications/classaccess/topic_html (658 conflicts since startup at 2005-11-11T01:26:40) -- 2005-11-14T06:20:40 ERROR txn.4100 Failed to abort object: Surrogate oid=object object at 0x4009d470 Traceback (most recent call last): File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 562, in abort self.manager.abort(o, txn) File /usr/local/src/zope/Zope2.8/lib/python/Shared/DC/ZRDB/TM.py, line 64, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-11-14T06:20:40 ERROR txn.4100 Failed to abort resource manager: MultiObjectResourceAdapter for Products.ZMySQLDA.db.DB instance at 0x42c6a52c at 1186514860 Traceback (most recent call last): File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 489, in abort rm.abort(self) File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 562, in abort self.manager.abort(o, txn) File /usr/local/src/zope/Zope2.8/lib/python/Shared/DC/ZRDB/TM.py, line 64, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock which is a pattern typical of the many such entries: a log entry for an abort which raises a lock error, a report of a conflict error, a report of a failure to abort a Surrogate oid, followed by a second abort of the first instance. We also see frequent KeyError failures in the session variable data structure which suggests that the copy-out/modify/copy-in mechanism used to tickle the persistence mechanism does not run to completion and so the Zope persistence fail and the session variable structure is pruned or removed at some random later time. Some diagnostics are seen which report that the REQUEST['SESSION'] is occasionally removed. Questions: ~~ What is the cause of the locking problem? Does ZMySQLDA/MySQL-Python/Zope have some path which should have called begin to get a lock and did not? What needs to be done to make session variables stable? Does the Temporary Storage method need to be extended to allow for rolling back a transaction? (What's a good approch if this is necessary? Is there any existing product supporting this?) Any other help or advice would be appreciated.
[Zope] conflicts, sessions, and programming best practices guidelines
Has anyone prepared a set of best practice guidelines on the techniques to use to minimize conflicts? It is becoming clear that we need to do to refactor some of our code to get us out of our current conflict pickle. A quick google produced lots of commentary but nothing I could hand to our programmers as guidelines. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] conflicts, sessions, and programming best practices guidelines
Peter, That's pretty much what we've done, but it is not really enough in our case. Our sites are interactive with lots of per user state. We make fairly heavy use of session variables to track the state. Our number of simultaneous users is also fairly high. In addition, during development, we structured things for ease of development rather than for mimimum liklihood of conflict errors. At this point we are seeing many conflicts and interactions with both the sessioning mechanism and the persistence mechanism--or so it appears. Even when we resolve the current database interaction problem, a refactoring seems in order to ensure the number of conflicts is kept to a minimum. On Tue, 15 Nov 2005, Peter Bengtsson wrote: I don't know of any documentation but (correct me if I'm wrong) but - install zope 2.8.0 or something higher that uses zodb 3.4 - use zeo's and load balancers so that certain requests don't clog your pipes On 11/15/05, Dennis Allison [EMAIL PROTECTED] wrote: Has anyone prepared a set of best practice guidelines on the techniques to use to minimize conflicts? It is becoming clear that we need to do to refactor some of our code to get us out of our current conflict pickle. A quick google produced lots of commentary but nothing I could hand to our programmers as guidelines. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] HTML to PDF.
We use htmldoc. It's a bit old and a bit grotty, but it does the job. On Wed, 16 Nov 2005, Fernando Lujan wrote: Does Anyone has this product in a production environment? It's easy to configure and stuff? We currently are working with the reportlab, but it's painful to change the document... :( I need a product which generates a PDF file on the fly. Any other application or suggestion will be welcome. Thanks, Fernando Lujan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] conflicts, sessions, and programming best practices guidelines
Chris, I am aware that using ZEO to back session database is likely to increase the opportunity for conflicts, but using a single session database seems to be reaquired if you want, as we do, to distribute out interactive application acrosss a cluster of processors cleanly. Under our current approach, we have multiple Zopes running on multiple machines, all referencing a shared database of session variables. This allows a HTTP request to be blindly distributed to any of the machines. Given the conflict problems, we may need to rethink this simplistic architecture. On Tue, 15 Nov 2005, Chris McDonough wrote: On Nov 15, 2005, at 5:54 PM, Dennis Allison wrote: Has anyone prepared a set of best practice guidelines on the techniques to use to minimize conflicts? It is becoming clear that we need to do to refactor some of our code to get us out of our current conflict pickle. A quick google produced lots of commentary but nothing I could hand to our programmers as guidelines. Nothing in general except the (probably too general) attempt not to allow one process/thread to write to the same object at the same time as another process/thread. One tip, since you've told us you use sessions heavily: using ZEO to back your session database provides more opportunity for conflicts than if you used a normal Zope storage. This is because: - the opportunity for conflict naturally grows as transaction time increases and using a ZEO storage rather than a local storage adds latency to the total transaction time that would not otherwise exist. - you have n processes attempting to write to essentially the same objects at the same time when you use a shared ZEO sessioning database. If these were instead local databases, each process would only compete with itself (between all of its threads). I tend to try to use local sessioning databases backed by tempstorage plus a frontend load balancer that allows for cookie-based session affinity (Pound). - C -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] conflicts, sessions, and programming best practices guidelines
Thank Chris, good pointsB. The session variable timeout is currently several hours as requird by our application. The timeout resolution remains at its default value. I have not dorked with disabling the inband housekeeping. I don't see how it would impact on our problem. Am I missing something? I suppose I could do a custom p_resolveConflict since the vast majority of conflicts are are non-conflicting and actual conflicts are fairly easy to identify. Still, on the whole, I think we have to refactor our programs to minimize the opportunities for conflicts as well as tuning the mechanism. As a temporary workaround, I'm backing out of the shared temporary storage in favor of a local temp storage. That should make things a bit better. On Wed, 16 Nov 2005, Chris McDonough wrote: Changing the architecture will likely get you the most bang but note also that there are a few knobs that you can turn on the transient object container UI that may limit the number of conflicts (in the temp_folder/session_data_container): - Data object timeout value -- the timeout value in minutes.. make this higher. - Timeout resolution -- the timeout resolution in seconds.. make this higher. You can also turn off inband housekeeping of session data by calling the method disableInbandHousekeeping on a transient object container. At that point, sessions will continue to work properly but old session data won't be garbage collected and you will need to do this every so often by calling the housekeep method on TOC. See this written by dunny about conflicts and sessions (although this was written before MVCC): http://www.plope.com/Members/dunny/conflicts/view HTH, - C On Nov 16, 2005, at 6:18 PM, Dennis Allison wrote: Chris, I am aware that using ZEO to back session database is likely to increase the opportunity for conflicts, but using a single session database seems to be reaquired if you want, as we do, to distribute out interactive application acrosss a cluster of processors cleanly. Under our current approach, we have multiple Zopes running on multiple machines, all referencing a shared database of session variables. This allows a HTTP request to be blindly distributed to any of the machines. Given the conflict problems, we may need to rethink this simplistic architecture. On Tue, 15 Nov 2005, Chris McDonough wrote: On Nov 15, 2005, at 5:54 PM, Dennis Allison wrote: Has anyone prepared a set of best practice guidelines on the techniques to use to minimize conflicts? It is becoming clear that we need to do to refactor some of our code to get us out of our current conflict pickle. A quick google produced lots of commentary but nothing I could hand to our programmers as guidelines. Nothing in general except the (probably too general) attempt not to allow one process/thread to write to the same object at the same time as another process/thread. One tip, since you've told us you use sessions heavily: using ZEO to back your session database provides more opportunity for conflicts than if you used a normal Zope storage. This is because: - the opportunity for conflict naturally grows as transaction time increases and using a ZEO storage rather than a local storage adds latency to the total transaction time that would not otherwise exist. - you have n processes attempting to write to essentially the same objects at the same time when you use a shared ZEO sessioning database. If these were instead local databases, each process would only compete with itself (between all of its threads). I tend to try to use local sessioning databases backed by tempstorage plus a frontend load balancer that allows for cookie-based session affinity (Pound). - C -- -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope 2.8.4 external methods do not autorefresh
test framework: dtml-var standard_html_header dtml-var expr=eproc() dtml-var standard_html_footer Setup an external method def eproc( self ): return 'hello world' and run the test method. It prints hello world Now, change the test method def eproc( self ): return 'hello world too' and run the test method. It prints hello world which is a deviation from the usual Zope2 behavior. To adopt the code for the modified method the external method needs to be resaved. Is this a bug or a feature? -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope 2.8.4 external methods do not autorefresh
blush Yup. or so it seems. Thanks. Stupid errors confume the day... On Fri, 18 Nov 2005, Paul Winkler wrote: On Fri, Nov 18, 2005 at 11:25:06AM -0800, Dennis Allison wrote: To adopt the code for the modified method the external method needs to be resaved. *Or* you need to have debug-mode on ... in your zope.conf. It's always been that way. I'm guessing you didn't turn on debug-mode for your 2.8 instance? -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] conflicts, sessions, and programming best practices guidelines
Thanks Dieter. On Fri, 18 Nov 2005, Dieter Maurer wrote: Dennis Allison wrote at 2005-11-15 14:54 -0800: Has anyone prepared a set of best practice guidelines on the techniques to use to minimize conflicts? * Localize out into separate persistent objects attributes with high write frequency. E.g. when you have a counter, put into its own persistent object (you can use a BTrees.Length.Length object for a counter). * Implement conflict resolution for your high frequently written persistent objects. Formerly, TemporaryStorage had only very limited history information to support conflict resolution (which limited the wholesome effect of conflict resolution). Rumours say that this improved with Zope 2.8. * Write only when you really change something. E.g. instead of session[XXX] = sss use if session[XXX] != sss: session[XXX] = sss (at least, if there is a high chance that session already contains the correct value). -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: session variables in the presence of conflicts
Florent, There were, of course, tracebacks. Any assist you can provide would be appreciated. -D -- 2005-11-18T12:50:16 ERROR txn.3075 Error in tpc_abort() on manager MultiObjectResourceAdapter for Products.ZMySQLDA.db.DB instance at 0x450431cc at 1190763820 Traceback (most recent call last): File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 462, in _cleanup rm.tpc_abort(self) File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 548, in tpc_abort self.manager.tpc_abort(txn) File /usr/local/src/zope/Zope2.8/lib/python/Shared/DC/ZRDB/TM.py, line 64, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-11-18T12:50:16 ERROR txn.3075 Error in tpc_abort() on manager MultiObjectResourceAdapter for Products.ZMySQLDA.db.DB instance at 0x41487acc at 1190774252 Traceback (most recent call last): File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 462, in _cleanup rm.tpc_abort(self) File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 548, in tpc_abort self.manager.tpc_abort(txn) File /usr/local/src/zope/Zope2.8/lib/python/Shared/DC/ZRDB/TM.py, line 64, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-11-18T12:50:16 INFO ZODB conflict error at /courses/topics/navigation_box (55 conflicts since startup at 2005-11-18T12:13:38) -- 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort object: Surrogate oid=object object at 0x4009d470 Traceback (most recent call last): File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 562, in abort self.manager.abort(o, txn) File /usr/local/src/zope/Zope2.8/lib/python/Shared/DC/ZRDB/TM.py, line 64, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort resource manager: MultiObjectResourceAdapter for Products.ZMySQLDA.db.DB instance at 0x450431cc at 1190763820 Traceback (most recent call last): File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 489, in abort rm.abort(self) File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 562, in abort self.manager.abort(o, txn) File /usr/local/src/zope/Zope2.8/lib/python/Shared/DC/ZRDB/TM.py, line 64, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort object: Surrogate oid=object object at 0x4009d470 Traceback (most recent call last): File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 562, in abort self.manager.abort(o, txn) File /usr/local/src/zope/Zope2.8/lib/python/Shared/DC/ZRDB/TM.py, line 64, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort resource manager: MultiObjectResourceAdapter for Products.ZMySQLDA.db.DB instance at 0x41487acc at 1190774252 Traceback (most recent call last): File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 489, in abort rm.abort(self) File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 562, in abort self.manager.abort(o, txn) File /usr/local/src/zope/Zope2.8/lib/python/Shared/DC/ZRDB/TM.py, line 64, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock -- On Sat, 19 Nov 2005, Florent Guillaume wrote: Dennis Allison wrote: We are using MySQL but are fully transactional using innodb. The sort of problems we are seeing are (cruft removed) are things like: 2005-11-18T12:50:16 ERROR txn.3075 Error in tpc_abort() on manager MultiObjectResourceAdapter for Products.ZMySQLDA.db.DB instance at 0x450431cc at 1190763820 There should be a traceback here, or if there's none you should patch _transaction.py to make it produced. Seeing that traceback is important and would point to an error somewhere in MySQLDA's code I expect. 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort resource manager: MultiObjectResourceAdapter for Products.ZMySQLDA.db.DB instance at 0x450431cc at 1190763820 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort object: Surrogate oid=object object at 0x4009d470 2005-11-18T12:50:16 ERROR txn.3075 Failed to abort resource manager: MultiObjectResourceAdapter
Re: [Zope] Re: session variables in the presence of conflicts
That was my conclusion long ago, but I have not been able to track it down. It only occurs under heavy load when backing out of a conflict error. I have not been successful at creating a test set that triggers it. I suppose it is time to again read the ZMySQLDA code... There are a few comments on the net about problems with MySQLDA which may also be related. On Sun, 20 Nov 2005, Florent Guillaume wrote: Dennis Allison wrote: Florent, There were, of course, tracebacks. Any assist you can provide would be appreciated. -D -- 2005-11-18T12:50:16 ERROR txn.3075 Error in tpc_abort() on manager MultiObjectResourceAdapter for Products.ZMySQLDA.db.DB instance at 0x450431cc at 1190763820 Traceback (most recent call last): File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 462, in _cleanup rm.tpc_abort(self) File /usr/local/src/zope/Zope2.8/lib/python/transaction/_transaction.py, line 548, in tpc_abort self.manager.tpc_abort(txn) File /usr/local/src/zope/Zope2.8/lib/python/Shared/DC/ZRDB/TM.py, line 64, in abort try: self._abort() File /opt/zope/zproducts/standard/ZMySQLDA/db.py, line 389, in _abort self._tlock.release() error: release unlocked lock Well undoubtedly the bug is in ZMySQLDA... Florent -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] minimizing conflict errors
I have a DTML method which provides the primary navigation control in a portion of our system and so is very heavily used. It is a primary source of conflict errors and so is being rethought. Zope 2.8.4, ZEO 3.4.2, ZODB 3.4.2, Python 2.4.2 or 2.3.5 MySQL 4.0.20, MySQL-Python 1.2.0, MYSQLDA 2.0.9 ZODB 3.4.2 does not raise a conflict error on a read-read conflict, one of the reasons fr moving to Zope 2.8.4. The structure of the naviagation method is simple enough. Everything is wrapped in a dtml-let which sets a number of parameters mostly by reading them from the SESSION (with an interface function) or plucking them from the relational database with a query. In the scope of the let is dtml code which, when rendered, provides the various navigation links. In various sections there are additional dtml-let blocks and additional queries to the relational database and several dtml-in loops. Looking at the code, I don't understand why I am seeing conflicts. As I understand things, neither variables in the dtml-let space nor the REQUEST/RESPONSE space are stored in the ZODB so modifications to them don't look like writes to the conflict mechanism. Am I incorrect in my understanding? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] DateTime mess
If I have a vote, +1 on deprecation and replacement with something less buggy. We have spent enormous amounts of time trying to sort out and then work around DateTime problems. On the other hand, we have a substantial amount of code that uses DateTime and finally, mostlyt works. We'd be unhappy if we had to rewrite everything--so a long deprecation period makes sense. Timezones, BTW are just the tip of the iceberg. Datetime tries to be a it just works solution to an inherently ambiguous problem and incorporates many surprises. On Fri, 25 Nov 2005, Chris Withers wrote: Peter Bengtsson wrote: -1 on making DateTime deprecated. I'm sure thousands of zope people use DateTime in thousands of places. Technically it might be correct deprecate it but politically it's just going to put people off from upgrading. I'm sure it would be fairly trivial to write an unpgrader, but really, deprecating it, even if the deprecation period is fairly long, is the only sane way to tackle this... cheers, Chris -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] DateTime mess
True. But the context of this thread was whether deprecation made sense or not. I was agreeing with Chris that the DateTime problems are better solved by replacement than by patching, but also that there is a large amount of code that has been made to work using the current DataTime and that investment needs to be protected. On Fri, 25 Nov 2005, Andreas Jung wrote: --On 25. November 2005 12:22:05 -0800 Dennis Allison [EMAIL PROTECTED] wrote: If I have a vote, +1 on deprecation and replacement with something less buggy. You can only deprecate when you have something better. As long as there is no useful replacemenet there is nothing to deprecate :-) -aj -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] (no subject)
-- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope 2.8.4 strange behavior
Python2.4.2 Zope 2.8.4 ZODB/Zeo 2.3.4 Mysql 4.0 Dual Athalon processors Linux RH7.3 We have two recent instances in our production sites where Zope suddenly stops responding. It is not a new problem, but we've now been confronted with two clean examples and nothing to blame them on. The problem appears to be independent of load as both incidents were on lightly loaded machines. A check of the logs (Linux and Zope) shows nothing obviously amiss except that the trace log (the old -M log) shows a sudden increase in active requests from the typical 0 or 1 to 1300 or more. In this context an active request' is total number of requests pending at the end of this request and is computed by post-processing. We front-end Zope with pound and make heavy use of MySQL. Both show a plethora of incomplete transactions. Examination of the raw trace log shows that Zope is continuing to accept requests, but nothing getting done. The raw log date-stamps four internal states for each transaction. The states are Begin (B), Input (I), action (A), and End (E). Inputs are gathered between B and I, outputs is made between A and E. The raw log shows B and I transactions, but apparently no processing is completing. I suspect that nothing is getting scheduled. I am at a loss as to where to begin to track this one down. The failure is spontaneous and apparently not triggered by any readily distinguishable inputs or pattern of inputs. The behavior smells a bit of resource limits or process synchronization problems, but there is not real evidence for either being the root cause. I am not sure what monitoring I should be doing to help locate the source of the problem. Has anyone seen seen a similar problem? Any advice as to how to proceed? -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: Zope 2.8.4 strange behavior
Just went throught that exercise with Andy and installed a patch to MySQLDA that effectively ignores the 'release unlocked lock' problem that has been plaguing us. I shoulda guessed that is the first place to look. I'll get and install the DeadlockDebugger forthwith. Thanks. On Mon, 28 Nov 2005, Florent Guillaume wrote: Dennis Allison wrote: We have two recent instances in our production sites where Zope suddenly stops responding. It is not a new problem, but we've now been confronted with two clean examples and nothing to blame them on. The problem appears to be independent of load as both incidents were on lightly loaded machines. A check of the logs (Linux and Zope) shows nothing obviously amiss except that the trace log (the old -M log) shows a sudden increase in active requests from the typical 0 or 1 to 1300 or more. In this context an active request' is total number of requests pending at the end of this request and is computed by post-processing. We front-end Zope with pound and make heavy use of MySQL. Both show a plethora of incomplete transactions. Examination of the raw trace log shows that Zope is continuing to accept requests, but nothing getting done. The raw log date-stamps four internal states for each transaction. The states are Begin (B), Input (I), action (A), and End (E). Inputs are gathered between B and I, outputs is made between A and E. The raw log shows B and I transactions, but apparently no processing is completing. I suspect that nothing is getting scheduled. I am at a loss as to where to begin to track this one down. The failure is spontaneous and apparently not triggered by any readily distinguishable inputs or pattern of inputs. The behavior smells a bit of resource limits or process synchronization problems, but there is not real evidence for either being the root cause. I am not sure what monitoring I should be doing to help locate the source of the problem. Has anyone seen seen a similar problem? Any advice as to how to proceed? Threads are hanging. You should install my DeadlockDebugger and track down where the hung threads are blocked at. From the description I'd wager that you'll find your threads stuck in a corner of the MySQL DA. In which case you'd have to find why it deadlocks and find a fix. Florent -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope 2.8.4 strange behavior
Chris, A helpful suggestion. The commit errors I've been seeing have to do with the intereaction of the ZODB, MySQL, session variables, and conflicts. These particular problems do not appear to be related to the ZODB/session variable/conflict issues, but I cannot completely exclude them since leaks easily cause failures far away from where the fault lies. It certainly is a worthwhile thing to check and monitor. -d On Sun, 27 Nov 2005, Chris McDonough wrote: On Nov 27, 2005, at 8:13 PM, Dennis Allison wrote: here to begin to track this one down. The failure is spontaneous and apparently not triggered by any readily distinguishable inputs or pattern of inputs. The behavior smells a bit of resource limits or process synchronization problems, but there is not real evidence for either being the root cause. I am not sure what monitoring I should be doing to help locate the source of the problem. Has anyone seen seen a similar problem? Any advice as to how to proceed? A pool of database connections is kept around to pull from for each ZODB database you have. By default, there are 7 of them per database; this number can be changed by the pool-size parameter in the zodb_db section in your config file. These connections are returned to the pool when a connection is closed. This usually happens when a transaction that is using that connection completes. The symptom you're describing has happened to me in the past. Zope stops serving requests and my CPU is not pegged (it's not hitting the CPU hard via an infinept loop I've made). I usually end up tracking it down to the fact that my instance is somehow leaking database connections. The connection is never closed so it never gets returned to the pool. This can happen when there is an error during a transaction commit or abort. Since you've previously posted to the list with commit errors of just this kind, it seems like that would be the logical place to start. You didn't mention seeing any of these errors, though, so if you don't see any of those in the event log, I'm not sure where to go from there other than debugging by binary exclusion which can be impossible when the failures happen so far apart. :-( - C -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope 2.8.4 strange behavior
Thanks Paul. I checked and did not find any Client disconnected entries in the event log so this may not be the cause of what I am seeing. I will bump the log level to ALL (which I think includes BLATHER) when I restart things tonight. We use a remote ZEO but at the moment Zope and ZEO share the same machine. On Sun, 27 Nov 2005, Paul Winkler wrote: On Sun, Nov 27, 2005 at 05:13:21PM -0800, Dennis Allison wrote: Examination of the raw trace log shows that Zope is continuing to accept requests, but nothing getting done. The raw log date-stamps four internal states for each transaction. The states are Begin (B), Input (I), action (A), and End (E). Inputs are gathered between B and I, outputs is made between A and E. The raw log shows B and I transactions, but apparently no processing is completing. I suspect that nothing is getting scheduled. I've seen the same symptoms a number of times recently with zope 2.7.x. In our case, it seems to be related to ZEO. Zope seems to have lost its connection to ZEO but doesn't realize it somehow. My theory is that the symptom starts when all worker threads are waiting for objects that aren't in the ZEO client cache, so they're all waiting on ZEO requests. Meanwhile, requests keep piling up in the queue. These are all the B and I lines you noticed. Eventually (10 minutes or so) one of these ZEO requests times out, and Zope then realizes it's lost its ZEO connection and successfully reconnects. This shows up in the Zope event log as a series of ZEO Client disconnected errors, and immediately thereafter we see a successful reconnect in the ZEO server's log. I have no idea what is causing the loss of connection in the first place, or why it takes Zope so long to realize there's a problem. Nothing in the logs so far gives me any clue. I've bumped up our log levels to BLATHER and hopefully next time this happens I'll get some more to work with and see if there's anythign I can do about it. It may be network issues between our Zope and Zeo boxes, which are on different subnets with a firewall between them, over which I have no control. -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope 2.8.4 strange behavior
We had been plagued with a threading error which manifests itself
as an exception reporting release unreleased lock. At Andy's
suggestion I added code to catch the exception in the appropriate
place in ZMySQLDA --
Here's the code from db.py ---
The code probably should be specific to the unreleased lock exception
since, presumably, the code could throw other exceptions. On the whole,
I think this is a ban-aid solution.
def _abort(self, *ignored):
try:
if self._mysql_lock:
self.db.query(SELECT RELEASE_LOCK('%s') %
self._mysql_lock)
self.db.store_result()
if self._transactions:
self.db.query(ROLLBACK)
self.db.store_result()
else:
LOG('ZMySQLDA', ERROR, aborting when non-transactional)
finally:
# just ignore the exception for release unlocked lock
try:
self._tlock.release()
except:
pass
On Sun, 27 Nov 2005, Chris McDonough wrote:
Chris,
A helpful suggestion. The commit errors I've been seeing have to
do with the intereaction of the ZODB, MySQL, session variables, and
conflicts.
So the patch that Andy sent over is a fix that prevents the mysql
adapter from raising an error when a conflict exception occurs? Do
you know if that error only happens after a conflict exception occurs
or can it happen without the presence of conflict exceptions? (I'm
curious because I also use the adapter and I'd like to know what the
patch fixes).
These particular problems do not appear to be related to the ZODB/
session
variable/conflict issues, but I cannot completely exclude them
since leaks
easily cause failures far away from where the fault lies.
I'm afraid I can't parse that sentence fully. But I'll try to
interpret as best possible. ;-)
I think I've said this before but it in case not... the use of
sessions is only one place where conflict errors can be generated.
Conflict errors are normal in any system that causes writes to a
ZODB database. If your application does any writes to a ZODB
database at all (besides the writes that occur from use of the
sessioning machinery), and the mysql adapter wasn't tolerant of
conflict errors, you'd be getting the same result, they'd just
probably happen further apart.
That said, I certainly am interested in making fewer conflicts errors
go unresolved when the sessioning machinery is used. If you were to
tell that you've taken the steps I've already suggested about
reducing the potential for conflicts during session usage (use 2.8
with mvcc, turn external housekeeping on, bump up the resolution
time, local zodb db for sessions), and you observed that you're still
having too many conflicts, I'd try to take some action, although to
do so I might still need to request your help in providing data about
your conflict rates.
It certainly is a worthwhile thing to check and monitor.
What is?
Thanks,
- C
--
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope 2.8.4 strange behavior
We do see occasional instances where things slow down, a backlog builds, and then it clears. But no disconnects. On Sun, 27 Nov 2005, Paul Winkler wrote: On Sun, Nov 27, 2005 at 08:08:20PM -0800, Dennis Allison wrote: Thanks Paul. I checked and did not find any Client disconnected entries in the event log so this may not be the cause of what I am seeing. OK. If you were having my problem you'd see them for sure when the problem stops (for me it always fixes itself eventually). I will bump the log level to ALL (which I think includes BLATHER) Yep. when I restart things tonight. We use a remote ZEO but at the moment Zope and ZEO share the same machine. You're reasonably unlikely to have my problem then :-) -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope 2.8.4 strange behavior
Resolved. Thanks to all that responded. The symptoms were due to an external method which runs a program (htmldoc) that converts html to pdf. The external method uses the old subprocess interface to fire off the process and then wait() for it to complete. To fix another problem, we had upgraded to the latest version, but that htmldoc version apparently is confused by some of our html input and never completes so the external method just sits there waiting for the child process to terminate. Multiple invokations of the blocked htmldoc process eventually brought Zope to its knees and no further progress was made. Confession is good for the soul The problem took a long time to find because failures were infrequent (we only saw three or for instances) so there was limited data. Moreover,the external method the was failing had been in service for almost two years and has not been a problem. And, other very similar external methods which also use htmldoc were known to work with the upgraded htmldoc. It was not until the external method was identified as the only commonality in the failures that I thought to look closely at the external method. And then I noticed the blocked htmldoc processes, etc. etc. The fix Our solution was to revert htmldoc to the earlier version. When time permits we'll add a timeout to the external method and try to extablish what in the current release htmldoc triggered the failure. We probably should move to the new subprocess module rather than use popen2 in the external method. On Sun, 27 Nov 2005, Dennis Allison wrote: Python2.4.2 Zope 2.8.4 ZODB/Zeo 2.3.4 Mysql 4.0 Dual Athalon processors Linux RH7.3 We have two recent instances in our production sites where Zope suddenly stops responding. It is not a new problem, but we've now been confronted with two clean examples and nothing to blame them on. The problem appears to be independent of load as both incidents were on lightly loaded machines. A check of the logs (Linux and Zope) shows nothing obviously amiss except that the trace log (the old -M log) shows a sudden increase in active requests from the typical 0 or 1 to 1300 or more. In this context an active request' is total number of requests pending at the end of this request and is computed by post-processing. We front-end Zope with pound and make heavy use of MySQL. Both show a plethora of incomplete transactions. Examination of the raw trace log shows that Zope is continuing to accept requests, but nothing getting done. The raw log date-stamps four internal states for each transaction. The states are Begin (B), Input (I), action (A), and End (E). Inputs are gathered between B and I, outputs is made between A and E. The raw log shows B and I transactions, but apparently no processing is completing. I suspect that nothing is getting scheduled. I am at a loss as to where to begin to track this one down. The failure is spontaneous and apparently not triggered by any readily distinguishable inputs or pattern of inputs. The behavior smells a bit of resource limits or process synchronization problems, but there is not real evidence for either being the root cause. I am not sure what monitoring I should be doing to help locate the source of the problem. Has anyone seen seen a similar problem? Any advice as to how to proceed? -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] MySql problems
I have not yet completed differential diagnosis, but I was hoping someone
had encountered a similar problem and could pass along a fix/workaround.
The following program fails to do the writes --
import MySQLdb
initdb='test'
dbuser = 'root'
dbpass = 'XX'
host = 'localhost'
connection = MySQLdb.connect( db=initdb, user=dbuser, passwd=dbpass,
host=host )
cursor = connection.cursor()
qd = delete from test.data
cursor.execute(qd)
qi = insert into test.data values('ardvark', 'homework')
print qi
cursor.execute( qi)
q2 = select * from test.data
cursor.execute( q2)
res = cursor.fetchall()
print res
when run with Mysql-Python-1.2.1c under Python 2.4.2, but works just
find with Mysql-Python-1.1.1 under Python 2.3.5.
The tables are Innodb tables (that is, transactional).
Running Zope with Python 2.4.2 and Mysql-Pyton 1.2.1c works just fine.
Is there something special I need to do that I have forgotten?
--
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] killing off individual Zope session
We just had an example of a run away process with one of our authenticated users. I would have liked to kill off the user's processing while leaving all the other users running. Any ideas as to how to accomplish this? I do know the user's login and password, but little else. We use the Cookie Crumbler and a slightly hacked UserFolder for authentication. -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Finding a match in a large dataset - btrees?
I cannot make any sense out of your example data. Since you already have the data in a MySQL table, I would think the best way to find the match would be to make a MySQL query. On Mon, 5 Dec 2005, Cameron Beattie wrote: I have a large set of data (that will be stored in MySQL) that I wish to match to and am wondering what the best method is. Assume the following data in table LOCATION_MATCH: LOCATION_IDLOCATION_PATTERNPARENT_ID 106 11410 13211 12911 14113 15213 The string 6438 should return 11, 6421 14, 6422 15 and 6499 12. I've read a bit about btrees on the zope wiki and wonder if that's the best way. However I am struggling with the basics: 1. How do I get the data from MySQL into a btree in Zope? Something like: from BTrees.IIBTree import * t = IIBTree() t.update(context.select_from_LOCATION_MATCH) # errr, no 2. How do I find the matching node i.e. when I want to know that 6422 relates to location_id 15? Any help or pointers to further documentation would be appreciated. Regards Cameron ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] resolving conflict errors
On Fri, 14 Oct 2005, Chris McDonough wrote: Dennis Allison asked: What data does Zope store in the ZODB when an object is evaluated? None that you don't tell it to. Typically conflict errors are a result of two threads calling code which changes the same object at the same time, but nothing that Zope does under the hood causes it; it is always caused by application code. One exception to this rule is conflict errors raised when using Zope sessions. It's not actually an exception to the rule, but programmers are shielded from the fact that sessions store data in ZODB when you use the session API (e.g. REQUEST.SESSION). The sessioning machinery needs to manage housekeeping info whenever the API is used to expire old sessions and create new ones, so although it may not look like you are writing to the ZODB when you use sessions (even to read data out of them), you potentially are. Zope 2.8 has a ZODB that support multiversion concurrency control, which eliminates a certain class of conflict errors (read conflict errors), so if you are getting a lot of these, and you can get away with using 2.8, I'd suggest doing so. The problem I am trying to resolve appears to be load related. The observed symptom is that (some) session variables spontaneously disappear. There appears to be some connection to conflicts, but the exact mechanism and the relationship is not yet clear. BTW, when I first began trying to resolve this problem, we were running Zope 2.7.6. We moved to Zope 2.8,4 to take advantage of the later ZODB. That move was a good one, but it has cause some migration pain. The session problem has been with us forever, at least since Zope 2.5.0. They are Heisenbugs and appear apparently randomly. Detailed logs should help some, but so far have not brought joy. So, I've been looking through the code trying to find places where some infrequent event could cause the problem. Chris pointed out that session variables can cause conflict errors (both read-read and read-write) when the session API is used. I've been trying to explore that interface and have not yet found all the pieces. Some hints to the reader would help. I see where HTTPRequest manages the Zope user interface to the session variables which includes a mechanism for lazy access--the dictionary _lazies provides a list of callables. When a variable is accessed, it is promoted to Request by executing the callable and storing the value. The _lazies entry corresponding to the session variable is deleted. What I have not been able to find is where this is maintained in a persistent fashion. Can someone provide a pointer? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] resolving conflict errors
Good idea, but it is hard to do in a production environment with a never lose data model. I have suspected the p_resolve_conflict which is clearly wrong for our model and am in the process of trying to rewrite it to take advantage of the semantics of sessions as we use them. The problem I am tracking manifests itself as KeyErrors in the session data structure. The session structure is used pretty much out of the box. Reading is done any which way (e.g, REQUEST['SESSION'][key]) but writing always uses a copy out of the session object, an update of the session object, and the a replacement of the session object back into REQUEST. The session data we lose are generally strings, for example, a user_id. For example, we can set the user_id into the session to a value, and then later, when we reference the session variable, we get a KeyError. While there is no direct causal tie, we suspect this is related to an intervening conflict error. Occasionally the entire SESSION data container disappears. At other times, we get a KeyError exceptions for one or more session variables. I have been trying to understand in detail the management of the session variables. I can see how accesses are managed in ZPublisher/HTTPRequest, but I am still unsure of how that session data is maintained persistent and how session data can generate read-read conflicts. I'll try your suggestion. I also plan to monitor session variable access to determine whether the SESSION data whether a KeyError signals that all session variables are missing or whether only a few are missing. On Sun, 11 Dec 2005, Michael Dunstan wrote: On 12/9/05, Dennis Allison [EMAIL PROTECTED] wrote: The problem I am trying to resolve appears to be load related. The observed symptom is that (some) session variables spontaneously disappear. There appears to be some connection to conflicts, but the exact mechanism and the relationship is not yet clear. A small possibility is that you are being bitten by the DWIM'ly nature of TransientObjects conflict resolution where the last modified state is chosen over the others. If you think this is biting you then try commenting out _p_resolveConflict of TransientObject. That's bound to increase the rate of conflict errors but should provide you with a consistent session state. Perhaps useful as a debugging step. michael -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] resolving conflict errors
Thanks, I'll take a look. I don't have much faith in getting to do it with the live system, but maybe I can find a way to get some sort of testbed. On Sun, 11 Dec 2005, Michael Dunstan wrote: On 12/11/05, Dennis Allison [EMAIL PROTECTED] wrote: Good idea, but it is hard to do in a production environment with a never lose data model. Have a go at recreating the problems you are seeing on a development host. SessionRig can be used to mount a brute force attack of the session machinery. You'll need to tune that somewhat to your particular application. http://cvs.zope.org/Packages/SessionRig michael -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] sessions in the presence of conflicts
Zope 2.8.4, ZODB 3.4.2 I am still trying to resolve a session variables in the presence of conflicts problem. The observed symptom is that suddenly either some session variables disappear or all session variables disappear. In both cases, Zope has raised a KeyError exception for the a particular session variable because it is not present and an attempt was being made to access it from the session. The cause appears to be related to the presence of a conflict error and a botched recovery, but the exact mechanism remains a mystery. Zope equates a session object with a TransientObject. For our instance, that's fine, since our only TransientObject is a session. For the general case, I would have thought a SessionObject inheriting from TransiendObject would have been used so that conflict resolution for sessions could be easily specialized. The conflict resolution for a TransientObject (that is, a Session Object) is clearly suboptimal. In the next paragraphs I'll review what is currently done and then propose a different approach which I am about to implement for our systems. Comments, corrections, and suggestions much appreciated. _p_resolveConflict(self, oldState, savedState, newState) returns the state of the object after resolving different changes. The arguments are: oldState -- state of the object at the beginning of the current transaction (mutable) savedState -- state currently stored in the database. This state was written after oldState and reflect changes made by a transaction that committed before the current transaction (immutable) newState -- state after changes made by the current transaction (immutable) The standard conflict resolution for a TransientObject resolves according to the following rules: 1. if any of the states are invalid (that is, has a key '_invalid') return the invalid state. 2. if any any of the attributes ['token','id','_created'] differ then there is a conflict, raise the conflict exception. 3. choose the state most recently modified, if possible. 4. otherwise, choose the state most recently accessed. It seems to me that we can do much better for sessions because we know a bit about the semantics of sessions. A session object is a dictionary-like object mapping key-value pairs. Adding or deleting keys or changing the value associated with a key are independent operations and do not conflict unless the keys are duplicated in both the transactions. Any conflict resolution mechanism needs to be able to manage multiple keys independently since the session object is modified as a unit. In addition, new keys may be added and old keys deleted; any conflict resolution mechanism at the key level needs to be comprehend those operations. A more session-friendly conflict resolution might use: 1. if any of the states are invalid (that is, has a key '_invalid') return the invalid state. 2. if any any of the states attributes ['token','id','_created'] differ then there is a conflict, raise the conflict exception. 3. order the newState and savedState by modification time (or if that cannot be computed, by access time). 4. any key appearing in oldState's dictionary but not appearing in both savedState and newState should be removed from all. This corresponds to a key-value pair being deleted in one of the transactions. Insertions will be managed automatically by the updates. 5. beginning with the oldest, update oldState dictionary of key-value pairs using the dictionary part of newState and savedState. Return oldState. This does several things. First, it captures independent key-value changes made in both potentially conflicting transactions. Second, it provides a reasonable ordering for multiple (potentially conflicting) key-value pair updates. Third, it manages insertions and deletions to the session variable set in the presence of conflicts. Does this make sense? I have yet to figure out how to map a TransientObject state back to the object it represents, but it clearly is possible. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] sessions in the presence of conflicts
Hi Davids. Thanks for your note. I remember your post. No, I have not used HappySession although we looked at it when we started. When we started development, sessions had just become part of the standard release Zope and we decided that in the long run we'd be better off using the release product. As I've said in other posts, we use the session mechanism fairly heavily. We have O(200) simultaneous interactive users each of whom has 40 or so session variables which maintain the dynamic state of their instance. Nearly every substantive operation touches one or more session variables. Everything works most of the time, but the error frequency is high enough for it to be of concern. I'm a bit cautious about moving to another sessioning system on a sizable production system running 24x7. Dennis, Im only curious and this is no recommendation ... and I asked this on the list maybe a year ago (and got zero reponses) Have you given HappySession a try? Has anyone? I've never had a problem with it, but then its never been under much load and is no longer actively supported (least last time I checked). I converted back and forth between Zope SESSION and HappySession (for reasons of legacy code). Their interfaces have just few differences, e.g., HappySession['this'] returns None (if 'this' is not a key) and SESSION raises an exception. That type of thing. HappySession does not read/write to zodb (if I recall it stores session data in RAM (session) cookies). I just do not know if its otherwise up to your requirements. All best, David -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: Triggering the persistence mechanism
Yes, I understand that. This is a development and code organization issue. From the developer's point-of-view there is a special namespace called session variables. What I want to be able to do is have the coders call getSessionVariable(name) to get the current value of the session variable and setSessionVariable(name, value) to save the value. The type of the variable should be transparent to the programmer--that is, it could be a number, a string, a dictionary, or a list. My question restated: Is there a way to write a generic setSessionVariable()routine that handles the mutated values properly for arbitrary types. That is, does something like def setSessionVariable( name, value ): request = container.REQUEST session = request.session a = session.get(name,0) a = value session[name]=a manage persistence correctly if value is a dictionary or a list and mutated from the old value, whatever it may have been. Or is it necessary to specifically include an assignment for each mutated component of the dictionary or list? What happens when the value to be stored is a dictionary of dictionaries of lists and has been mutated? Is there an advantage to using the persistent Dictionary and List classes in the session rather than ordinary ones? Thanks again for your insight. On Thu, 15 Dec 2005, Chris McDonough wrote: There is nothing that needs to be done here. It's only if you *mutate* values stored in the session that you need to do explicit persistence triggering. I provided an example of doing this in my last email. - C On Dec 15, 2005, at 4:20 PM, Dennis Allison wrote: Chris McDonnough pointed out a problem with my setSessionVariable code which I am now fixing. I want a generic routine to set a session variable whether it is a simple variable, a list, or a dictionary. The question is one of triggering the persistence mechanism. If a dictionary or list is stored in a session variable, what is needed to ensure that the setSessionVariable() routine triggers the persistence mechanism? Is it adequate to simply read the session variable in the setSessionVariable() routine prior to writing it when the session variable is a dictionary or list? Or is something else needed? -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: [Zope-dev] Re: sessions in the presence of conflicts
Chris McDonough identified a persistence problem with the routine(s) that
manage sessions variables. (Thanks Chris) I have put the correction in
place which resolved some (but not all) of the problems.
There are still problems which are apparently due conflicts in accessing
the session variables. To minimize frequency of conflicts, I am rewriting
several routines using Dieter's rules of the thumb (Thanks Dieter).
One routine being modified is a Script(Python) that initializes a number
of session variables. I am collecting the session values in a dictionary
and then use update to set their value, for example:
s = {}
s['alpha'] = 'a'
s['beta'] = 'b'
request['SESSION'].update(s)
Is the persistence machinery smart enough to detect this as a change? I
suspect that it has to be flagged since the assignment won't be seen.
Usually this means setting the _p_changed=1 attribute, but it is not clear
to me where to set it in this particular context.
--
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
